As high coupling efficiency and return loss are crucial in fiber-optic transmission systems, they have attracted widespread attention. This study proposes a ray-transfer matrix-based mathematical analysis method and experiment. As high coupling efficiency and return loss are crucial in fiber-optic transmission systems, they have attracted widespread attention. This study proposes a ray-transfer matrix-based mathematical analysis method and experimentally demonstrates a collimator based on a gradient-index lens with an angle polish. The propagation characteristics and coupling mechanisms of the collimators are introduced. A beam-steering technology based on a wedge prism and flat glass is proposed to improve the coupling efficiency of the Gaussian beam using collimators. The proposed method is validated via simulation and experiment. The results are significant in free-space optical communication, optical signal processing, and optical fiber connectors.••••Proposed a new analytical method for determining the length of collimators based on a Gradient-index lens with an angle polish.••Beam-steering technology make the system less sensitive to misalignment and fabrication tolerance.••Among the three deviations of the coupling mismatch, the angular deviation has the greatest influence.Coupling efficiencyReturn lossGradient-index lensBeam-steering technologyOptical fibers are considered an emerging technology in the industry, medical surgery, and military fields as they offer wider bandwidth, large capacity, simple structure, and good confidentiality, and do not require a spectrum license,,,,. Fiber coupling efficiency (FCE) and return loss (RL) are the key factors in fiber-optic sensing, optical coherence tomography, space laser communication, and fiber-to-chip couplers. As the core diameter of the common single-mode fiber (SMF) is small, it is difficult to achieve direct alignment of the two fibers in free space. Generally, a Gaussian beam is expanded and collimated by optical fiber collimators to achieve high coupling efficiency. Optical fiber collimators consist of optical fibers and lenses such as C-lenses, G-lenses, ball lenses, and spherical lenses. In addition to FCE, RL i. 2.1. Analysis of light transmission by ray-transfer matrixThe optical fiber collimator used for expanding the Gaussian beam is introduced as a GRIN lens with an angle polish. The end face of the SMF was polished at an angle to connect it to the GRIN lens. The GRIN lens connected to the SMF is illustrated in Fig. 1. Generally, there are two types of GRIN lenses—axial and radial. In this study, radial-type GRIN lenses are considered important parts of the fiber collimators. The GRIN lens has a parabolic-shaped refractive index profile, as shown in Fig. 2, where the refractive index is expressed as:(1)nr=n01-g2r22,#where n0 is the refractive index at the center of the GRIN lens, g is the gradient constant, and r is the distance to the central axis.Fig. 1.