To obtain satisfactory performance in characterizing optical freeform surfaces with local features, this paper proposes a model of a radial basis function with slope-based shape factor and distribution (RBF-SSD). Compared to previous RBF-slope models with only slope-based shape factors, the RBF-SSD model relates both shape factors and distribution with the surface slope, ensuring greater fitting ability can be achieved when fitting a surface with local features. Fitting experiments for two different surfaces demonstrated the fitting ability of the RBF-SSD model. An off-axis three-mirror system with 3  °    ×  3.6  °   field of view was designed as an example to show the optical design efficacy of our model.