Abstract: The process of vegetation restoration on loose deposit slopes is complex and challenging. Micro-topography, vegetation distribution, and soil characteristics are key factors influencing vegetation restoration. This study aims to further elucidate the effects of these factors on vegetation distribution and provide a scientific basis for the ecological restoration of loose deposit slopes. The loose deposit slopes of Huangyuan quarry were selected as the study area, and a vegetation survey and soil sample collection were conducted on naturally restored slopes. Statistical tests and Pearson correlation analysis were used to investigate the distribution characteristics of vegetation and interpret the environmental factors influencing its distribution. The results of the study show: (1) The loose deposit slopes exhibited a large number of vegetation patches and a high landscape shape index, while the shady and semi-shady slopes had relatively large patch area and the maximum patch index, with the largest patch area as 22.37 m² and the highest patch index as 11.57%. These results suggest that vegetation distribution was relatively fragmented and its growth was significantly influenced by the spatial heterogeneity of the environment. Compared to sunny slopes, shady and semi-shady slopes were more favorable for plant aggregation and growth. (2) Vegetation diversity differed significantly across slope aspects. The Simpson dominance index (C) was significantly higher on sunny slopes than on shady and semi-shady slopes (P < 0.01), whereas the Shannon-Wiener diversity index (H), Margalef richness index (D), and Pielou evenness index (E) showed the opposite trend. This indicates that the vegetation community on sunny slopes was still unstable, with a dominance of a few species, while the vegetation on shady and semi-shady slopes exhibited higher richness and more even distribution. (3) Loose deposit soils were relatively poor, with extremely low levels of organic matter and total nitrogen content, and very deficient levels of available phosphorus. Without artificial intervention, natural vegetation recovery is unlikely. The physical properties of the soil were significantly affected by surface soil cover. The average soil porosity was 29.92%, and the physical structure of the soil changed from large pores with poor water retention to small pores with poor air permeability. (4) Soil moisture and available phosphorus content were the primary physicochemical factors influencing vegetation diversity on loose deposits, while slope aspect was the main topographic factor driving vegetation change. It is crucial to create microhabitats suitable for plant growth through micro-topographic adjustments to promote vegetation growth on loose deposit slopes. Soil moisture content is the key factor for vegetation restoration and should be prioritized during terrain remodeling.