Abstract: The revegetation of iron mine tailing dams is currently an active research topic in ecological restoration globally. Studying the characteristics of herbaceous plant communities under different revegetation types and screening for herbaceous plant species suitable for the ecological restoration of tailing dams are of great significance for improving the ecological restoration effect of revegetation. The characteristics of herbaceous plant communities under different revegetation types on a reclaimed tailing dam of Jianshan iron mine in Shanxi, China, were studied using two-way indicator species analysis (TWINSPAN) classification, canonical correspondence analysis (CCA) ordination, M. Godron stability analysis, species diversity index, and interspecific relationships. The TWINSPAN classification was used to divide 96 quadrats of herbaceous plant communities under different revegetation types in the study area into 11 associations, and the characteristics of each association varied significantly due to different restoration years and revegetation combinations. With increasing restoration years, the life forms of herbaceous plant species under different revegetation types gradually changed from xerophyte to mesophyte, and the number of perennial plant species increased significantly. The CCA ordination results show the gradient change in grazing disturbance to different herbaceous plant communities and the soil moisture content with increasing restoration years. That is, the degree of grazing disturbance to the communities gradually decreased, while the soil moisture content gradually improved from left to right along the first axis of DCA. Furthermore, restoration years gradually increased from the bottom to the top along the second axis of CCA. These results indicate that the habitat conditions of the iron mine tailing dam gradually improved with increasing restoration years. The trend of changes in species diversity and community stability was consistent. Restoration years, revegetation types, the degree of grazing disturbance, and soil moisture were the major environmental factors affecting the characteristics of herbaceous plant communities under different revegetation types. The planted grasslands of Medicago sativa (belonging to the legume family) and Agropyron desertorum (belonging to the Gramineae family) were severely degraded due to the high degree of grazing disturbance and the deficiency of soil moisture; therefore, scientific management and protection policies should be strengthened in the future. The following dominant species pairs showed significant positive correlations: Caragana korshinskii and Hippophae rhamnoides, and Bothriochloa ischaemum and Cleistogenes squarrosa. Revegetation using these species pairs for mixed planting is therefore recommended for the future restoration of abandoned tailing dams of iron mines in this area.