Tillage can strongly affect the long-term productivity of an agricultural system by altering the composition and spatial distribution of nutrients and microbial communities. The impact of tillage methods on the vertical distribution of soil microbial communities is not well understood, and the correlation between microbial communities and soil nutrients vertical distributions is also not clear. In the present study, we investigated the effects of conventional plowing tillage (CT: moldboard plowing), reduced tillage (RT: rotary tillage), and no tillage (NT) on the composition of bacterial and fungal communities within the soil profile (0–5, 5–10, 10–20, and 20–30 cm) using high-throughput sequencing of the microbial 16S/ITS gene. Microbial communities differed by soil properties and sampling depth. Tillage treatment strongly affected the microbial community structure and distribution by soil depth, and changed the vertical distribution of soil bacterial and fungal communities differently. Depth decay of bacterial communities was significantly smaller in CT than in RT and NT, and that of fungal communities were significantly greater in RT than CT and NT. The presence/absence of species was the main contributing factor for the vertical variation of bacterial communities, whereas for fungal communities the main factor was the difference in relative abundance of the species, suggesting niche-based process was more important for bacterial than fungal community in structuring the vertical distribution. Soil total carbon was correlated more with soil bacterial (especially the anaerobic and facultatively anaerobic groups) than with fungal community. These results suggested different roles of bacteria and fungi in carbon sequestration of crop residue and in shaping soil carbon distribution, which might impact on soil fertility.