Floret development is critical for grain setting in wheat (Triticum aestivum), but more than 50% of grain yield potential (based on the maximum number of floret primordia) is lost during the stem elongation phase (SEP, from the terminal spikelet stage to anthesis). Dynamic plant (e.g., leaf area, plant height) and floret (e.g., anther and ovary size) growth and its connection with grain yield traits (e.g., grain number and width) are not clearly understood. In this study, for the first time, we dissected the SEP into seven stages to investigate plant (first experiment) and floret (second experiment) growth in greenhouse- and field-grown wheat. In the first experiment, the values of various plant growth trait indices at different stages were generally consistent between field and greenhouse and were independent of the environment. However, at specific stages, some traits significantly differed between the two environments. In the second experiment, phenotypic and genotypic similarity analysis revealed that grain number and size corresponded closely to ovary size at anthesis, suggesting that ovary size is strongly associated with grain number and size. Moreover, principal component analysis (PCA) showed that the top six principal components PCs explained 99.13, 98.61, 98.41, 98.35, and 97.93% of the total phenotypic variation at the green anther, yellow anther, tipping, heading, and anthesis stages, respectively. The cumulative variance explained by the first PC decreased with floret growth, with the highest value detected at the green anther stage (88.8%) and the lowest at the anthesis (50.09%). Finally, ovary size at anthesis was greater in wheat accessions with early release years than in accessions with late release years, and anther/ovary size shared closer connections with grain number/size traits at the late vs. early stages of floral development. Our findings shed light on the dynamic changes in plant and floret growth-related traits in wheat and the effects of the environment on these traits.