Grain minerals in rice, especially those in milled grains, are important sources of micro-nutrition elements, such as iron (Fe), zinc (Zn), manganese (Mn), copper (Cu), and selenium (Se), and of toxic heavy metal elements, especially cadmium (Cd), for populations consuming a rice diet. To date, the genetic mechanism underlying grain mineral concentrations (GMCs) in milled grain remains largely unknown. In this report, we adopted a set of 698 germplasms consisting of two subsets [indica/Xian (X-set) and japonica/Geng (G-set)], to detect quantitative trait loci (QTL) affecting GMC traits of Fe, Zn, Cd, Mn, Cu, and Se in milled grains. A total of 47 QTL regions, including 18 loci and 29 clusters (covering 62 Cd loci), responsible for the GMCs in milled grains were detected throughout the genome. A joint exploration of favorable haplotypes of candidate genes was carried out as follows: (1) By comparative mapping, 10 chromosome regions were found to be consistent with our previously detected QTL from linkage mapping. (2) Within eight of these regions on chromosomes 1, 4, 6, 7, and 8, candidate genes were identified in the genome annotation database. (3) A total of 192 candidate genes were then submitted to further haplotype analysis using million-scale single nucleotide polymorphisms (SNPs) from the X-set and the G-set. (4) Finally, 37 genes (19.3%) were found to be significant in the association between the QTL targeting traits and the haplotype variations by pair-wise comparison. (5) The phenotypic values for the haplotypes of each candidate were plotted. Three zinc finger (like) genes within two candidate QTL regions (qFe6-2 and qZn7), and three major GMC traits (Fe, Zn, and Cd) were picked as sample cases, in addition to non-exhausted cross validations, to elucidate this kind of association by trait value plotting. Taken together, our results, especially the 37 genes with favorable haplotype variations, will be useful for rice biofortification molecular breeding.