The host plant is often the main variable explaining population structure in fungal plant pathogens, because specialization contributes to reduce gene flow between populations associated with different hosts. Previous population genetic analysis revealed that French populations of the grey mould pathogen Botrytis cinerea were structured by hosts tomato and grapevine, suggesting host specialization in this highly polyphagous pathogen. However, these findings raised questions about the magnitude of this specialization and the possibility of specialization to other hosts. Here we report specialization of B. cinerea populations to tomato and grapevine hosts but not to other tested plants. Population genetic analysis revealed two pathogen clusters associated with tomato and grapevine, while the other clusters co-occurred on hydrangea, strawberry and bramble. Measurements of quantitative pathogenicity were consistent with host specialization of populations found on tomato, and to a lesser extent, populations found on grapevine. Pathogen populations from hydrangea and strawberry appeared to be generalist, while populations from bramble may be weakly specialized. Our results suggest that the polyphagous B. cinerea is more accurately described as a collection of generalist and specialist individuals in populations. This work opens new perspectives for grey mold management, while suggesting spatial optimization of crop organization within agricultural landscapes.