This paper investigates the relationship between high-speed rail (HSR) and agglomeration economy in the scope of specialization and diversity to answer two questions: first, to determine whether specialization or diversity promotes economic productivity, and second, to determine whether HSR promotes specialization or diversity. Specialization/diversity agglomeration index based on the coefficient of variation of localization agglomeration is proposed to measure city’s specialization and diversity. Our analysis utilizes data of agglomeration across 17 industrial sectors in Japanese municipality level. Depending on the definition of agglomeration diversity, one of the results reveals a U-curve relationship as productivity is plotted in Y-axis and specialization agglomeration in X-axis. In other words, both specialization and diversity benefit to economic productivity.