Existing electricity market simulation research assumes that energy providers share theoretical and homogeneous strategy spaces. However, practical bidding strategies differ significantly from those based on theoretical simulation. The factors that determine energy providers’ bidding patterns are heterogeneous. Ignorance of practical and heterogeneous bidding strategy spaces distorts real market results. To fill the research gap, this paper proposes a data-driven multi-agent framework for electricity market simulation using practical and heterogeneous bidding strategy spaces. Firstly, a clustering algorithm is proposed for extracting practical and heterogeneous bidding strategy spaces from historical bidding strategies. The bidding strategy spaces are formed by the centers. Then, energy providers are modeled by agents with practical and heterogeneous strategy spaces in a multi-agent framework. Finally, the proximal policy optimization is implemented by agents to model the bidding behaviors in markets. Numerical results based on real data from the Australian Energy Market Operator show that market clearing results under practical and heterogeneous strategy spaces are closer to reality than those under theoretical and homogeneous strategy spaces.