More Accurate Simulation Does Not Mean More Equal Representation
Giving a large model specified demographic attributes such as country, age, and gender and then asking it to answer social survey questionnaires in the corresponding identity has become an important approach in large-model social simulation.
The "respondents" generated this way are usually called synthetic respondents.
In recent years, such methods have gradually entered social science research, used for pre-surveys, experimental design testing, and simulation of group attitudes and behavior.
As model capabilities improve, researchers have begun to pay more attention to the reliability, applicable boundaries, and group differences of large-model social simulation. This also raises a key question: when large models are used to represent different populations, can different groups receive equally reliable representation?
Existing studies show that large models can reproduce overall attitude distributions and experimental effects on some tasks, but this simulation capability is not stable across groups, questions, and levels of analysis.
Even if a model is fairly close to real human responses overall, it may simulate some groups relatively accurately while consistently producing large deviations for others.
Therefore, average accuracy alone makes it hard to fully judge a social simulation system's ability to simulate different populations.
This issue is especially important in cross-country simulation. The language coverage of large-model training corpora, the visibility of web data, and the distribution of cultural knowledge are themselves highly uneven, and existing research has also found that models more easily reflect WEIRD societies — Western, educated, industrialized, rich, and democratic — in their views and value patterns.
Against this backdrop, whether large models also show systematic differences in simulating populations across countries becomes a question worth further testing.
To address it, Tsinghua University, together with Fudan University, Nankai University, and Shanghai Jiao Tong University, conducted a systematic evaluation of 9 open-source large models based on the World Values Survey (WVS), covering 59 countries, 160 value questions, 11 value dimensions, and 2,420 comparable subgroup units.
The study further proposes and quantifies representational equality, extending the evaluation focus from "how accurate the model is on average" to "how this simulation capability is distributed across countries."
On this basis, the study brings several common strategies for improving large-model simulation into a unified framework and examines their effects on average simulation accuracy and cross-country representational equality respectively.
The study finds significant and systematic differences in cross-country simulation accuracy: countries with higher levels of economic development, stronger digital infrastructure and innovation capacity, and higher governance quality generally have higher model simulation accuracy.
A further comparison of four types of intervention strategies shows that improvements in accuracy and in cross-country representational equality often do not move in step.
Native-language prompting and continued post-training can improve simulation accuracy for the corresponding countries, but the gains may concentrate in particular countries or language groups; preference alignment showed no systematic effect on accuracy or representational equality; and in this study's evaluation, supplementing information about the target group more often improved both average accuracy and cross-country representational equality at the same time.
The paper was accepted by Computational Linguistics (CL) in July 2026.
Paper title:
Representational Equality in Cross-country Value Simulation: A Systematic Analysis of Large Language Models
Paper link:
https://doi.org/10.1162/COLI.a.648
The study uses WVS Wave 7 data and compares model-predicted response distributions with the response distributions of the corresponding populations in the real survey. The final evaluation includes 160 questions comparable across 59 countries, covering 11 value dimensions.
The base evaluation first measures how close the model-generated answer distributions are to the real survey distributions in each country, and further examines whether this simulation capability is balanced across countries. On this basis, the study further compares two main paths for improving large-model social simulation: in-context adaptation and model training.
〓 Study overview: 59 countries, a values questionnaire, and large-model population simulation brought into one comparable evaluation framework
Not only how accurate the model is, but how that accuracy is distributed
Average accuracy measures how close the model's predictions are to real human response distributions overall, but it cannot reflect how this simulation capability is distributed across populations.
For example, two models may have the same average accuracy. One simulates countries fairly evenly; the other is very accurate for a few countries and has clearly larger errors for others. For cross-country social simulation, the two have different representational capacity.
The study therefore proposes the concept of representational equality to describe whether a model's simulation accuracy is comparable when simulating different populations. It focuses on whether the model can simulate different groups' own real response distributions with similar accuracy.
Different populations have real cultural, social, and value differences. Representational equality concerns whether, given these real differences, the model can simulate different populations' own response distributions with similar accuracy.
This study further operationalizes the concept at the country level, examining whether the model simulates the values of populations in different countries with similar accuracy.
The study measures it in three steps.
Simulation accuracy. For each question, the study uses Jensen–Shannon divergence (JSD) to measure the difference between the model's predicted response distribution and the real survey distribution, and uses 1 minus JSD as simulation accuracy. The study first computes average accuracy within each value dimension, then aggregates the 11 value dimensions with equal weights to avoid value dimensions with more questions dominating the final result, finally obtaining simulation accuracy for each country.
〓 H is the subgroup, M is the model, q is the question, and D(·) is the corresponding response distribution
Representational equality. After obtaining simulation accuracy for each country, the study uses the coefficient of variation (CV) of accuracy as the main representational equality indicator. The lower the CV, the smaller the dispersion of simulation accuracy across countries, and the higher the cross-country representational equality.
The study also uses the range, the max-min ratio, and the Gini coefficient for robustness checks, and the different indicators give highly consistent judgments of the overall pattern.
〓 Ac is the simulation accuracy of country c
Accuracy–Equality composite indicator. Accuracy and equality need to be interpreted together. A model may perform poorly for all countries yet obtain high representational equality because the gaps between countries are small.
Therefore, the study further constructs an Accuracy–Equality (AE) composite indicator, taking the geometric mean of average simulation error and the representational equality indicator. The lower the AE, the lower the model's overall simulation error and the higher its cross-country representational equality at the same time.
This indicator serves as a supplement to accuracy and equality for comparing models' combined performance on the two dimensions.
Average accuracy and cross-country representational equality produce different model rankings
Based on the above indicators, the 9 models show different rankings on average simulation accuracy and cross-country representational equality.
In average simulation accuracy, GLM-4-9B performs highest among the 9 models, but its cross-country representational equality is relatively low.
By contrast, ChatGLM3-6B's average accuracy is not the highest, yet it has the highest level of representational equality among the 9 models, and achieves the best result on the Accuracy–Equality composite indicator.
In addition, in some model families, increasing parameter scale improves average accuracy but does not steadily improve cross-country representational equality.
This result shows that in selecting and optimizing models for social simulation, average performance and cross-country representational equality need to be examined separately and judged together. Leading on overall indicators does not directly imply that a model simulates different countries equally steadily.
Cross-country differences also show a clear regional pattern. Australia, North America, and European countries are generally easier to simulate accurately, while Middle Eastern countries such as Egypt, Jordan, and Iraq consistently show larger simulation deviations.
〓 Distribution of simulation accuracy of the 9 models across 59 countries; the width of the shape indicates how densely countries cluster in that accuracy interval
Which countries are easier to simulate accurately?
To further understand these cross-country differences, the study examined the relationship between each country's simulation accuracy and economic, technological, governance, and cultural indicators.
The results show that in most models, GDP per capita, internet usage rate, and the Global Innovation Index (GII) are all positively correlated with simulation accuracy. The World Bank's six governance indicators are also generally positively correlated with simulation accuracy; countries with higher governance quality usually have higher simulation accuracy.
Cultural dimensions also show relatively systematic associations. Most models simulate countries with lower power distance, higher individualism, and higher indulgence more accurately; associations with other cultural dimensions are relatively weak, and their strength varies by model.
These results show that representational inequality in cross-country simulation has systematic structural associations. Country differences in model performance, together with economic development, digital infrastructure, governance quality, and cultural characteristics, present a relatively stable cross-country pattern.
The comparison between the United States and Mexico offers a more intuitive example. The item asks respondents "how concerned they are about unemployment or not being able to find a job." In the real survey, Mexican respondents' answers mainly concentrate on "very concerned," while the share of US respondents choosing this option is clearly lower.
However, Llama-3-8B gives fairly similar predicted distributions for the US and Mexican populations, and both are closer to the real US value distribution.
Extending the comparison to all 160 questions, in about two-thirds of them the model's predictions for the Mexican population are closer to the real US value distribution; this trend persists even when the real value distributions of the two countries differ greatly.
This phenomenon echoes the compression of group heterogeneity observed in recent research: models struggle to fully preserve the pre-existing value differences between groups, making simulation results more homogeneous.
The US–Mexico case further suggests that this homogenization may not simply shrink toward a common "average pattern" but may show asymmetric convergence: predictions for weakly represented groups may drift excessively toward groups the model simulates more accurately.
In other words, representational inequality shows not only as differences in simulation accuracy across countries but also as the model's insufficient capture of the distinctive value patterns of weakly represented groups, causing its predictions to converge toward value patterns the model represents more fully.
〓 Left: distribution comparison for Llama-3-8B on Q142 "how concerned are you about unemployment or not being able to find a job." Gray is the real human distribution, pink is the model prediction; solid is the United States, hatched is Mexico. Right: each dot represents one question; the horizontal axis is the difference between the real US and Mexican value distributions, and the vertical axis measures whether the Mexican simulation result is closer to the US or to the real Mexican distribution. Solid dots indicate predictions closer to the real US value distribution, semi-transparent dots closer to Mexico.
Does improving average accuracy improve representational equality at the same time?
After observing significant cross-country representational differences in the base setting, the study brings two common paths for improving simulation into a unified evaluation framework: in-context adaptation includes native-language prompting and supplementation of relevant group information, and model training includes continued post-training and preference alignment.
Each experiment examines average simulation accuracy, cross-country representational equality, and the Accuracy–Equality (AE) composite indicator at the same time, comparing the effects of different interventions on the two dimensions of accuracy and representational equality.
Native-language prompting can improve simulation accuracy for some target groups, but its effect on representational equality varies by model. The study used Chinese, Arabic, and Spanish instead of English for questioning.
In many language–model combinations, native-language prompting improved simulation accuracy, but the gains differed clearly across models and language groups.
For some models, the gains across language groups were relatively balanced, and representational equality improved or stayed stable; for others, the gains concentrated in particular language groups, and language groups that were originally simulated more accurately might be replaced by other language groups, without substantially narrowing representational inequality.
So native-language prompting can improve simulation for some target language groups but does not steadily improve cross-country representational equality.
Supplementing relevant group information improved both accuracy and representational equality in most evaluation settings. This strategy builds on profiles containing only a few demographic attributes and further adds specific background and value information relevant to the target group.
In most models and retrieval settings, this kind of information supplementation improved both average accuracy and cross-country representational equality.
At the same time, its effect is influenced by the retrieval method. Direct retrieval corrected existing prediction bias more clearly in some models and cases; expanded retrieval, where the model rewrites the query itself, may reintroduce the model's existing priors in some settings, weakening the improvement in representational equality.
In addition, this strategy depends on available information relevant to the target group. For populations whose relevant knowledge and digital traces are themselves scarce, the external information available for supplementation may also be more limited.
〓 Left (Fig. 5): changes in simulation accuracy of 8 models for Chinese-, Arabic-, and Spanish-speaking countries under native-language prompting. Right (Fig. 7): changes in average simulation accuracy from different strategies for supplementing relevant group information; asterisks indicate significant differences after BH-FDR correction.
Continued post-training improved average accuracy for target language groups, but the gains were unevenly distributed across countries and questions.
Continued post-training for Arabic, Japanese, Russian, Serbian, Thai, and Turkish all improved average simulation accuracy at the corresponding language-group level, but changes at the country and question levels differed clearly.
For example, the relative accuracy change for Arabic-speaking countries was about −1.2% to 3.4%. This result shows that, under this study's continued post-training setting, adding target-language data can improve overall simulation capability at the corresponding language-group level but cannot guarantee a simultaneous increase in cross-country representational equality.
Preference alignment produced no systematic improvement in simulation accuracy or representational equality. The study used two optimization methods, DPO and GRPO, and compared two preference data sources: human annotation and GPT annotation.
The results show that changes in accuracy and representational equality relative to the base model lacked a consistent direction across alignment settings. Human-annotated preference data generally preserved simulation accuracy better, but this advantage did not extend steadily to representational equality.
〓 Left: changes in simulation accuracy for each target country after continued post-training (CPT) on six languages; error bars indicate 95%
confidence intervals. Right: changes in average simulation accuracy under different DPO and GRPO preference alignment settings; asterisks indicate significant differences after BH-FDR correction.
The comparison of the four types of strategies further shows that evaluating the improvement effect of large-model social simulation cannot look only at average accuracy; representational equality must be examined at the same time.
Even if an intervention improves overall simulation accuracy, its gains may concentrate in particular countries or language groups, thus failing to improve — or even possibly reducing — cross-country representational equality.
Therefore, whether comparing models' own social simulation capabilities or evaluating new prompting, training, or alignment strategies, accuracy and representational equality both need to be included in evaluation.
This not only helps judge whether a model simulates more accurately overall, but also further identifies in which countries and groups performance improvements occur, and how those improvements ultimately change overall representational equality.
Conclusion: from average accuracy to representational equality
As large models are increasingly used to simulate public opinion, values, and social behavior, evaluating their social simulation capability by average accuracy alone is no longer sufficient.
A model being closer to real human responses overall does not mean this simulation capability covers different countries and groups equally; likewise, an intervention strategy improving overall accuracy does not mean all kinds of populations gain similar improvements from it.
This study proposes representational equality, emphasizing that while attending to "how accurately the model simulates," we also need to examine how this simulation capability is distributed across populations.
In cross-country value simulation, this means evaluating both the model's overall fit to each country's real response distribution and whether it has similar simulation accuracy for different countries.
As synthetic respondents gradually enter pre-surveys, experimental design, and cross-cultural social simulation, building more reliable large-model social simulation systems requires improving overall simulation capability while reducing the systematic differences in this capability across populations.
Including representational equality and accuracy together in the core evaluation framework helps assess models' and their improvement methods' social simulation capability more completely, and offers a new evaluation perspective for building more globally representative large-model social simulation systems.