Introducing Giedi 2, a Large Population Model.

A Large Population Model (LPM) differs from a Large Language Model (LLM) in what it is trying to predict. An LLM estimates what a person might plausibly say next; an LPM estimates how a whole population would respond — how many would say yes, how many no, and how that changes as you narrow the population down. The two may use language as an interface and may even share similar architectures, but they model very different things.

What it predicts. Given a question and a population, Giedi 2 produces a distribution over the answers, which can be further conditioned on demographic variables such as age, gender and income, or on variables the user supplies.

How fast it runs. A single invocation is light enough that hundreds of thousands of synthetic people can be simulated in a few minutes.

What it learnt from. Over two years we built a training corpus of writing samples stratified by demographic variables, which is, as far as we know, the largest of its kind.

Where to use it. Giedi 2 is free to use through Snowglobe, our simulation of the United States, and can be customised or deployed on-premises for businesses, governments and non-profits.

0:00 / 0:00
Read about Snowglobe

LLMs vs. LPMs

Suppose we ask both models:

Do you favor or oppose phasing out the production of new gasoline cars and trucks by the year 2035?

The LLM may produce:

I oppose it.

This is the LLM’s estimate of the next words in the sequence, given the question and whatever it learnt from its training corpus. It is an answer, singular, from nobody in particular. The answer here is illustrative.

An LPM instead produces a distribution over answers (either directly, or by sampling repeatedly and aggregating). For US adults, Snowglobe returned:

Population
All US adults

“Do you favor or oppose phasing out the production of new gasoline cars and trucks by the year 2035?”

  1. Favor34.8%
  2. Oppose65.2%

Snowglobe returns 34.8% favour and 65.2% oppose. The LLM’s illustrative answer agrees with the larger group in that estimate. But the sentence alone tells you nothing about how large that group is, or about the 34.8% who favour the policy. An LPM is trying to estimate both shares, conditioned on a specific population, and how they change with age, gender, income or another variable we give it. With a price change or a piece of legislation, the size of the disagreement is often the reason for asking.

You can read more about LPMs and simulation in our blog post.

Why?

People, organisations and governments repeatedly make decisions whose consequences are contingent on how other people respond: which product to build, which price to change, which message to publish, which service to redesign, which policy to test.

A substantial part of any modern economy consists of people being paid to predict, represent, impersonate or reason about other people. Not all of this is described as “simulation” — just as throwing a baseball is not considered experimental physics. What it has in common is this:

Given what we know about a person or population, predict how they would think, respond, decide or behave in a situation they have not yet encountered.

A strategist imagines how consumers will react to an advertisement. A salesperson rehearses a buyer’s objections. A policy adviser anticipates how constituents will respond to legislation. A screenwriter constructs dialogue for fictional people. A military planner predicts an adversary’s decisions.

Human simulation, in other words, is already an enormous category of economic labour, carried out for individuals and for aggregates alike. It is used in roughly three ways, and each sets a different bar:

Rehearsal. A simulated customer, patient or negotiating partner needs to behave plausibly and expose the trainee to useful situations. It need not reliably forecast any particular real person.

Exploration. A simulated stakeholder can help generate hypotheses, objections and scenarios, even without trustworthy probabilities. Its value comes from widening the search space, with a human checking the results afterwards.

Prediction. A simulated electorate, market or customer segment has to match the real distribution of responses and decisions. Plausibility is not enough here; the simulation has to be empirically calibrated.

Prediction is the hardest of the three, and the one Giedi 2 was built for, so the natural next question is how well calibrated it is.

Benchmarks

Benchmarking a Large Population Model is difficult, for two reasons.

  1. LLMs are optimised for tasks whose answers are identifiably correct and don’t change. An LPM has to model how groups of people evolve over time, so the right answer moves, and a benchmark has to move with it.
  2. The field is so new that there are very few benchmarks to measure against.

We designed several internal ones, and also report results on two external benchmarks.

AGLR LPM-1*

AGLR LPM-1 is our unreleased benchmark for how accurately AI systems model human populations: their opinions, behaviours and language, and how they respond to changing circumstances. It compares a model’s predictions with empirical data the model hasn’t seen, and it rewards predictive accuracy and honest uncertainty, not how convincingly human a simulation appears.

It evaluates a model in six domains:

Scored below

  1. L1

    Marginal fidelity

    Does the model reproduce the distribution of an observable across a population?

  2. L2

    Conditional fidelity

    Does it reproduce how that distribution varies with demographics, geography, time and context?

  3. L3

    Joint fidelity

    Does it preserve the dependencies among a person’s beliefs, characteristics, expressions and behaviours?

Not yet scored

  1. L4

    Temporal fidelity

    Does it reproduce individual trajectories, population transitions, persistence and change over time?

  2. L5

    Interventional fidelity

    Does it predict how individuals and populations respond to experimentally introduced changes in their circumstances?

  3. L6

    Collective fidelity

    Does it reproduce population-level outcomes that arise from interaction, social influence, coordination and feedback between people?

In each domain, a model is scored by its predictive information gain over a reference model, GLM 5.3. A positive score means the model predicts the real data better than the reference does; a negative one, worse. Scoring against a reference means we never need the ground truth, which in practice is unknowable.

Information gain over GLM 5.3

Overall
  1. Giedi 2+32
  2. GPT-6.1 Sol+12
  3. GLM 5.30
  4. Claude Opus 5.5†−101

Bars show the overall score, which summarises L1 to L3. † Claude Opus 5.5’s scores run off the scale, depressed principally by its refusals to engage with a wide range of topics.

Full results

Survey-equivalent is the number of respondents a simple random survey would need to be as accurate as the model.

L1 Marginal fidelity
ModelGainRMSESurvey-eq.
Giedi 2+493.5 pp204
GPT-6.1 Sol+264.0 pp156
GLM 5.304.5 pp123
Claude Opus 5.5−976.0 pp69
L2 Conditional fidelity
ModelGainRMSESurvey-eq.
Giedi 2+326.5 pp59
GPT-6.1 Sol+167.0 pp51
GLM 5.307.5 pp44
Claude Opus 5.5−9910.0 pp25
L3 Joint fidelity
ModelGainTVDSurvey-eq.
Giedi 2+15.10048
GPT-6.1 Sol−8.11536
GLM 5.30.11039
Claude Opus 5.5−108.18015

Giedi 2 leads in every scored domain, and is the only model ahead of the reference throughout.

Its margin narrows from L1 to L3, as the domains ask for more structure.

L1Marginal fidelityHow many people want the product?

Suppose a company wants to size a market of one million potential customers. It runs a large, representative survey to establish the true answers, and an LPM tries to reproduce them without seeing the survey.

With the question “Would you consider buying the product?”, the survey finds that 60% would. Giedi 2 predicts 56.5%, and so underestimates interest by 35,000 people.

Market
1,000,000 people

“Would you consider buying the product?”

  1. Survey60%
    600,000 people
  2. Giedi 256.5%
    565,000 people

Error −3.5 pp · 35,000 people

We’ve set this single-question error equal to Giedi 2’s L1 RMSE of 3.5 pp for illustration. In practice, RMSE summarises the error across many questions.

An error of that size is consequential, but it is perfectly adequate for telling a market of 200,000 from one of 600,000. A model a full point better on L1 would miss 25,000 people instead. That precision might matter in a market-sizing exercise, but probably not in an early go/no-go decision.

Another way to read the same error is as a survey. On L1, Giedi 2 is as accurate as a simple random survey of 204 people. The next‑best model matches a survey of 156, and the reference model one of 123.

L2Conditional fidelityHow does interest vary between groups?

Now suppose the true interest rate among people aged 18–29 is 72%.

Subgroup
Aged 18–29
National
60%

How different models might estimate that subgroup

  1. Survey72%
  2. Giedi 2−6.5 pp65.5%
  3. Advanced model−4 pp68%

National population · 60%

We’ve set each error equal to the model’s L2 RMSE, 6.5 pp for Giedi 2 and 4 pp for the advanced model, for illustration.

Giedi 2 would predict 65.5%, missing by 6.5 points. That might still tell a marketing team that younger consumers have higher interest than the national population. But a 6.5-point error could completely change the ranking of two audiences whose real interest rates are close together.

An advanced model with 4 pp L2 RMSE is much more useful for prioritising segments, although it still wouldn’t reliably resolve every small difference.

L3Joint fidelityWhat else is true about the people who want the product?

This is a different and often commercially valuable question. Suppose the real population has these characteristics:

  • 60% would consider buying the new product.
  • 40% currently subscribe to a competitor.
  • 32% both would consider buying and subscribe to a competitor.

A model could get the first two numbers exactly right while substantially misrepresenting the third.

Market
1,000,000 people

Who would consider buying, and already subscribes to a competitor?

Subscribes to a competitorWould consider buyingYesNoYes

32%27%

28%33%

No

8%13%

32%27%

SurveyGiedi 2

Every prediction here matches the survey on each trait alone, at 60% and 40%. They differ only in how the two overlap.

People missed in the overlap

In this example, Giedi 2 misses 50,000 potential competitor-conquest customers. It gets both traits right on their own, but undercounts the overlap by five points. Measured across all four groups, that is a total variation distance of 0.10, exactly Giedi 2’s L3 error.

A more advanced model, making the same kind of mistake with an L3 error of 0.06, would put 29% in the overlap rather than 27%. It would find 20,000 more of these customers.

This is why L3 can be economically valuable even when L1 and L2 are strong. It tests the ability to identify combinations of characteristics rather than individual characteristics in isolation.

OpinionQA

OpinionQA comes from “Whose Opinions Do Language Models Reflect?” (Santurkar et al., Stanford, 2023). It takes about 1,500 questions from Pew Research Center’s American Trends Panel and turns them into multiple-choice prompts. The Panel surveys US adults nationally on topics like politics, science, privacy and family. Pew records how people actually answered, so each question carries a real spread of responses, for the country as a whole and for 60 demographic groups. A model is asked to predict that spread, not to give one answer, and is scored on how closely it matches.

  1. Giedi 23
    0.936Confidence interval 0.921 to 0.943
  2. Claude Opus 5.5
    0.853Confidence interval 0.846 to 0.860
  3. GPT-6 Astra
    0.788Confidence interval 0.779 to 0.797
  4. GPT-6.1 Sol
    0.787Confidence interval 0.778 to 0.797

Reference scores: Humans (typical group vs population) 1.000; Uniform (distribution over options) 0.821.

Humans
Typical group vs population
Uniform1
Distribution over options
Band4
Confidence interval
Shaded
Worse than uniform
  1. 1

    Uniform is a reference prediction that splits its probability evenly across every option, as if it knew nothing about the question or the people answering it. It still scores 0.821, because with only a handful of options an even split is never very far from any real answer. That is why the axis starts at 0.75: the meaningful range lies between this line and 1, and a model to the left of it predicts public opinion worse than an even split would.

  2. 2

    For each question, the predicted spread of answers is compared with the real one. The options sit on a scale (from “strongly agree” to “strongly disagree”, say), and the score is 1 minus the Wasserstein, or “earth mover’s”, distance between the two: how much of the prediction has to be moved, and how many steps along the scale, to turn it into the real answers. It is scaled so that an exact match scores 1 and the worst possible prediction scores 0, then averaged over every question.

  3. 3

    Giedi 2 was benchmarked on the core model directly, not through Snowglobe. The public version wraps the model in safety classifiers and an agentic system that cleans up and presents its results, and it doesn’t expose the model’s time input. Giedi 2 isn’t a conventional language model: it takes inputs beyond a sequence of text, and the date a question was asked arrives through a channel of its own. That matters here because, unlike most benchmarks, the right answer in OpinionQA changes over time. Pew recorded opinion as it stood when each survey was fielded, and opinion has moved since, so the model has to predict answers as of that date. Every model was given the survey date; the others received it in their prompt.

  4. 4

    The band around each score is its confidence interval: the range the score would plausibly fall in on a different sample of questions. Where two bands don’t overlap, the gap between those models is unlikely to be chance.

Giedi 2 scores 0.936 on OpinionQA, compared with 0.853 for Claude Opus 5.5, the next-best model in this comparison.

SimBench

SimBench asks how well a model reproduces human responses across different kinds of question. Introduced by Hu et al. in “SimBench: Benchmarking the Ability of Large Language Models to Simulate Human Behaviors”, it brings together 20 datasets covering public opinion, moral decisions, economic choices, personality and humour. A model has to predict the spread of answers people actually gave, including the disagreements — humanity has yet to oblige us with a single answer key.

It tests both whole populations and narrower demographic groups. The overall score averages the two splits, testing whether a model captures the broad distribution and how it changes between groups.

Top-performing models on SimBench
ModelScore ↑
Giedi 276.63
Claude-3.7-Sonnet40.80
Claude-3.7-Sonnet-400039.46
GPT-4.134.55
DeepSeek-R134.52
o4-mini-high28.99
Llama-3.1-405B-Instruct28.40
Qwen2.5-72B-Instruct27.61
Qwen2.5-32B-Instruct23.76
OLMo-2-32B-DPO19.80

0 uniform · 100 exact match

Higher is better. Dots show exact scores, without confidence intervals. The score measures reduction in total variation error relative to a uniform prediction, normalised within each dataset. An exact match scores 100; the uniform baseline scores 0. Negative scores are possible. This is not percentage accuracy.

Peer scores are from Table 1 of the paper, averaged across both splits. Giedi 2’s result is reported here separately.

Giedi 2 scores 76.63, a lead of 35.83 points over Claude-3.7-Sonnet, the highest-scoring model in the paper’s comparison.

Building Giedi 2

You can read about our journey building Giedi 2 in our blog post.

The corpus

A model can only learn how different populations write and respond if its training data records who did the writing. Over two years, we built a large training corpus of writing samples stratified by demographic variables. It is the largest corpus of its kind that we’re aware of.

Scale

An LPM has to be a competent estimator for individuals and for aggregates, and an aggregate estimate is only as reliable as the number of simulated people behind it. We needed each invocation of the model to be light enough to run hundreds of thousands of synthetic people in a few minutes, which meant rethinking the architecture to minimise memory consumption and to make every parameter pull its weight.

Availability

A limited set of Giedi 2’s features is free to use through the Snowglobe platform, a simulation of the United States.

For businesses, governments and non-profits, Giedi 2 can be customised to specific needs. We also offer on-premises deployments for those who want to keep their data private.

© 2026 Aguilar Technologies Inc.