Is it the same as Bradley - Terry?

No. Common Thurstone and Bradley - Terry formulations use different probability links: normal cumulative versus logistic.

Does a probability of 0.5 prove equivalence?

No. It can reflect equal fitted means under the simple model; limited evidence may still leave substantial uncertainty.

Must every pair be observed?

Not necessarily, but the comparison design must support the intended estimates and comparisons.

AI Glossary / Causal inference and experimentation / Advanced

Thurstone model

Definition

A Thurstone model represents comparative judgments using latent values disturbed by normally distributed variation. An option is chosen when its realized value exceeds that of its competitor. In a common equal-variance, independent-noise version, the probability of choosing A over B is a normal cumulative distribution function applied to their mean-value difference divided by the standard deviation of that difference.

What it means

People can disagree with their own earlier choices without every response being an error. Two interfaces might be similarly appealing, while attention, context, and perception vary between comparisons. A Thurstone model expresses this variability probabilistically rather than assuming that the higher-valued option wins every time.
The estimated values describe a particular comparison task on a model-defined scale. They are not direct measurements of universal quality, and their zero point does not signify complete absence of appeal. The model has roots in Thurstone's law of comparative judgment and now appears in preference learning and statistical choice modeling.

How it works

Write an option's realized utility as U_A = V_A + epsilon_A, where V_A is its mean value and epsilon_A is Gaussian noise. With independent noises of standard deviation sigma for both options, U_A minus U_B has mean V_A minus V_B and variance 2 sigma squared. Thus P(A beats B) = Phi((V_A − V_B)/(sqrt(2) sigma)), where Phi is the standard normal cumulative distribution function.
Observed comparisons help estimate the values. The model needs location and scale conventions because shifting every value equally changes no choice probability, and rescaling values together with noise changes none either. Unequal variances or correlated noises require a different difference variance. The independent equal-variance formula is a useful special case, not the whole family.

Three practical examples

A team compares pairs of generated illustrations for visual clarity. It fits a model to repeated judgments, then examines both the estimated order and uncertainty about closely matched alternatives.
Researchers compare short audio clips for perceived naturalness. Playback conditions and listener identities are recorded so that equipment differences and repeated judgments can be investigated.
A business compares product descriptions for ease of understanding. It uses the model to summarize pairwise choices while separately checking whether readers correctly recall essential product facts.

Think of it this way

Imagine each option's appeal as a marker that wobbles around a typical position. On any comparison, the higher marker wins. Widely separated typical positions produce more predictable choices; overlapping movements produce less predictable ones. The wobble is a mathematical representation of variability, not evidence that the brain literally draws numbers from a Gaussian distribution.

Worked example

Consider an original hypothetical comparison with V_A = 3, V_B = 1, and independent noise standard deviations sigma = sqrt(2) for each option. Each noise variance is 2, so the difference variance is 2 + 2 = 4 and its standard deviation is 2.
The standardized mean difference is (3 − 1)/2 = 1. Therefore P(A beats B) = Phi(1), approximately 0.8413. The reverse probability is approximately 0.1587. Across 100 independent comparisons under these assumptions, the expected number of A wins is 84.13; the observed count must be an integer and need not equal 84.
Now add 10 to both mean values. Values 13 and 11 give the same difference and therefore the same probability. Alternatively, double both original values and both noise standard deviations: the difference becomes 4 and its standard deviation becomes 4, again producing Phi(1).
These calculations demonstrate why the origin and unit of the latent scale require conventions. They do not show that A is 84.13 percent better than B. If the values were estimated from limited data, uncertainty about those estimates would add another consideration beyond the assumed judgment noise.

Illustrative case study

Fictional publisher PlainGuide asks readers to compare explanations of a technical concept for clarity. Its analyst initially reports one explanation as the winner because its fitted value is slightly higher. A reviewer asks whether that difference is sufficiently supported and whether the choices represent the intended readership.
The team examines held-out comparisons and separates responses by relevant reading experience. It also tests comprehension with questions whose answers appear in each explanation. The preference model remains useful, but the final editorial choice includes comprehension evidence and uncertainty. This fictional case describes a decision process, not a documented implementation or measured outcome.

Common misconceptions and mistakes

The model is not a claim that all human preferences satisfy Gaussian assumptions. Its suitability must be assessed for the task. Equal variances and independence should not be silently applied when an implementation allows different variances or correlations.
A frequent numerical mistake is dividing by one option's noise standard deviation instead of the standard deviation of the difference. Another is treating fitted values as ratio-scale quantities: a value of four does not mean twice the quality of a value of two. Ties, abstentions, and repeated responses from the same person also require deliberate treatment rather than automatic conversion into independent binary observations.

Business use

Use a Thurstone model when pairwise judgments are meaningful and a probabilistic summary supports a decision. Specify the comparison attribute, recruit relevant participants, randomize presentation where appropriate, and retain enough detail to assess order effects and participant dependence. Report uncertainty and predictive performance alongside rankings. A fitted ordering should help allocate attention or select candidates for further evaluation; it should not silently replace requirements such as factual correctness or accessibility.

Check your understanding

Why is the worked example's denominator two?

Answer: Independent variances add, giving difference variance four and standard deviation two.

Does adding ten to both values change the prediction?

Answer: No; the difference stays unchanged.

Does winning more comparisons establish factual accuracy?

Answer: No. That requires a separate accuracy assessment.

Related terms

Preference learning
Probit model
Latent variable
random utility model
pairwise comparison
Bradley–Terry model
Gaussian noise
identifiability

Learn next

Study how pairwise comparison data determine relative values and how location and scale constraints make those values identifiable. Then compare probit and logistic preference models using predictive checks on the same evaluation task.

Sources

Thurstone, A Law of Comparative Judgment, 1927, original article reproduced by Brock University's Mead Project: https://brocku.ca/MeadProject/Thurstone/Thurstone_1927f.html Stanford, Machine Learning from Human Preferences glossary, Thurstone model: https://mlhp.stanford.edu/src/glossary.html

Previous: Threshold function Next: Tied covariance in Gaussian discriminant analysis