AI Glossary / AI and machine learning / Intermediate
Bayes factor
Definition
A Bayes factor compares how well two specified models predict observed data. It is the ratio of their marginal likelihoods, and it tells us how the data change the odds between the models under their assumptions and parameter priors.
What it means
A model can often fit data well after its parameters have been adjusted. A Bayes factor asks a different question: how much predictive support did the model assign to the observed data after accounting for its permitted parameter values and their prior weights?
The direction of the ratio matters. A Bayes factor of 5 for model A over model B means that the data multiply the prior odds for A versus B by five. It does not automatically mean a 5-to-1 final belief in A.
For business readers, the useful distinction is between evidence supplied by data and the complete conclusion. The prior model odds, the candidate models and the assumptions behind their predictions all affect the final assessment.
How it works
Specify the competing models and their parameter priors before calculating their marginal likelihoods. A marginal likelihood averages the likelihood over parameter values according to the prior, rather than using only the best-fitting parameter setting.
Divide one model's marginal likelihood by the other's. A ratio above one favors the numerator model relative to the denominator; below one favors the denominator. Multiplying by prior model odds gives posterior model odds.
Computing marginal likelihoods can be difficult and sensitive to prior choices. Sensitivity analysis examines whether reasonable alternatives change the comparison. A large Bayes factor only distinguishes the models offered: both can still be poor descriptions of reality. Model checking and practical effect sizes remain important.
Three practical examples
- An analyst compares two specified explanations of a demand pattern using a Bayesian model comparison.
- A research team examines whether observed data support a constrained model relative to a more flexible alternative.
- A decision-maker asks how strongly a new dataset should change earlier odds between two hypotheses.
Think of it this way
Think of a piece of evidence that changes the balance between two explanations. Its contribution is separate from how plausible the explanations were before the evidence arrived.
The analogy is limited because a Bayes factor needs precise probability models and, where relevant, parameter priors. Informally calling evidence “five times stronger” does not calculate one.
Worked example
Updating odds rather than confusing them with probability
Suppose the prior odds for model A versus model B are 1:4. A new dataset yields a Bayes factor of 6 in favor of A over B. The posterior odds are 6 × 1:4, or 6:4.
If A and B are the only models under consideration, those odds correspond to a posterior probability of 6/(6+4) = 60% for A. The Bayes factor of 6 did not make A's probability six times its prior probability of 20%, nor did it guarantee certainty.
These are invented numbers illustrating the update. The result depends on the models, their priors and the assumption that the comparison is limited to A and B. Introducing another plausible model can change the model probabilities even when the pairwise evidence ratio remains the same.
Illustrative case study
A fictional subscription business studies whether a change in sign-ups is better explained by a stable-rate model or a model allowing a shift. Its analyst computes a Bayes factor and initially reports only the evidence ratio.
During review, the team asks how the shift-size prior was chosen and whether seasonal variation was represented. The analyst checks sensitivity and adds a model check before drawing a business conclusion. The estimated size of the change is reported separately.
The case illustrates that model comparison is one part of analysis. A persuasive ratio cannot compensate for omitting a relevant alternative explanation.
Common misconceptions and mistakes
- A Bayes factor is a posterior probability. It is an evidence ratio that updates prior odds.
- It compares only the best-fitting parameters. Marginal likelihood accounts for parameter uncertainty under the prior.
- A large factor proves the preferred model is true. It provides relative support among specified models.
Business use
Bayes factors can help communicate how data change support between explicit hypotheses. They are most useful when the models and prior assumptions are defensible and the audience understands the difference between evidence and final probability.
Ask for the ratio's direction, prior sensitivity and practical implications. Avoid relying on a verbal evidence category alone. A strongly supported but operationally tiny difference may matter less than a moderately supported effect with substantial consequences, depending on the decision and its costs.
People also ask
What does a Bayes factor of one mean?
The data provide equal marginal likelihood under the two models, so their prior odds are unchanged.
Does it use parameter priors?
Usually yes for models with unknown parameters, and those choices can affect the result.
Is it the same as a p-value?
No. They answer different statistical questions and should not be substituted without explanation.
Check your understanding
Prior odds are 1:4 for A versus B, and the Bayes factor is 6 for A over B. What are the posterior odds?
Answer: 6:4, giving 60% probability for A if these are the only two models considered.
Related terms
- Bayesian inference
- Model selection
- Sensitivity analysis
- marginal likelihood
- prior odds
- posterior odds
- likelihood ratio
Learn next
Read marginal likelihood and prior probability. Practice converting probabilities to odds and back before interpreting a model-comparison result.
More in AI and machine learning
- Basis function regression - Basis function regression is a way of fitting curved relationships by transforming an input into a set of fixed building-block functions, such as powers or...
- Batch - In machine learning, a batch is a group of training examples processed together in one step before the model updates its parameters.
- Batch size - Batch size is the number of training examples a model works through before it makes one adjustment to itself, a setting that trades speed, memory use and...
- Bayesian information criterion - The Bayesian information criterion, or BIC, is a model-selection score that balances fit to observed data against model complexity.
- Bayesian interpretation of kernel regularization - The Bayesian interpretation of kernel regularization explains how certain penalized kernel-learning methods correspond to probabilistic models with priors.
- Bayesian linear regression - Bayesian linear regression is a method for predicting a numerical outcome with a linear model while expressing uncertainty about its coefficients and other...
Sources
Robert E. Kass and Adrian E. Raftery, Bayes Factors: https://www.stat.cmu.edu/~kass/papers/bayesfactors.pdf. Author-hosted statistical reference on evidence ratios and model comparison. The odds calculation and subscription-business case are original teaching examples, not reported empirical findings.
Previous: Bayes estimator Next: Bayes linear statistics