Must the prior be strong?

No. Its informativeness should reflect justified knowledge and the model's needs, with sensitivity checks.

Does Bayesian inference always require simulation?

No. Some models have analytic solutions, including the worked beta-binomial example.

Can it compare hypotheses?

Yes, through specified Bayesian procedures. Such comparisons require careful modeling and prior choices and are not equivalent to conventional significance testing.

AI Glossary / Statistics and probability / Intermediate

Bayesian inference

Definition

Bayesian inference is the process of updating probability distributions for unknown quantities using observed data and a specified statistical model. A prior distribution represents uncertainty before incorporating the data, the likelihood describes the observation process, and Bayes' theorem combines them into a posterior distribution. Conclusions remain conditional on these modeling assumptions.

What it means

Bayesian inference represents uncertainty about quantities such as success probabilities, regression effects, or latent states with distributions rather than single estimates.

The posterior can support estimates, credible intervals, comparisons, and predictions. These outputs answer different questions: uncertainty about an average rate is not the same as uncertainty about the next individual outcome.

The framework requires judgment about the prior and likelihood. A flat prior does not automatically represent absence of information, especially after changing parameterization. The approach can incorporate existing knowledge, but it does not protect conclusions from unrepresentative data, omitted variables, or an unsuitable observation model.

How it works

Specify the unknown quantities and the data-generating assumptions. For a parameter theta and observations y, the posterior density is proportional to the likelihood p(y given theta) multiplied by the prior p(theta). A normalizing constant ensures that the posterior integrates or sums to one.

Some combinations, called conjugate models, give an analytic posterior. More complex models may require numerical integration, Markov chain Monte Carlo, or approximate methods such as variational inference. Computation must be checked: an apparently smooth posterior summary is not evidence that an algorithm explored the distribution adequately.

Summarize the posterior in ways relevant to the question. For prediction, average the distribution of future outcomes over posterior uncertainty in the parameters. For decisions, combine uncertainty with the consequences of available actions rather than choosing solely from a point estimate.

Check what the prior predicts before fitting and whether the fitted model can reproduce important features of the observed data afterward. Examine sensitivity to reasonable alternative specifications. Bayesian hypothesis comparison is possible, but is distinct from simply interpreting a conventional p-value as the probability of a hypothesis.

Three practical examples

Quality monitoring: update uncertainty about a defect rate as inspection results arrive, with an observation model appropriate to sampling and production changes.
Forecasting: combine uncertainty about model parameters with future-outcome variability to produce predictive distributions.
Hierarchical analysis: estimate related groups jointly so that limited information in small groups can be informed by a shared population model.

Think of it this way

Think of a map of explanations whose shading changes after observations. Evidence can shift or narrow it, or reveal problems with the map. Updating cannot automatically add explanations the model excluded.

Worked example

Suppose an unknown success probability p has a Beta(2, 8) prior, with mean 2 / 10 = 0.20. Observe six successes and four failures in ten trials assumed conditionally independent with a common probability p.

The likelihood is proportional to p to the sixth power times (1 - p) to the fourth power. Multiplying by the prior gives a Beta(8, 12) posterior. Its mean is 8 / 20 = 0.40, between the prior mean of 0.20 and the observed success fraction of 0.60.

The posterior variance is 8 × 12 / [20 squared × 21], approximately 0.01143. This is uncertainty about p, not the variance of a binary future outcome.

The predictive probability of success on the next trial is 0.40. For two future trials sharing the same uncertain p, the probability both succeed is E[p squared given data] = 8 × 9 / (20 × 21), approximately 0.17143. Plugging the posterior mean into p squared would give 0.16 and would ignore parameter uncertainty. The future trials are conditionally independent given p but need not be independent after averaging over uncertainty about p.

Illustrative case study

Fictional case: Birch Support estimates the probability that a new guided workflow resolves a request. The initial dataset is small, so analysts document a prior based on relevant previous experience and assess whether that experience is transferable.

After collecting new outcomes, they report the posterior distribution and predictive performance, not only a success-rate estimate. A check reveals that outcomes vary substantially by request category, making a single shared-rate model questionable.

The team revises the model and evaluates it on later requests. Managers use the results to understand uncertainty and identify where more evidence is needed, while keeping the decision to expand the workflow separate from the statistical update.

Common misconceptions and mistakes

Do not treat the posterior as assumption-free truth. It is conditional on the model, prior, data, and observation process.

Do not interpret a 95% credible interval as a guarantee that a procedure covers the true value in 95% of repeated datasets. That is a different, frequentist coverage statement; the properties can be studied but are not identical by definition.

Do not assume more data always repair a misspecified model. Large samples can produce concentrated uncertainty around an inadequate description.

Do not discard parameter uncertainty when making nonlinear predictions. The worked example shows why averaging a function of p can differ from applying the function to the average p. Also avoid reusing observations as both prior evidence and new likelihood data without accounting for that reuse.

Business use

Bayesian inference can support decisions where uncertainty itself matters, especially with limited observations, related groups, or sequential information. Its explicit assumptions make sensitivity analysis and updating easier to discuss.

Good practice records the prior rationale, likelihood assumptions, computational diagnostics, and checks against observations. Reports should explain whether probabilities concern parameters, hypotheses, or future outcomes. Business value comes from making those distinctions actionable, rather than presenting a posterior mean as a uniquely correct answer to every operational question.

Check your understanding

What is the posterior in the example?

Answer: Beta(8, 12), obtained by adding six successes and four failures to the prior shape parameters.

Why is the two-success predictive probability greater than 0.40 squared?

Answer: Averaging over the uncertain shared probability includes its positive posterior variance.

Related terms

Learn next

Work through a conjugate update, then compare parameter and predictive uncertainty. Next study prior predictive checks, posterior predictive checks, and sensitivity analysis before moving to more complex computational models.

Sensitivity analysis

Sources

Frank Harrell, Glossary of Statistical Terms, Bayesian inference entry, for the updating perspective: https://hbiostat.org/glossary/ Siva Balakrishnan, Carnegie Mellon University, Lecture 10, beta-binomial updating: https://www.stat.cmu.edu/~siva/teaching/700/lec10.pdf Stan Development Team, Posterior and Prior Predictive Checks, for model assessment: https://mc-stan.org/docs/stan-users-guide/posterior-predictive-checks.html

Previous: Bayesian history matching Next: Bayesian information criterion