Does Bayes' theorem apply only to subjective beliefs?

No. It is a probability identity and can be used with different interpretations of probability.

Must evidence raise a hypothesis's probability?

No. Evidence can raise, lower, or leave it unchanged depending on relative likelihoods.

Is the posterior the same as a likelihood?

No. The posterior also incorporates the prior and normalization over alternatives.

AI Glossary / Statistics and probability / Intermediate

Bayes theorem

Definition

Bayes theorem, usually written Bayes' theorem, is a probability identity relating the probability of a hypothesis after observing evidence to its prior probability and the likelihood of that evidence under the hypothesis. For events H and E with P(E) greater than zero, it states P(H given E) = P(E given H) P(H) / P(E).

What it means

The probability of seeing an alert when a fault exists is not the same as the probability of a fault when an alert appears. Bayes' theorem connects these two directions by accounting for how common faults are and how often alerts occur overall.

In an updating interpretation, the prior describes uncertainty before incorporating the specified evidence, and the posterior describes uncertainty after incorporating it. The likelihood measures how compatible the evidence is with the hypothesis under the model.

The theorem is a consequence of conditional probability, not a guarantee that supplied numbers are accurate. Reliable use requires probabilities appropriate to the population, observation process, and information already included. It does not establish a causal explanation merely by reversing the direction of conditioning.

How it works

Write the joint probability of H and E in two ways: P(H given E) P(E) and P(E given H) P(H). Equating them and dividing by the positive evidence probability gives the theorem.

When the alternatives are H and its complement, compute P(E) as P(E given H) P(H) plus P(E given not H) P(not H). For several mutually exclusive and exhaustive hypotheses, sum the corresponding likelihood-times-prior products over all alternatives.

The denominator normalizes the result so the posterior probabilities add to one. In continuous-parameter inference, densities and integration replace the relevant discrete probabilities and sums; a density value is not itself a probability mass at one exact parameter value.

For two hypotheses, posterior odds equal prior odds multiplied by the likelihood ratio. Sequential updating is possible, but later evidence must use the correct conditional likelihood given earlier evidence. Multiplying repeated likelihood ratios as if correlated observations were independent can exaggerate confidence. A hypothesis assigned exact prior probability zero cannot gain positive posterior mass through ordinary updating with a finite positive normalizing constant.

Three practical examples

Equipment monitoring: interpret the probability of a genuine fault after a warning, using fault prevalence and the warning system's behavior.
Message classification: combine class prevalence with evidence from message features to estimate class probabilities under a specified model.
Forecast revision: update uncertainty about a scenario after receiving new observations, while checking whether the observations add independent information.

Think of it this way

Think of filtering a large tray of mixed items into a smaller tray containing only those with a particular mark. The marked tray's composition depends both on how common each item type was originally and how likely each type was to carry the mark. Bayes' theorem calculates that new composition.

Worked example

Suppose 3% of components in a hypothetical population are defective. A screening system flags 80% of defective components and 4% of nondefective components. Assume these rates apply to the screened population.

Imagine 10,000 components. There are 300 defective components, of which 240 are flagged. There are 9,700 nondefective components, of which 388 are flagged. Among the 628 flagged components, 240 are defective.

The probability of a defect given a flag is therefore 240 / 628, approximately 38.2%. Equivalently, Bayes' theorem gives (0.80 × 0.03) / [(0.80 × 0.03) + (0.04 × 0.97)]. The result is neither the 80% detection rate nor the original 3% prevalence.

If the same conditional flag rates instead applied to a population with 20% defects, the corresponding posterior would be 0.16 / (0.16 + 0.032), approximately 83.3%. This second calculation illustrates why population prevalence matters; it does not assume that real screening performance necessarily stays constant across different populations.

Illustrative case study

Fictional case: Cedar Works installs a warning system for a production component. Supervisors initially interpret an 80% detection rate as meaning that 80% of alerts indicate defects.

The analyst separates detection rate from the probability of a defect among alerts and combines the relevant prevalence and false-positive information. The team uses the resulting estimate to plan a confirmatory inspection process and to anticipate inspection workload.

When the production mix changes, staff reassess both prevalence and alert behavior. They keep the probability calculation separate from the decision to stop a line, which also depends on the consequences of missed defects and unnecessary stoppages.

Common misconceptions and mistakes

Do not interchange P(E given H) and P(H given E). Their denominators refer to different groups.

Do not ignore the base rate, or use a prevalence measured in a different population without justification. Selection into a testing or review process can change the relevant probabilities.

Do not treat a posterior probability as certainty or as a direct instruction to act. Decisions also depend on costs, benefits, and available alternatives.

Do not count the same evidence twice. Information already included in a prior or earlier update must be handled consistently. Likewise, an extremely small likelihood for every proposed hypothesis can indicate that the model's alternatives are incomplete rather than that the least implausible explanation is necessarily adequate.

Business use

The theorem helps businesses interpret alerts, predictions, and evidence without confusing system performance measures with the probability of a particular condition. It also makes assumptions visible: teams can see how conclusions change with prevalence, measurement reliability, or competing explanations.

A useful implementation records the source and applicability of each input probability, tests sensitivity, and checks calibration against later outcomes where possible. The equation is often straightforward; choosing defensible inputs and a suitable observation model is usually the more demanding work.

Check your understanding

Why are there more false than true flags in the first example?

Answer: Nondefective components are much more common, so their small false-positive rate still produces many flags.

What must hold for the basic event formula's denominator?

Answer: The observed evidence must have positive probability under the model.

Related terms

Learn next

Practice translating conditional percentages into expected counts. Then study odds-form updating and examples with dependent evidence to understand when multiplying likelihood ratios is justified.

Likelihood

Sources

Hossein Pishro-Nik, Introduction to Probability, Statistics, and Random Processes, Bayes' Rule: https://www.probabilitycourse.com/chapter1/1_4_3_bayes_rule.php Same author, Law of Total Probability, for normalization across a partition of alternatives: https://www.probabilitycourse.com/chapter1/1_4_2_total_probability.php

Previous: Bayes risk Next: Bayesian active learning by disagreement