Why not sample evidence variables normally?

Fixing them avoids discarding samples that disagree with observations, while weights correct for their conditional probabilities.

Does more sampling always solve the problem quickly?

No. Weight degeneracy can make convergence inefficient even with many draws.

Can continuous evidence be used?

Yes in suitable models, using conditional density contributions and appropriate numerical handling rather than requiring exact equality from random continuous draws.

AI Glossary / Statistics and probability / Advanced

Likelihood weighting

Definition

Likelihood weighting is an importance-sampling method for approximate inference in a Bayesian network. It fixes evidence variables to their observed values, samples the remaining variables in an order that respects the network's parent-child structure, and assigns each sample a weight based on the evidence probabilities given that sample's parent assignments. Posterior queries are estimated from normalized weighted counts or averages. The method avoids rejecting samples merely because their evidence values disagree, but it can still suffer from highly unequal weights and poor coverage of important posterior configurations.

What it means

A Bayesian network defines a joint distribution through local conditional distributions. When evidence is observed, the goal is often a conditional probability that would be expensive to calculate exactly. Ordinary forward sampling may generate mostly evidence-incompatible samples if the evidence is rare. Likelihood weighting keeps the evidence fixed and compensates through weights. The samples are not generally direct draws from the posterior; their unequal weights correct for the difference between the sampling proposal and the desired conditional distribution.

How it works

Traverse variables in a topological order so parents are assigned before children. If a variable is unobserved, sample from its conditional distribution given the assigned parents. If it is observed, set it to the evidence value and multiply the current weight by that value's conditional probability or density. Repeat to generate weighted samples. For a query event, sum weights of samples satisfying the event and divide by the total weight. Use log weights when necessary for numerical stability, monitor weight concentration and repeat runs to assess Monte Carlo variability. An all-zero total weight cannot produce a valid normalized estimate.

Three practical examples

Diagnostic networks: approximate the probability of a hidden condition after observing sensor evidence.
Decision-support research: estimate posterior quantities in a Bayesian network too large for convenient exact enumeration.
Algorithm evaluation: compare likelihood weighting with rejection sampling, exact inference or more adaptive proposals on controlled problems. A successful calculation under a network does not establish that its conditional probabilities represent the real system accurately.

Think of it this way

Think of generating possible explanations while keeping the observed facts pinned in place. Each explanation receives a ticket weight according to how well it predicts those facts. Explanations with larger weights count more in the final average. If almost all the weight lands on a few tickets, the apparent sample size greatly overstates the amount of useful information in the simulation.

Worked example

Let a binary hidden variable A have prior probability 0.2. An observed event E has probability 0.8 when A is true and 0.1 when A is false. Bayes' rule gives the exact posterior P(A given E) = 0.16/(0.16+0.08) = 2/3. In likelihood weighting, sample A from its prior, fix E true, and assign weight 0.8 or 0.1 accordingly. Suppose ten simulated draws happen to contain three true A values and seven false ones. Their total weights are 2.4 and 0.7, giving an estimate 2.4/3.1 = 24/31, approximately 0.7742. The common weight-based effective-sample-size diagnostic is (sum w) squared/sum(w squared) = 9.61/1.99, approximately 4.829, below ten. This original finite sample does not equal the exact posterior, illustrating Monte Carlo error and the difference between raw draw counts and weighted inference.

Illustrative case study

In a fictional case, Poplar Diagnostics uses a Bayesian network to investigate equipment alerts. Likelihood weighting produces results quickly for ordinary observations, but rare combinations of alerts leave most samples with negligible weight. The team increases monitoring of weight concentration, compares small subnetworks against exact calculations and evaluates a better proposal for difficult cases. It does not treat the mere generation of many samples as evidence of a precise answer. Reports distinguish simulation uncertainty from uncertainty about the network probabilities.

Common misconceptions and mistakes

Likelihood weighting is not maximum likelihood estimation and does not fit model parameters merely by assigning weights. Samples are not unweighted posterior draws. Avoiding rejection does not eliminate rare-evidence difficulties, especially when evidence strongly changes the distribution of ancestors sampled from their prior conditionals. The normalized estimator is generally biased at finite sample size, though it can be consistent under appropriate conditions. A large weight-based effective sample size is useful but does not prove that every important region was explored. Numerical underflow can mimic zero-probability evidence if weights are handled carelessly.

Business use

Businesses should match inference accuracy to the consequence of the decision and assess both model uncertainty and computational error. Record evidence, network version, sampling budget and diagnostics. Validate the implementation on cases with known answers and investigate unstable results across repeated runs. Where weights concentrate severely, a different inference method or proposal can be more useful than simply increasing the sample count. Interpret the output as conditional on the specified model, not as an independently verified probability of real-world events.

Check your understanding

Why is the example's estimate 24/31 rather than the unweighted fraction 3/10?

Answer: True-A samples each receive weight 0.8 and false-A samples weight 0.1. The evidence favors true A, so posterior estimation uses weighted totals. The finite weighted estimate still differs from the exact value 2/3 because the proposal sample is small.

Related terms

Importance sampling
Bayesian network
Evidence
Rejection sampling
Effective sample size
posterior inference
topological order
weight degeneracy
Monte Carlo error

Learn next

Study the importance-sampling weight derivation and self-normalized estimators. Then compare likelihood weighting with variable elimination, adaptive importance sampling and Markov chain Monte Carlo, paying attention to rare evidence and numerical stability.

Sources

Poole and Mackworth, Artificial Intelligence: Foundations of Computational Agents, third edition, Stochastic Simulation and Likelihood Weighting: https://www.cs.ubc.ca/~poole/aibook/3e/html/ArtInt3e.Ch9.S7.html . The two-variable calculation, effective-sample-size check and fictional diagnostic case above are original illustrations.

Previous: Likelihood tempering Next: Likert scale