Is simple averaging always wrong?

No. It is a transparent pooling rule, but its suitability should be evaluated for the task.

Must the experts agree afterward?

No. An aggregate can be used while their individual assessments remain different.

Is a pool a Bayesian posterior?

Not necessarily. That interpretation requires a defensible probabilistic model of the information being combined.

AI Glossary / Statistics and probability / Advanced

Belief aggregation

Definition

Belief aggregation combines probability assessments from several people, models, or other sources into a single probability distribution. In this statistical sense, it is also called probabilistic opinion pooling. The aggregation rule determines how the inputs influence the result. Producing a mathematically valid distribution is necessary, but it does not by itself establish accuracy, calibration, or appropriate treatment of shared information.

What it means

Sources may disagree because of different evidence, models, or interpretations. Aggregation provides an explicit way to combine those assessments for a common purpose.

The inputs must refer to the same event definitions, time horizons, and conditioning information, or be translated carefully into a compatible form. Averaging a probability of delay this week with a probability of delay this month does not create a meaningful consensus.

This entry concerns combining probabilistic beliefs rather than majority voting on preferences or merging logical statements. Those tasks have related aggregation questions but different representations and requirements. The combined distribution need not imply personal agreement among experts.

How it works

A linear opinion pool takes a weighted arithmetic average of the input distributions, using nonnegative weights that sum to one. Applied consistently over the same outcome space, it produces a valid mixture distribution.

A logarithmic pool uses a weighted geometric combination, followed by normalization. It can behave differently from a linear pool, particularly where inputs assign very small or zero probability. Its normalizing constant must exist and be positive; conflicting zero-support assessments can cause problems.

Another approach explicitly models the experts' reports as evidence and derives a Bayesian posterior. That requires assumptions about how reports relate to the event and to one another. It is not generally equivalent to applying a convenient averaging formula.

Choose a rule and weights for the intended purpose, assess overlap between information sources, and evaluate performance where suitable outcomes are available. Rules differ in properties such as how pooling interacts with conditioning or marginalization. No general label of “consensus” removes these tradeoffs, and separately pooling pieces of a graphical model can alter the intended joint relationships.

Three practical examples

Forecast panels: combine several specialists' probability distributions for the same demand event.
Model ensembles: combine predictive distributions from different statistical models while documenting the pooling rule and validation procedure.
Planning workshops: summarize structured expert uncertainty about a project outcome, retaining individual assessments so disagreements remain visible.

Think of it this way

Think of combining several maps of uncertainty. A linear pool overlays them with chosen weights; a geometric pool emphasizes their overlap differently. The overlay is useful only if the maps describe the same territory and their common sources are understood. Several copies of one map do not provide several independent views.

Worked example

Two experts assess the same binary event H. Expert A assigns probability 0.20 and expert B assigns 0.60. Their probabilities for not-H are therefore 0.80 and 0.40. Give them equal pooling weights.

The linear pool assigns P(H) = (0.20 + 0.60)/2 = 0.40 and P(not-H) = 0.60. The logarithmic pool first forms weights sqrt(0.20 × 0.60), approximately 0.346410, and sqrt(0.80 × 0.40), approximately 0.565685. Normalization gives P(H) approximately 0.379796 and P(not-H) approximately 0.620204.

Both results are valid distributions, but they answer different aggregation specifications. Neither calculation alone establishes which is better calibrated for the application.

For a separate evidence-combination illustration, suppose both expert posteriors started from prior odds 1:1 and arose from private evidence that is conditionally independent given H and given not-H. Their likelihood ratios would then be 0.20/0.80 = 0.25 and 0.60/0.40 = 1.5. Combining that evidence gives posterior odds 0.375 and probability 0.375/1.375 = 3/11, approximately 0.272727.

This third answer relies on additional assumptions about evidence and a common prior. It demonstrates why pooling reported probabilities is not automatically the same operation as Bayesian combination of independent evidence.

Illustrative case study

Fictional case study: Summit Components asks three teams to forecast whether a delivery target will be missed. Before aggregation, an analyst checks that all teams use the same deadline and definition of a missed target.

The review reveals that two teams rely on the same supplier dashboard while the third has an additional inspection report. Rather than presenting three assessments as independent confirmations, the team records this overlap and compares several justified pooling choices.

Historical forecasts are evaluated using outcomes from comparable situations, with future evaluation kept separate from weight selection. The published dashboard retains the individual probabilities alongside the aggregate. The case illustrates transparency about disagreement and shared evidence, not a guaranteed improvement from combining opinions.

Common misconceptions and mistakes

Do not equate more contributors with more independent information. Copying one assessment repeatedly can change a naive equal-weight average without adding evidence.

Do not average incompatible event definitions or use arbitrary event-specific weights that break consistency across a joint distribution. Also avoid assuming that any geometric product is already normalized.

An aggregate need not preserve every independence assumption shared by the inputs. A coherent probability distribution can still be poorly calibrated, and apparent agreement may reflect shared blind spots. These limitations should guide evaluation rather than being hidden by a consensus label.

Business use

Businesses can use belief aggregation to make forecasting and expert consultation more systematic. Explicit rules reduce ambiguity about whose assessment influences a shared planning number and permit comparison with later outcomes.

Keep records of definitions, information sources, individual distributions, pooling weights, and the dates of forecasts. Review whether performance-based weights were selected with enough relevant evidence and whether changes in the environment make them stale. Where disagreement drives a decision, additional investigation may be more useful than compressing all opinions into one number.

Check your understanding

What probability does the linear pool give in the example?

Answer: 0.40.

Why does the logarithmic result need normalization?

Answer: Its initial geometric weights do not generally sum to one.

Why is the third result different?

Answer: It combines specified independent evidence using a common prior, rather than averaging the reported beliefs.

Related terms

Calibration
opinion pooling
linear pool
logarithmic pool
expert elicitation
forecast combination
Bayesian updating
dependence

Learn next

study how pooling interacts with conditioning and marginalization, examine proper scoring rules, and compare strategies for selecting weights. Then explore how dependence among expert information sources affects evidence combination.

Sources

Sources: David Pennock and Michael Wellman, Graphical Representations of Consensus Belief, https://strategicreasoning.org/wp-content/uploads/2010/03/UAI99.pdf . The primary paper's opinion-pooling definitions and aggregation-property discussion were read. The binary pooling and separate evidence-combination examples are original, with independently checked arithmetic and explicitly stated assumptions.

Previous: Beier - Neely field warping Next: Belief function