Must the experts be machine-learning models?

No. Any source that provides the required predictions can serve as an expert.

Are correct experts rewarded by increasing their weights?

In the basic discounting version their weights stay unchanged, while mistaken experts lose relative influence.

Does the algorithm know the best expert beforehand?

No. Its theoretical comparison uses the best expert's performance measured after the sequence.

AI Glossary / AI and machine learning / Intermediate

Weighted majority algorithm (machine learning)

Definition

The weighted majority algorithm combines predictions from several experts by assigning each a weight, predicting with the weighted vote and reducing the weights of experts that make mistakes. It is a foundational method for learning from a sequence of prediction outcomes.

What it means

An expert can be a person, rule, model or any process that supplies a prediction. The algorithm does not need to know how each expert works. It observes predictions and subsequent correct answers, then changes their influence.

Unlike choosing one expert forever after a short trial, weighted majority continues to combine advice. Its theoretical appeal is a bound comparing its total mistakes with those of the best individual expert in the pool, under a specified online-learning setup. This comparison is retrospective and relative; it does not promise perfect predictions or identify the best expert in advance.

How it works

In a common binary-prediction version, all experts begin with positive weights. On each round, add the weights supporting each of the two labels and predict the label with greater total weight, using a specified rule for ties.

After the true label is revealed, multiply each mistaken expert's weight by a discount factor beta between zero and one. Correct experts retain their weights. Relative influence therefore changes even without increasing any weight.

Analysis tracks the total remaining weight and compares its decline with the weight retained by the best expert. The resulting mistake bound depends on beta, the number of experts and that best expert's mistakes. Randomized variants and more general multiplicative-weights methods have related but different guarantees.

Three practical examples

Sequential alerts: Several simple rules predict whether an operational threshold will be exceeded. Their weights are updated once the actual outcome is known.
Educational forecasting exercise: Students combine weather indicators and compare cumulative mistakes with each individual indicator, using an explicitly defined binary event.
Model monitoring research: A team studies how different prediction components perform over a stream and uses weighted voting as a transparent baseline before adopting a more complex ensemble.

Think of it this way

Think of a panel of advisers whose influence decreases after incorrect advice. The method keeps listening to everyone but pays less attention to those with repeated mistakes. The analogy assumes the correct answer becomes available; many real decisions have delayed, ambiguous or partially observed outcomes that need additional handling.

Worked example

Three experts A, B and C start with weight 1 each. On the first round, A and B predict yes, while C predicts no. The algorithm predicts yes because the total yes weight is 2 versus 1.

The actual answer is no. With beta=0.5, A and B fall to 0.5 each, while C remains at 1. On the second round, A predicts yes and B and C predict no. The no side has weight 1.5 versus 0.5, so the algorithm predicts no.

If that answer is correct, only A is penalized, falling to 0.25. These weights record relative influence under the rule; they are not calibrated probabilities that each expert is correct.

Illustrative case study

Fictional case: Queuewatch compares three rules for predicting whether tomorrow's incoming requests will exceed a fixed capacity threshold. It logs every rule's prediction before observing the outcome and updates weights afterward.

The team discovers that one proposed expert uses a report released only after the prediction deadline. It removes that unavailable information and reruns the historical exercise. The evaluation compares cumulative mistakes with each admissible rule under the same timeline. This fictional example illustrates a fair online comparison, not a claimed forecasting gain.

Common misconceptions and mistakes

The best-expert guarantee does not mean the ensemble beats the best expert on every round or in every finite sequence. If all experts are poor, a relative bound can still accompany many mistakes.

Do not interpret weights as probabilities without a separate probabilistic model. Do not silently replace deterministic voting with randomized sampling and assume the same bound applies. Standard weighted majority also accumulates the entire history; responding to a changing best expert may require specialized variants. Delayed feedback, missing labels and changing expert pools need explicit treatment.

Business use

The algorithm offers an understandable baseline when several predictors provide repeated advice and outcomes are observable. It can help teams avoid overcommitting to one plausible rule without evidence.

Before using it operationally, define the prediction target, feedback timing, mistake costs and tie handling. Binary mistakes may not reflect unequal business consequences, so a suitable loss-based extension may be needed. Compare against simple majority voting and individual experts using a chronological evaluation that avoids access to future information.

Check your understanding

An expert starts at weight 1 and makes two mistakes with beta=0.5. What is its weight afterward?

Answer: 0.25, because each mistake multiplies its current weight by one half.

Related terms

Online learning
Ensemble learning
Regret
Concept drift
prediction with expert advice
multiplicative weights
mistake bound
weighted voting

Learn next

Create a short sequence with three binary experts and hand-calculate two rounds of voting and updates. Compare the ensemble's total mistakes with each expert's total. Then vary beta and observe the response. Study the distinction between deterministic weighted majority and randomized expert-advice algorithms before citing a guarantee.

Sources

Littlestone and Warmuth, The Weighted Majority Algorithm, published in Information and Computation 108 (1994): https://doi.org/10.1006/inco.1994.1009. Primary technical-report version, revised October 1992: https://tr.soe.ucsc.edu/sites/default/files/technical-reports/UCSC-CRL-91-28.pdf. Introduction, algorithm description and footnote 3 consulted: bounds cover both updating on master mistakes and updating after every trial; this entry uses the latter version. The separate scanned journal copy did not provide usable extracted text.

Previous: Weighted linear function Next: Weighted model counting