Is MBR the same as beam search?

No. Beam search is a search procedure commonly used to approximate high-probability outputs; MBR is a decision criterion based on expected loss.

Can MBR use a learned metric?

Yes, but its decisions can inherit that metric's weaknesses.

Must the candidate and reference-sample sets be identical?

No. Some implementations share them, while others separate candidate exploration from expectation estimation.

AI Glossary / Natural language and speech / Advanced

MBR

Definition

MBR stands for minimum Bayes risk. In language and speech processing, MBR decoding chooses an output that minimizes expected loss under a specified probability distribution and loss function. It can select a different output from the single most probable sequence because different mistakes can have different costs.

What it means

A model may assign the highest probability to one exact wording while spreading substantial probability across several similar alternatives. If the task rewards closeness to likely acceptable outputs, selecting the most probable exact sequence is not necessarily the best decision.

MBR makes the decision criterion explicit. It asks how costly each proposed output would be, on average, across possible reference outcomes. The averaging distribution is usually model-based in practical language systems, so the result depends on both the model and the chosen loss.

“Minimum risk” is therefore a mathematical statement about that setup. It does not mean zero error, guaranteed factual correctness or minimum real-world business risk. A poor distribution or unsuitable metric can still favor an undesirable answer.

How it works

Let h be a candidate output and y a possible reference outcome. With input x, the expected loss is R(h|x) = Σy P(y|x)L(h,y). MBR selects a candidate h with the smallest value. An equivalent utility formulation maximizes expected utility when utility is defined consistently with the loss.

In translation, evaluating every possible sentence is impractical. Implementations restrict candidate outputs and approximate the expectation using samples, an n-best list or another representation. Candidate generation and the distribution used for comparison should both be documented.

When samples are drawn from the distribution of interest, an average of their losses can estimate the expectation. An arbitrary candidate list is not automatically an unbiased sample from that distribution, and applying extra probability weights to already sampled outcomes changes the estimator unless justified.

The loss matters. Exact-match zero - one loss recovers a maximum-probability decision when optimization is over the same outcome space. Other losses, such as sequence dissimilarity measures, can reward a different kind of consensus.

Three practical examples

A translation system compares candidate sentences using a specified quality-related utility instead of relying only on sequence probability.
A speech recognizer evaluates expected transcription error among alternative hypotheses under its chosen approximation.
A model-evaluation team tests whether a reranking metric favors omissions or common wording before using it as an MBR utility.

Think of it this way

Think of choosing a plan by comparing its consequences across several possible situations, weighting each situation by its estimated likelihood. The most likely single situation need not determine the best overall plan.

The analogy does not make model-generated alternatives independent evidence about reality. If all possibilities come from a biased model, their agreement can preserve that bias. The quality of the scenarios and the cost definition remains central.

Worked example

A lower-risk choice than the most probable output

In an invented three-outcome example, model probabilities are P(A) = 0.45, P(B) = 0.35 and P(C) = 0.20. The candidate set is also A, B and C. Define zero loss for an exact match, loss two between A and either B or C, and loss 0.5 between B and C. These losses are symmetric by construction.

The expected losses are R(A) = 0.35 × 2 + 0.20 × 2 = 1.10; R(B) = 0.45 × 2 + 0.20 × 0.5 = 1.00; and R(C) = 0.45 × 2 + 0.35 × 0.5 = 1.075. MBR chooses B, although A has the highest individual probability.

Under zero - one loss instead, the risks would be 0.55, 0.65 and 0.80, so A would win. The change demonstrates dependence on the loss function, not a universal superiority of B. All values are teaching inputs, not empirical translation scores.

Illustrative case study

A fictional translation team introduces sample-based MBR reranking for product descriptions. Its automatic metric rewards fluent similarity between candidates, but reviewers notice that candidates can agree while omitting the same model number.

The team investigates the metric and candidate pool, adds task-specific checks for required identifiers and evaluates with human-reviewed examples. It reports the sampling setup and computational cost rather than describing agreement among candidates as a guarantee of faithfulness.

The lesson is that a better decision rule must be evaluated against the actual task. MBR optimizes the criterion it is given; it does not independently discover every business requirement or correct a shared model error. The fictional case makes no measured quality claim.

Common misconceptions and mistakes

MBR always selects the most probable sequence. That occurs for particular losses, not every loss.
Generated comparison samples are verified references. They are model outputs used in an approximation.
A lower estimated risk guarantees a better real-world result. The distribution, metric and approximation may be imperfect.
MBR retrains the model by definition. Decoding can change output selection while leaving model parameters fixed.

Business use

MBR can help align generation choices with an explicit task metric, especially where exact sequence probability is a poor proxy for acceptable output. It also makes the chosen notion of error visible for review.

Assess quality, latency and cost together. Comparing many candidates against many sampled references can be expensive, and the best tradeoff depends on the implementation. Test for metric weaknesses and correlated errors, including missing names, altered numbers and unsupported content. Report which parts of the expectation are approximated and avoid presenting a model-based consensus as external factual evidence.

Check your understanding

If an MBR system uses zero - one loss over complete outputs and searches the full same outcome space, which output minimizes risk?

Answer: A most-probable output, because its risk is one minus its probability. Different losses can lead to different decisions.

Related terms

Learn next

Construct a small probability distribution and two loss tables. Calculate expected loss for each candidate under both tables, then explain which task preference caused the selected output to change.

Probability distribution

Sources

Kumar and Byrne, Minimum Bayes-Risk Decoding for Statistical Machine Translation: https://aclanthology.org/N04-1022.pdf. Foundational formulation. Müller and Sennrich, Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation: https://aclanthology.org/2021.acl-long.22.pdf. Sample-based decisions and metric-dependent limitations. The three-outcome calculation is original and checked.

Previous: Maze / labyrinth artifacts Next: MCA

MBR