Is this the same as ordinary A/B testing?

Not exactly. A/B testing can supply comparative evidence, while preferential Bayesian optimization adds a model-driven sequence of candidate choices aimed at search.

Does it guarantee the best design?

No general finite-budget guarantee follows from the label. Results depend on assumptions, feedback quality, candidate coverage, and the acquisition strategy.

Can different people have different optima?

Yes. A single pooled model may obscure that variation.

AI Glossary / Causal inference and experimentation / Advanced

Preferential Bayesian optimization

Definition

Preferential Bayesian optimization is a sequential search method that uses comparisons between alternatives to seek a preferred solution. Instead of requiring an absolute numerical quality score for each candidate, it learns from judgments such as choosing one design over another. A Bayesian model represents uncertainty, and an acquisition rule helps decide which comparison to request next.

What it means

Imagine adjusting a reading interface's line spacing and text width. A participant may readily choose the more comfortable of two versions while struggling to assign either a meaningful comfort score out of one hundred. Preferential optimization treats that comparative judgment as useful evidence.
The objective is to find a good setting within a comparison budget. It is not necessarily to reconstruct every preference across the entire design space. The answer also depends on whose preferences are represented, the task they perform, and the conditions under which they compare alternatives.

How it works

A team defines feasible candidates and a comparison protocol, gathers initial judgments, and fits a probabilistic preference model. One approach models latent utilities and links utility differences to choice probabilities. Another represents preferences over pairs more directly. Gaussian processes are widely used, but the term does not mandate one particular implementation.
An acquisition function then selects an informative or promising comparison. After receiving feedback, the model updates and the process repeats. The final recommendation should follow a stated selection rule. Optimizing the questions asked and choosing the final candidate are related but distinct decisions; the next comparison need not contain the current favorite.

Three practical examples

An accessibility team compares alternative caption layouts while holding the video and viewing task constant. Participants choose the layout they find easier to follow; eligibility checks separately enforce minimum text size.
A sound designer compares synthesized notification tones and uses judgments to explore pitch and duration settings. Playback equipment and loudness are controlled because they could otherwise dominate preferences.
A business experiments with dashboard arrangements. Analysts compare pairs while completing the same information-finding task, and the team checks task accuracy alongside expressed preference before adopting a design.

Think of it this way

Think of a careful tailor asking which of two adjustments feels better, learning from each answer, and deciding which adjustment to try next. The tailor does not need you to report comfort to three decimal places. However, a preference for one fitting says little about weather resistance or durability unless those properties are tested separately.

Worked example

Consider an original, deliberately simplified Bayesian update with two hypotheses. H1 says A is usually preferred to B; H2 says B is usually preferred to A. Give each hypothesis prior probability 0.5. Suppose P(A wins | H1) = 0.8 and P(A wins | H2) = 0.2. These probabilities are assumptions for this example, not measured results.
A participant chooses A. The evidence probability is 0.5 × 0.8 + 0.5 × 0.2 = 0.5. Bayes' rule gives P(H1 | A wins) = 0.4 / 0.5 = 0.8 and P(H2 | A wins) = 0.2.
Assuming another judgment follows the same conditionally independent likelihood, the posterior predictive probability that A wins again is 0.8 × 0.8 + 0.2 × 0.2 = 0.68. Notice the difference: 0.8 is belief in H1, whereas 0.68 is the predicted outcome probability after averaging over both hypotheses.
This calculation illustrates updating and prediction only. It is not a complete optimization algorithm: a real search also needs a candidate representation, a rule for selecting comparisons, and a stopping criterion. Repeated answers from the same participant may violate the independence assumption.

Illustrative case study

Fictional company ClearPanel develops a dashboard for warehouse supervisors. Its first preference exercise asks employees to compare screens without specifying a task. Some favor spacious layouts; others favor dense displays. The team recognizes that the answers mix different work situations.
It redesigns the exercise around locating delayed orders, records viewing conditions, and runs separate sessions for relevant user groups. Its optimizer proposes successive layout comparisons, but every candidate must meet preset accessibility requirements. Before launch, ClearPanel evaluates task completion and error rates on held-out scenarios. This fictional example illustrates a workflow; it reports no verified deployment or measured improvement.

Common misconceptions and mistakes

A preferred candidate is not automatically safer, more accurate, or more profitable. Those outcomes require their own evidence. A choice probability is also not a numerical measurement of how much someone likes a candidate.
Another mistake is interpreting uncertainty about preferences as permission to ignore inconsistent feedback. Disagreement may reveal different populations, changing contexts, fatigue, or shortcomings in the model. Ties and abstentions should be handled explicitly when the task permits them. Forcing every response into a binary winner can conceal useful information.

Business use

Use this approach when comparative judgments are practical, each evaluation consumes meaningful time, and the candidate space is structured enough for a model to generalize. Define the target population, comparison cost, feasible settings, and acceptance criteria before running the search. Keep preference collection separate from automatic deployment. A business should retain the judgment history and the conditions of each comparison so that recommendations can be investigated and reproduced.

Check your understanding

Why is the worked example's next-win probability 0.68 rather than 0.8?

Answer: Because prediction averages the likelihood under both remaining hypotheses.

What is missing if a system only predicts pairwise winners?

Answer: A search procedure still needs to select future comparisons and decide what to recommend.

Why test task errors separately?

Answer: Preference judgments do not establish objective task performance.

Related terms

Learn next

Study the distinction between learning an unknown preference model and using that model to optimize a decision. Then examine how acquisition rules balance exploration with promising candidates, and how participant differences affect the usefulness of a shared preference model.

Sources

Stanford, Machine Learning from Human Preferences glossary, preferential Bayesian optimization: https://mlhp.stanford.edu/src/glossary.html González, Dai, Damianou and Lawrence, Preferential Bayesian Optimization, ICML 2017, primary research paper: https://proceedings.mlr.press/v70/gonzalez17a/gonzalez17a.pdf

Previous: Preference regression Next: Preferential entailment