Is CEV the same as majority voting?

No. It concerns extrapolated, reflected preferences and their coherence, not simply counting current votes.

Does it assume all people already agree?

No. Handling differences is part of the proposal's challenge.

Can it tell a business which values to choose?

It does not remove the need for human governance and explicit decisions.

AI Glossary / AI and machine learning / Advanced

Coherent extrapolated volition

Definition

Coherent extrapolated volition, or CEV, is a proposal in AI-alignment thinking that an advanced AI should be guided by what people would collectively want after suitable reflection, learning and development, rather than simply optimizing their immediate expressed preferences.

What it means

People can want different things before and after understanding a consequence. They can also disagree with one another. CEV attempts to frame an objective that considers informed, reflected preferences and the ways those preferences might fit together.

The proposal is associated with Eliezer Yudkowsky's 2004 paper. It is a conceptual alignment idea, not an established product feature or a solved method for determining everyone's values.

Its difficult questions are central to its meaning: whose preferences count, what kind of reflection is appropriate, how disagreement is handled and who is entitled to choose those procedures. Calling a system CEV-aligned does not answer those questions.

How it works

At a high level, the idea imagines an extrapolation process from people's current states toward more informed and reflective preferences. It then seeks coherence where those extrapolated preferences can be combined.

This description should not be mistaken for a practical implementation recipe. The original proposal leaves demanding conceptual and technical problems, including representing people accurately and avoiding an extrapolation that merely imposes the designer's preferred values.

In business discussion, it is best treated as a lens for examining objectives and authority. It does not justify overriding a user's actual instructions on the assumption that software knows what the user would eventually want.

Three practical examples

An AI-governance seminar uses CEV to compare immediate user requests with preferences after learning consequences.
A product workshop considers how a recommendation objective might differ from simply maximizing clicks.
An alignment research team examines the unresolved problem of combining reflected preferences across people with persistent disagreements.

Think of it this way

Think of a group postponing a major decision until its members have read the evidence, heard one another and considered the consequences.

The analogy has a major limit. Real deliberation involves actual people, accountable procedures and the possibility of refusal. CEV imagines an extrapolation of preferences, which raises harder questions about whether a simulated or predicted change genuinely represents what someone would endorse.

Worked example

Distinguishing reflection from permission

Imagine a fictional travel assistant asked to book the cheapest route. One route costs less but includes an overnight connection that the user has not noticed. The assistant explains the trade-off, and the user chooses a more expensive direct route.

This is an ordinary example of a preference changing with information. It helps illustrate why immediate wording may not capture every concern. It is not an implementation or validation of CEV.

Now suppose the assistant silently books the expensive route because it predicts that a more informed user would prefer it. That introduces an authority problem: a prediction of preference has replaced actual permission.

The practical lesson is to distinguish three things in a design document: the instruction received, the information offered to improve the decision and the action authorized. A philosophical idea about ideal preferences does not erase those operational boundaries.

Illustrative case study

A fictional software company's governance team discusses whether its assistant should optimize engagement or longer-term user benefit. Someone proposes CEV as the answer.

The team finds the idea useful for exposing limitations in a simple engagement target, but it does not claim to have inferred ideal preferences. Instead, it gives users understandable controls, studies their stated goals and records unresolved trade-offs.

For major choices, the product asks for an actual decision rather than presenting predicted reflection as consent. The case shows how a theoretical concept can sharpen a discussion without becoming a misleading promise that the system knows everyone's true interests.

Common misconceptions and mistakes

Treating CEV as a deployed algorithm with proven results: it is a proposal with substantial unresolved questions.
Equating coherent with unanimous: disagreement and aggregation are part of the difficulty.
Assuming extrapolation is neutral: choices about information and reflection can influence outcomes.
Using predicted preferences to bypass actual authority: a forecast of endorsement is not the same as permission.

Business use

CEV is mainly relevant to AI strategy, ethics and alignment education. It helps distinguish satisfying current behavior from supporting considered human goals.

When a supplier invokes the term, ask what has actually been implemented and how users retain control. Separate the aspiration from measurable system behavior. For most projects, explicit objectives, feedback mechanisms and accountable decision procedures are more concrete requirements than a claim to have solved collective ideal preferences.

Check your understanding

An assistant predicts that a better-informed user would approve an expensive action. Does that prediction itself authorize the action?

Answer: No. Preference prediction and actual authority are different. The system should follow the applicable permission process and offer relevant information for a real decision.

Related terms

Preference learning
AI alignment
value aggregation
human oversight
objective specification

Learn next

Read AI alignment and preference learning. For one product objective, distinguish observed behavior, stated preferences and the authority to act on either.

Preference learning

Sources

Yudkowsky, Coherent Extrapolated Volition (2004): https://intelligence.org/files/CEV.pdf. Primary proposal opened; explanatory framing and business examples are original. This entry describes a conceptual proposal, not demonstrated implementation. The proposal framing, extrapolation definition and coherence limitations were checked in the original paper, sections 2-3 and question 9.

Previous: Cohen's kappa Next: Cohort