AI Glossary / Graphs and symbolic AI / Intermediate
Decision theory
Definition
Decision theory studies how choices relate to beliefs, uncertainty and preferences. In AI, a central approach evaluates available actions by their expected utility: the probability-weighted value of their possible consequences under a specified model.
What it means
Knowing what is likely does not by itself determine what should be done. A rare outcome can matter greatly if its consequence is severe, while a likely outcome may contribute little to the decision if all actions handle it equally well.
Decision theory separates beliefs about outcomes from preferences over those outcomes. Probabilities describe uncertainty; utility represents the decision maker's evaluation of consequences. A policy connects available information to an action.
Normative decision theory examines principles for coherent choice under assumptions. Descriptive theories examine how people actually decide. The two are related, but an expected-utility model should not be presented as a complete psychological description of human behavior.
How it works
Specify the available actions, relevant states or outcomes, current evidence and how actions affect outcomes. Assign probabilities appropriate to that evidence and action model, and define a utility function covering the consequences that matter.
For each action a, expected utility is the sum over outcomes of their probability under that action multiplied by their utility. The maximum-expected-utility principle chooses an action attaining the largest total.
The expected-utility framework has a formal foundation in consistency conditions on preferences over uncertain prospects, including transitivity and appropriate treatment of equivalent lotteries. It is not justified merely by choosing convenient numbers and calling them utilities.
Utility need not equal money, task count or a model's prediction score. Its scale represents preferences over uncertain outcomes. Positive affine transformations preserve expected-utility choices: multiplying every utility by the same positive number and adding the same constant changes the units without changing the ranking. Arbitrary nonlinear transformations can change choices under risk.
Three practical examples
- An AI assistant compares acting immediately with asking for clarification when the consequences of misunderstanding differ.
- A scheduling tool weighs possible completion outcomes according to an explicitly defined operational objective.
- A classifier's output is converted into a review decision by combining predicted probabilities with the consequences of correct and incorrect handling.
Think of it this way
Think of choosing a route using both your beliefs about travel conditions and what matters for the trip. The fastest average route may not be preferred if arriving late has a particularly serious consequence.
The analogy does not mean a person's preferences can always be captured by one easily measured quantity. The model must make its treatment of uncertainty and competing consequences explicit.
Worked example
Different utilities can reverse the preferred choice
Consider two invented options measured in reward points. Option A gives 40 points for certain. Option B gives 100 points with probability 0.5 and 0 points otherwise.
If utility equals points, A has expected utility 40 and B has expected utility 50, so B is preferred.
Now suppose the decision maker's utility for nonnegative points is U(x)=sqrt(x). A has utility sqrt(40), approximately 6.325. B has expected utility 0.5×sqrt(100)+0.5×sqrt(0)=5. Under these preferences, A is preferred even though its expected points are lower.
The certainty equivalent of B under this utility is 25 points, since sqrt(25)=5. Replacing U with 2U+7 preserves the choice: A becomes about 19.65 and B becomes 17.
These are two different preference models, not competing arithmetic answers. The square-root function is an illustrative assumption rather than a claim about everyone's attitude toward uncertainty.
Illustrative case study
A fictional automation team, Willow Workflow, initially selects actions by maximizing the average number of requests completed. Its users explain that some failures consume much more recovery effort than others, so completion count alone misses an important consequence.
The team revises its outcome model and works with the responsible operators to define a more appropriate preference measure. It separates uncertainty in its failure predictions from disagreement about the relative importance of recovery effort and speed.
The revised policy is evaluated on those stated consequences. The team does not claim that a higher expected score guarantees a better outcome for every individual run, or that optimizing one group's score automatically represents every stakeholder's interests.
Common misconceptions and mistakes
- Rational choice means picking the most likely outcome. Actions produce uncertain consequences; the decision compares their complete relevant distributions.
- Utility is always monetary value. It can represent other preferences, provided the scale is used consistently.
- Maximizing expected utility eliminates bad outcomes. It selects according to a model before uncertainty is resolved.
- A mathematically optimal policy must have the right objective. The optimization can be correct while the specified utility omits important consequences.
Business use
Decision theory helps distinguish prediction quality from decision quality in AI systems. Improving a probability estimate matters operationally when it improves the actions taken under the relevant consequences and constraints.
Review the action model, uncertainty estimates and utility assumptions separately. Test how policy choices change when those inputs change. For repeated decisions, evaluate calibration and realized outcomes over suitable data while retaining a record of the information available at each choice. For multi-party decisions, make clear whose preferences are represented and how disagreements are handled.
People also ask
Does expected utility require knowing the future?
No. It uses probabilities to represent uncertainty about future or hidden outcomes.
Can two people rationally choose different actions from the same probabilities?
Yes. Different coherent preferences can produce different utility rankings.
Is decision theory the same as game theory?
Game theory specifically studies interacting decision makers whose choices affect one another. Decision theory also includes choices without that strategic interaction.
Check your understanding
Option A has expected monetary value above option B. Must every expected-utility decision maker prefer A?
Answer: No. Expected monetary value and expected utility can differ when preferences over monetary outcomes are nonlinear or other consequences matter.
Related terms
- Expected utility
- Probability
- Utility function
- Decision analysis
- Certainty equivalent
- Bounded rationality
- Game theory
- risk attitude
Learn next
Recalculate the two reward options using U(x)=x and U(x)=sqrt(x). Find each option's certainty equivalent under the second model. Explain which calculations concern uncertainty and which depend on the preference function.
More in Graphs and symbolic AI
- Decision lists - A decision list is an ordered sequence of rules in which the first matching condition determines the output.
- Decision network - A decision network is a graphical model that combines uncertain variables, available decisions and utilities to support choosing actions under uncertainty.
- Decision nodes - Decision nodes represent choices that an agent or decision maker can control in a decision model.
- Decision-theoretic metareasoning - Decision-theoretic metareasoning uses decision theory to choose which computations to perform, how much effort to spend and when to stop thinking and act.
- Decomposition - Decomposition is the process of expressing a problem or structure as smaller parts whose solutions or properties can be combined to address the original whole.
- Deductive learning - Deductive learning is learning that uses existing knowledge and logical reasoning to derive reusable knowledge or procedures.
Sources
Russell and Norvig, AIMA Chapter 16 materials, presents preference consistency, expected utility, utility scales and information value: https://aima.cs.berkeley.edu/4th-ed/slides-pdf/chapter16.pdf
Decision Analysis editors discuss the expected-utility framework and its scope: https://pubsonline.informs.org/doi/full/10.1287/deca.2020.0408
The reward calculation and Willow Workflow case are original examples.
Previous: Decision-theoretic metareasoning Next: Decision tree