Can a risk-averse agent choose a risky action?

Yes, when its expected utility exceeds that of available alternatives.

Does risk aversion imply inaccurate probability estimates?

No. It concerns valuation of outcomes, although probability errors can separately affect decisions.

Is lower variance always preferred?

Not as an unrestricted rule for every pair of distributions. The full outcome distribution and utility function matter; equal-mean comparisons require appropriate conditions.

AI Glossary / Governance, ethics and safety / Intermediate

Risk-averse

Definition

Risk-averse describes a decision maker who prefers a certain outcome to an uncertain prospect with the same expected outcome value. In expected-utility models, this preference is represented by a concave utility function over the relevant benefit or wealth: additional gains still help, but their incremental value diminishes.

What it means

Suppose two arrangements deliver the same average number of useful resource units. One provides a fixed amount; the other sometimes provides much more and sometimes much less. A risk-averse decision maker values avoiding the downside enough to favor the fixed arrangement. This is a statement about preferences under specified uncertainty, not a claim that the uncertain option has a lower average.
For AI, risk aversion can be encoded in how an agent evaluates possible outcomes. A planning system can prefer dependable task completion even when a more variable strategy has an attractive expected reward. The preference must be specified; it is not automatically supplied by a capable model.

How it works

An expected-utility calculation first lists possible outcomes and their probabilities. It then converts each outcome into utility, a numerical representation of preference, and averages those utilities using the probabilities. The important order is to transform each outcome before averaging.
For an increasing concave utility function, utility of the average outcome is at least as great as average utility of the uncertain outcomes. With strict concavity and genuine variation, the preference is strict. This mathematical relationship explains why merely maximizing an average quantity can give different choices. The relevant range, starting resources, and other consequences all matter when choosing a utility model.

Three practical examples

A scheduling assistant values a dependable amount of completed work more than a variable plan with the same expected output, because shortfalls disrupt the next team.
A simulated robot planner penalizes unreliable completion sufficiently to choose a steadier route over a faster-on-average alternative.
A computing team chooses an arrangement with predictable available capacity when uncertain shortfalls would interrupt a time-sensitive internal workload.

Think of it this way

Think of packing supplies for a trip where running short hurts more than an equivalent surplus helps. Predictability can be valuable even when two supply arrangements have the same average. The analogy is limited: a specific shortfall rule may involve a hard constraint or a threshold, so it does not by itself establish a concave utility function across all possible quantities.

Worked example

Consider an artificial resource-allocation choice with positive benefit units. Plan A delivers 25 units for certain. Plan B delivers 9 or 49 units with equal probability. Its expected benefit is 0.5 times 9 plus 0.5 times 49, which equals 29 units.
Let utility be the square root of benefit. Plan A has utility 5. Plan B has expected utility 0.5 times 3 plus 0.5 times 7, also 5. The decision maker is indifferent despite Plan B's higher expected benefit. Its certainty equivalent is 25 units, and the difference between expected benefit and that equivalent is 4 units. A certain 29 units would have utility about 5.39 and would be preferred to Plan B. These invented preferences illustrate the model; they do not establish how a real organization should value capacity.

Illustrative case study

At fictional Arbor Analytics, an AI planner compares two ways to schedule an overnight reporting job. Its original objective rewards average throughput, and it repeatedly proposes schedules with substantial variation. Operations staff explain that delayed reports have consequences that a raw throughput average omits.
The team models the complete outcomes, checks whether a utility representation matches its preferences, and compares recommendations under several plausible settings. It separately preserves mandatory completion constraints. The case shows why an objective may need revision; it does not claim that adding a concave transformation guarantees a safe or operationally optimal system.

Common misconceptions and mistakes

Risk-averse does not mean unwilling to take any risk. A sufficiently rewarding uncertain option can still be preferred. Nor is it the same as pessimism: two decision makers can agree on probabilities while valuing the resulting prospects differently.
Do not confuse aversion to variability with uncertainty about whether the probability estimates are trustworthy. Also avoid applying the concavity rule blindly to costs: a cost formulation has different signs and interpretation from an increasing utility of benefits. A person's behavior in one situation does not establish a permanent attitude across every domain.

Business use

The concept helps teams discuss service reliability, planning objectives, and downside exposure explicitly. Before introducing a risk-sensitive objective, identify whose preferences it represents, what is being measured, and which constraints remain non-negotiable. Test sensitivity to plausible probability errors and utility choices. Otherwise, a numerical preference can conceal a managerial assumption behind an apparently objective recommendation.

Check your understanding

An agent rejects an uncertain 9-or-49-unit outcome in favor of a certain 29 units under square-root utility. Is it failing to maximize its objective?

Answer: No. Both have expected benefit 29, but the certain option gives about 5.39 utility versus expected utility 5. The choice is consistent with risk aversion.

Related terms

Learn next

Compare utility of an expected outcome with expected utility of outcomes. Calculate both for a two-outcome example, then change the utility curve. This separates uncertainty in what happens from preferences about what happens and makes an AI planning objective easier to inspect.

Expected utility

Sources

Sources: Russell and Norvig, AIMA teaching slides, Rational decisions: https://aima.cs.berkeley.edu/slides-pdf/chapter16.pdf . MIT OpenCourseWare, Jonathan Gruber, Uncertainty: https://ocw.mit.edu/courses/14-01sc-principles-of-microeconomics-fall-2011/pages/unit-6-topics-in-intermediate-microeconomics/uncertainty/ . UC Berkeley CS188 original course notes, indexed explanation of risk attitudes: https://www-inst.eecs.berkeley.edu/~cs188/fa20/assets/notes/note03.pdf .

Previous: Riemann manifold Hamiltonian Monte Carlo Next: Risk control