AI Glossary / Governance, ethics and safety / Intermediate
Risk-seeking
Definition
Risk-seeking describes a decision maker who prefers an uncertain prospect to a certain outcome with the same expected value. Within expected-utility theory, an increasing convex utility function over the relevant benefit can represent this preference. The attraction concerns the pattern of possible outcomes, not merely a mistaken belief that success is certain.
What it means
A risk-seeking AI objective can favor a strategy with a chance of an unusually large benefit over a dependable strategy with the same average benefit. The extra value assigned to larger outcomes outweighs the reduction in value associated with smaller outcomes under that objective.
Risk-seeking is a technical description of preferences in a specified range. It is not a blanket personality label, a synonym for innovation, or a recommendation to expose people to harm. An organization may permit adventurous exploration inside a simulation while imposing strict constraints on the system's real-world actions.
How it works
List the outcomes of each action and their probabilities. Transform each outcome into utility, then calculate the probability-weighted average utility. With increasing convex utility, the gain in utility from an additional benefit unit grows as the benefit increases.
Consequently, the expected utility of a genuinely variable prospect can exceed the utility of its mean. This statement assumes the expected-utility model and the specified convexity conditions. It does not mean that every distribution with a larger numerical variance is preferred to every other distribution. The complete outcome distributions, the utility function, and feasibility constraints still determine the comparison.
Three practical examples
- A game-playing agent is deliberately rewarded for rare high scores using a convex score transformation, so it can favor volatile strategies.
- A research simulator compares exploration policies that place extra value on finding a very strong candidate, while keeping experiments inside a controlled environment.
- A planning demonstration contrasts a dependable output with a two-outcome prospect to show how a change in utility curvature changes the chosen action.
Think of it this way
Think of a fictional contest in which larger point totals receive disproportionately larger bonuses. A variable route can become attractive because the high outcome earns much more credit. The analogy has a limit: a contest's actual threshold or bonus schedule may have mixed curvature, so it need not produce risk-seeking preferences over every possible score range.
Worked example
An artificial agent chooses between 10 benefit units for certain and a prospect giving 6 or 14 units with equal probability. Both have expected benefit 10 because 0.5 times 6 plus 0.5 times 14 equals 10.
Suppose its utility is the square of the nonnegative benefit. The certain outcome has utility 100. The uncertain prospect has expected utility 0.5 times 36 plus 0.5 times 196, which equals 116. The agent therefore prefers the uncertain prospect. A certain benefit of the square root of 116, about 10.77 units, would give the same utility as the prospect. This certainty equivalent exceeds its expected benefit. The example uses an invented utility function over a limited nonnegative domain; extending the same square function to negative benefits would break the assumption that more benefit is always better.
Illustrative case study
At fictional Summit Lab, a simulation agent receives a bonus that rises sharply with the quality of its best discovered design. The agent chooses trials with uneven results, even though a steadier method has the same average score. Reviewers initially interpret this as faulty probability estimation.
They inspect the reward function and discover that the preferences are consistent with the assigned objective. They then compare the intended purpose with the behavior and retain explicit limits on experiment cost and permitted actions. The case illustrates objective diagnosis, not evidence that the strategy improves research or is suitable for deployment.
Common misconceptions and mistakes
Do not confuse risk-seeking with optimism. Optimism concerns beliefs about likely outcomes; risk preference concerns how those outcomes are valued. A person or agent can use accurate probabilities and still prefer the uncertain prospect.
Do not assume exploration in machine learning proves risk-seeking either. An agent can explore because information improves future decisions under an otherwise risk-neutral or risk-averse objective. Finally, choosing a risky option with a sufficiently higher expected benefit does not establish risk-seeking. The defining comparison holds expected value constant, or otherwise examines the complete preference model.
Business use
The term helps explain why a reward function may encourage volatile behavior. Teams designing agents should examine nonlinear bonuses, best-case targets, and incentives that overlook repeated failures or costs borne by others. Compare expected benefit, downside outcomes, and expected utility separately. If the observed preference is unintended, revise the objective and constraints rather than relying on a vague instruction to be sensible.
People also ask
Does risk-seeking mean choosing the worst downside available?
No. It describes a preference for particular uncertain prospects, while higher benefits and other constraints can still matter.
Is an exploratory AI automatically risk-seeking?
No. Information gathering may be valuable without any preference for variability itself.
Can the same decision maker be risk-seeking in one range and risk-averse in another?
Yes. A utility representation can have different curvature in different ranges, and context can change the relevant preferences.
Check your understanding
An agent chooses a prospect with expected benefit 20 instead of a certain benefit 5. Does that alone show risk-seeking?
Answer: No. The much larger expected benefit can make the prospect attractive even without a preference for variability. Compare equal-mean prospects or examine its utility model before assigning the label.
Related terms
Learn next
Draw three utility curves over the same positive benefit range: linear, concave, and convex. For one two-outcome prospect, compare expected utility with utility of the mean. Then add a separate constraint to see why preferences and permitted actions are different parts of a decision system.
More in Governance, ethics and safety
- Risk set - A risk set is the collection of study subjects or units that are being observed and are eligible to experience a specified event at a particular time.
- Risk stratification - Risk stratification is the process of assigning cases to groups based on estimated risk of a specified outcome.
- Risk tolerance - Risk tolerance is the amount and type of uncertainty or potential harm that an organization or person is prepared to accept while pursuing an objective.
- Robot ethics - Robot ethics examines the moral questions raised by designing, building, deploying, and interacting with robots.
- Robust AI - Robust AI is AI designed and evaluated to keep functioning acceptably when inputs, operating conditions, or other relevant circumstances vary.
- Robustness - Robustness is the ability of a model or system to maintain acceptable behavior when specified conditions change or disturbances occur.
Sources
Sources: Russell and Norvig, AIMA Rational decisions slides: https://aima.cs.berkeley.edu/slides-pdf/chapter16.pdf . MIT OpenCourseWare, Jonathan Gruber, Uncertainty: https://ocw.mit.edu/courses/14-01sc-principles-of-microeconomics-fall-2011/pages/unit-6-topics-in-intermediate-microeconomics/uncertainty/ . UC Berkeley CS188 original notes, indexed linear, square-root, and squared utility discussion: https://www-inst.eecs.berkeley.edu/~cs188/fa20/assets/notes/note03.pdf .
Previous: Risk-neutral Next: Risk set