Is it the same as asking people which factor matters most?

No. It infers relationships from judgments across varying profiles, although direct questions can provide a useful comparison.

Must the model be linear regression?

No. The response type and judgment pattern may call for other models, including nonlinear or classification approaches.

Can it identify individual differences?

Yes. Repeated judgments can support person-specific or hierarchical modeling when the design supplies enough information.

AI Glossary / Statistics and probability / Intermediate

Policy capturing

Definition

Policy capturing is a research method for describing how people combine information when making judgments or decisions. Participants typically evaluate a series of scenarios in which selected attributes, called cues, vary systematically. Researchers model the relationship between those cues and the judgments, often using regression. Here policy means a person's judgment rule or pattern, not necessarily a government policy, formal organizational procedure or reinforcement-learning policy. The fitted model describes observed responses within the studied task.

What it means

People may say that several factors matter, yet their actual judgments can give those factors different weights. Policy capturing examines the pattern across repeated decisions rather than relying only on a direct question about importance. For example, a participant might rate alternatives differing in reliability and cost. A fitted model can summarize how those changes relate to the ratings. Such a model can reproduce choices without revealing the person's actual mental process. Its interpretation depends on the cue ranges, scenario realism, response scale and the model used.

How it works

Define a realistic decision task and select cues and levels relevant to it. Construct scenarios that allow the desired effects to be estimated without creating implausible combinations. Ask participants for judgments or choices, using an appropriate presentation order and sufficient observations. Fit a model matching the response type and account for repeated judgments from the same person. Individual-level models can reveal differences between decision makers; hierarchical models can represent both within-person and between-person variation. Check prediction on additional scenarios and consider nonlinearities or interactions instead of assuming every decision follows a purely additive rule.

Three practical examples

Supplier research: examine how decision makers trade off delivery reliability, price and support in hypothetical supplier profiles.
Workplace studies: investigate how several job attributes influence ratings of hypothetical roles.
Human - AI evaluation: compare how people respond to recommendation accuracy, explanation detail and uncertainty information, using a design that distinguishes those cues and measures the judgments of interest.

Think of it this way

Think of listening to many decisions to infer the pattern behind them. A single judgment gives little information about which cue mattered. Carefully varied scenarios provide contrasts, much like changing ingredients across several recipes to understand their contributions. The inferred pattern remains tied to the combinations and outcomes that were actually studied.

Worked example

Consider an original artificial participant who rates four supplier profiles. Reliability A and cost B are coded -1 for low and +1 for high. Suppose the ratings follow y = 50 + 10A - 5B. The four profiles (-1,-1), (-1,+1), (+1,-1) and (+1,+1) receive ratings 45, 35, 65 and 55. Average ratings are 60 at high reliability and 40 at low reliability, a 20-point contrast. For cost, the high and low averages are 45 and 55, a -10-point contrast. Under this coding, the regression coefficients are half those contrasts: +10 and -5. This constructed example is noiseless and deliberately additive. It does not demonstrate that reliability is twice as important in every real decision: the comparison depends on the specific low-to-high changes selected for both cues, and different ranges could produce different apparent importance.

Illustrative case study

In a fictional case, Harbor Purchasing wants to understand disagreements about supplier selection. Managers evaluate a set of realistic profiles, and analysts model each person's ratings. Some managers emphasize delivery reliability more strongly, while others respond sharply to cost only above a particular level. The team tests follow-up profiles and discusses the differences rather than averaging them into a single supposedly universal rule. The findings inform a review of decision criteria; they are not automatically converted into an AI policy that imitates every observed preference.

Common misconceptions and mistakes

Policy capturing is not mind reading. A statistical model of judgments need not describe the cognitive steps that produced them. Coefficients measured in different cue units are not directly comparable without context. Even standardized weights can depend on the cue distribution and design. Hypothetical ratings may not transfer perfectly to consequential real decisions. Combining all judgments as if they came from independent people can understate uncertainty. High predictive fit does not establish that the captured policy is fair, accurate or desirable, and an additive model can hide thresholds and cue interactions.

Business use

Businesses can use policy capturing to clarify decision criteria, investigate inconsistent judgments and design training or decision-support tools. Start with the question of whose judgments should be studied and whether the scenarios represent actual work. Preserve differences between participants when those differences are relevant. Compare captured preferences with external outcomes or agreed objectives before using them operationally. For AI development, distinguish learning how people currently decide from defining how a system ought to decide. A model that reproduces expert judgments may still reproduce expert errors or context-specific habits.

Check your understanding

A reliability coefficient is larger than a cost coefficient. Does this prove reliability matters more in every setting?

Answer: No. Coding, units, cue ranges, interactions and the task context affect the comparison.

Related terms

Conjoint analysis
Judgment analysis
experimental vignette
regression
cue weighting
multilevel model
decision support

Learn next

Learn how scenario design affects coefficient interpretation, then compare individual and hierarchical models and evaluate whether captured judgment patterns predict decisions on realistic new cases.

Sources

Aiman-Smith, Scullen and Barr, Conducting Studies of Decision Making in Organizational Contexts: https://journals.sagepub.com/doi/pdf/10.1177/109442802237117 Lafond and colleagues, Capturing Non-linear Judgment Policies Using Decision Tree Models of Classification Behavior: https://journals.sagepub.com/doi/abs/10.1177/1541931215591251

Previous: Policy Next: Policy evaluation