AI Glossary / Governance, ethics and safety / Intermediate

Sensitivity analysis

Definition

Sensitivity analysis examines how a model's outputs or a decision's conclusions change when inputs, parameters, or assumptions are varied. It helps identify influential factors and fragile conclusions. In AI, it can be applied to predictions, evaluation metrics, operating thresholds, and the assumptions used to estimate benefits or harms.

What it means

A forecast might change sharply when one input moves slightly, or remain stable across a wide range. A business recommendation might depend strongly on an uncertain review-time estimate. Sensitivity analysis makes those dependencies visible rather than hiding them inside a single result.
The term does not mean the classification metric called sensitivity, which is also known as recall or true-positive rate. Here the focus is a what-if analysis of a model or conclusion. The result describes dependence within the analysis; it does not automatically establish what would happen after a real-world intervention.

How it works

Choose the output or conclusion of interest and identify uncertain inputs or assumptions. Set plausible ranges and, where appropriate, probability distributions. Decide how to vary them. Local methods examine behavior near a reference setting; global methods explore a broader input space and can assess interactions.
Run the model for the chosen combinations and summarize changes in output, rankings, thresholds, or variability. Interpret the results in light of the selected ranges and dependencies. If inputs are correlated, independently varying them may generate unrealistic scenarios. Repeat or redesign the analysis when an influential assumption has been poorly specified.

Three practical examples

• A forecasting team varies uncertain demand inputs to see which assumptions most affect a staffing recommendation.
• An AI service owner examines how different review rates and handling times change the staff capacity required after launch.
• A model evaluator varies an operating threshold and records how false positives and false negatives change, rather than reporting a single threshold without context.

Think of it this way

Think of adjusting the controls on a mixing desk and listening for changes in the output. Some controls have large effects, and combinations can matter. The analogy has limits: the real system may contain dependencies and feedback that are absent from the model, so simulated responsiveness is not a complete causal explanation of the world.

Worked example

A fictional AI workflow receives 1,000 items per day. Its estimated human-review requirement is items times review fraction times minutes per review. At a 10% review fraction and 4 minutes per review, the requirement is 1,000 times 0.10 times 4, or 400 minutes per day.
Holding handling time fixed, increasing the review fraction to 15% raises the requirement to 600 minutes. Holding the fraction at 10%, increasing handling time to 6 minutes also gives 600 minutes. Changing both produces 900 minutes, not 800: the inputs multiply, so their joint effect exceeds the sum of the two separate 200-minute increases. This simple analysis identifies a capacity concern but does not assign probabilities to the scenarios. The chosen ranges need evidence before they support planning.

Illustrative case study

At fictional Juniper Translation, managers assess whether an AI-assisted workflow can handle a larger backlog. An initial estimate uses one correction rate and one average correction time. Analysts vary both and discover that staffing requirements depend strongly on the combination.
The team collects more information about difficult documents and separates them from routine cases in the next analysis. It retains the earlier model as a transparent baseline. The fictional case illustrates using sensitivity results to guide measurement; it does not claim that the workflow is cost-effective or that the selected scenarios cover every operational uncertainty.

Common misconceptions and mistakes

A sensitivity ranking is not a universal ranking of importance. It depends on the output, input ranges, distributions, model, and method. A factor held nearly fixed can appear unimportant even when a larger plausible change would matter greatly.
Do not mistake a small local derivative for global stability. Thresholds and nonlinear interactions can create large changes elsewhere. One-at-a-time variation is easy to explain but can miss interactions. Also distinguish uncertainty analysis, which describes uncertainty in outputs, from sensitivity analysis, which investigates how inputs or assumptions contribute to changes in those outputs.

Business use

Sensitivity analysis helps teams decide where to improve data, which assumptions need explicit approval, and whether a recommendation remains useful under plausible alternatives. It is especially valuable when a business case rests on uncertain adoption, error, or review estimates. Present the assumptions and ranges alongside the conclusion so that decision makers can see what would need to change for the recommendation to reverse.

People also ask

Does sensitivity analysis prove causation?

No. It shows how the modeled output changes under specified variations; causal interpretation requires additional assumptions and evidence.

Must it use a complex statistical method?

No. A transparent scenario table can be useful, although complex models may need methods that capture interactions and broad input variation.

Can different methods give different rankings?

Yes. Local effects, variance contributions, and decision-threshold changes answer different questions.

Check your understanding

A one-at-a-time analysis changes two inputs separately, each adding 200 minutes of review work. Can their combined effect automatically be reported as 400 extra minutes?

Answer: No. Interactions can alter the joint effect. In the worked example, changing both adds 500 minutes because the inputs multiply.

Related terms

• Robustness
• uncertainty analysis
• scenario analysis
• local sensitivity
• global sensitivity
• interaction effect
• model assumptions

Learn next

Choose one decision model and list its three least-certain assumptions. Vary each separately, then vary them together where combinations are plausible. Record whether the decision changes, not only whether the numerical output changes. This connects technical sensitivity to the practical choice being made.

More in Governance, ethics and safety

• Self-healing system - A self-healing system detects certain faults and automatically takes predefined or policy-governed actions to restore acceptable operation.
• Semantic mapping - Semantic mapping connects items according to their meaning, rather than relying only on matching words or formats.
• Sensitive attribute - A sensitive attribute is a characteristic of a person that deserves particular care because of legal, ethical, social, or personal considerations.
• Shallow learning - Shallow learning is an informal term for machine-learning approaches that do not rely on a long hierarchy of learned transformations.
• Sibyl attack - A Sybil attack occurs when one controlling entity presents multiple identities to gain influence that a system intends to distribute among independent...
• Simulation component metadata - Simulation component metadata describes an individual component used in a simulation.

Sources

Sources: European Commission Joint Research Centre, Sensitivity Analysis methods: https://joint-research-centre.ec.europa.eu/sensitivity-analysis-samo/methods_en . SALib official project documentation, analysis workflow and indices: https://github.com/SALib/SALib/blob/main/docs/user_guide/basics.rst . NIST AI RMF Playbook, Measure: https://airc.nist.gov/airmf-resources/playbook/measure/ .

Previous: Sensitive attribute Next: Sensitivity study