Is AI risk management only a technical task?

No; operational knowledge and affected people's perspectives can reveal consequences that model tests miss.

Does it require a monetary value for every harm?

No. Some consequences need other quantitative or qualitative treatment.

Is NIST AI RMF a guarantee of compliance?

No; its voluntary framework does not replace applicable obligations or sound judgment.

AI Glossary / Governance, ethics and safety / Advanced

AI risk management

Definition

AI risk management is the ongoing work of identifying, assessing, responding to and monitoring risks associated with developing or using AI. It considers a system's purpose, operating environment, affected people and potential consequences. The aim is to support informed decisions and reduce unacceptable harm while enabling useful applications, without promising zero failures.

What it means

A model error is not a complete description of risk. The same incorrect output can have different consequences depending on who sees it, what action follows and whether a correction is possible. An inaccurate internal draft reviewed by an editor differs from an incorrect instruction acted on automatically.
Risk management therefore concerns the system in use, including people, data, suppliers and surrounding processes. A useful assessment states a scenario: what could happen, under which conditions, to whom, and with what consequences. Listing broad words such as bias or hallucination without connecting them to an actual outcome leaves important questions unanswered.

How it works

NIST AI RMF 1.0 organizes the work around Govern, Map, Measure and Manage. They support iteration: establish responsibilities, understand context, collect relevant evidence, choose responses and revisit decisions as conditions change. They are not simply four boxes completed once in sequence.
Responses may change the design, restrict a use, add a control, improve monitoring or stop an unacceptable activity. Residual risk is what remains after treatment. Recording it makes limitations visible and supports an accountable decision about further action.
Measures should reflect the scenario and their uncertainty. NIST cautions that risks can be difficult to measure and that measurements from controlled settings may differ from operational experience. Lack of a reliable number is not evidence that risk is absent. Qualitative evidence and unresolved questions can be important parts of the assessment.

Three practical examples

A support team studies the risk that an assistant gives outdated return instructions. It identifies who maintains the source material and how incorrect replies are detected and corrected.
A manufacturer examines what follows when an inspection model misses a defect. It evaluates the entire inspection workflow, including sampling, human checks and the handling of uncertain results.
A learning publisher considers inaccurate AI-generated explanations. It connects source review, subject expertise and correction procedures to specific reader harms rather than treating fluent writing as sufficient evidence of reliability.

Think of it this way

Think of checking a proposed journey: the destination, conditions, possible failures, available precautions and what happens if those precautions fail. A checklist can help organize the work, but the relevant judgment depends on the actual route. A successful trip yesterday does not establish that today's changed conditions are acceptable.

Worked example

Consider a fictional internal help service handling 10,000 requests per month. For one error type, assume 2% of requests produce an incorrect instruction. That corresponds to 10,000 × 0.02 = 200 such instructions in expectation.
Suppose a review control catches 90% of those particular incorrect instructions before delivery. The expected remaining count is 200 × (1 − 0.90) = 20. Here the 90% figure is explicitly conditional on an instruction already being incorrect; it is not the reviewer's overall accuracy.
If each remaining error is assumed to cause exactly 15 minutes of additional internal work, expected rework is 20 × 15 = 300 minutes, or five hours. This estimate excludes other errors and consequences. It is not a complete measure of safety, financial loss or impact on people.
Now suppose the catch-rate estimate is uncertain and could be 80% rather than 90%. The expected residual count becomes 40 and the rework becomes ten hours. The exercise reveals sensitivity to the control assumption. Before using it to make an operating decision, the team needs evidence that the estimates apply to current requests, plus a separate assessment of any rare but more serious outcomes.

Illustrative case study

Fictional Hazel Help approves an assistant for staff to consult when answering routine product questions. Its initial assessment assumes every answer is checked against the source article before reaching a customer.
As workload grows, staff begin copying answers without that check. A monitoring report still lists the review control as present because the written procedure has not changed. An investigation finds that the operating assumption, rather than the model version, has changed.
The team revises the assessment using observed work practices. It examines whether a more usable verification step is feasible, assigns responsibility for checking its effectiveness and records what remains unresolved. It compares the revised workflow with the previous process before deciding how to continue. The case shows why an assessment must track actual conditions and control performance, not just the existence of a policy.

Common misconceptions and mistakes

Equating risk with model error rate: consequences and exposure also matter.
Treating a control's existence as proof of effectiveness: it may be unused, unreliable or poorly matched to the failure.
Converting uncertain judgments into precise-looking scores without explanation: numerical presentation does not remove uncertainty.
Assuming supplier assurances cover the deployment context: local integration can introduce different risks.
Forgetting residual risk after mitigation: a reduced risk can still require action or an operating limit.

Business use

Create a record that connects each material scenario to evidence, an owner, a response and a review trigger. Prioritize work by relevant consequences and uncertainty, not only by what is easiest to count. Include the benefits and realistic alternatives in the decision, while preserving applicable constraints. Revisit assessments after incidents, significant updates or changes in who uses the system and what actions follow its outputs.

Check your understanding

Why are 20 errors expected after review in the example?

Answer: The assumed control misses 10% of 200 incorrect instructions.

Why is that not a complete risk estimate?

Answer: It covers only one error type and one simplified consequence.

What changed in Hazel Help's case?

Answer: Actual review behavior changed, undermining the earlier control assumption.

Related terms

AI governance
Impact assessment
Risk tolerance
Uncertainty
residual risk
control effectiveness
incident management
monitoring

Learn next

Write one concrete failure scenario for an AI service. Identify the affected people, evidence for likelihood and consequences, proposed control, remaining uncertainty and the event that would trigger reassessment.

Uncertainty

Sources

NIST AI RMF 1.0, Framing Risk, measurement challenges, context and residual risk; revision is noted as in progress. https://airc.nist.gov/airmf-resources/airmf/1-sec-risk/ NIST AI RMF 1.0, AI RMF Core, Govern, Map, Measure and Manage. https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ OECD AI Principles, accountability and continuing lifecycle risk management. https://oecd.ai/en/dashboards/ai-principles/P9 The help-service calculation and Hazel Help case are original teaching illustrations.

Previous: AI principles Next: AI safety