AI Glossary / Governance, ethics and safety / Intermediate

Risk control

Definition

A risk control is a measure intended to modify a risk, such as reducing the likelihood of an unwanted event, limiting its consequences, or improving detection and response. In AI systems, controls can be technical, procedural, or organizational. A control should be tied to a specific risk and supported by evidence that it works under the relevant conditions.

What it means

Naming a risk does not control it. A concern that an assistant might send information to the wrong destination needs a corresponding mechanism, such as restricted permissions, destination validation, or an appropriate approval process. The mechanism's purpose and limits should be explicit. Some controls prevent actions, others detect problems, and others help recover after an incident. They can complement one another, but their presence does not make residual risk disappear. A policy statement becomes an effective control only when people and systems can apply it consistently.

How it works

Identify a concrete risk scenario, including its causes, affected people, and consequences. Choose measures that interrupt relevant pathways or reduce impact. Assign ownership, implement the measures, and test their operation using realistic conditions. Monitor whether they remain effective after changes. Review possible side effects, such as denying legitimate users access or creating excessive review workload. Risk management also includes deciding whether to avoid a use or accept a justified residual risk; not every response is an added software filter. The NIST AI RMF emphasizes ongoing governance, assessment, measurement, and management rather than a one-time checklist.

Three practical examples

• An agent receives read-only access when its task requires analysis but no modification, reducing the scope of unauthorized changes.
• A document workflow routes uncertain extractions to a trained reviewer who has enough information and authority to correct them.
• A monitoring process detects stale inputs and pauses affected recommendations until the data issue is resolved. Each control addresses a particular pathway and needs its own effectiveness check.

Think of it this way

Think of a fire door that limits the spread of a fire. Its value depends on being correctly installed, kept usable, and suited to the building; a label saying fire door is not enough. The analogy is limited because many AI risks involve ambiguous outputs and human interpretation rather than a single physical barrier. Controls may require judgment, and their interactions can be difficult to predict.

Worked example

Suppose a fictional extraction process produces 100 erroneous records in a test batch. A validation rule detects 70 of them, leaving 30 undetected. A subsequent reviewer catches 20 of those remaining errors, leaving 10. The combined observed detection is 90 of 100, or 90%, for this test. It is not valid to assume the same result merely from two separately advertised detection rates, because the controls may catch the same cases. The team also needs to count correct records wrongly rejected and the review effort required. This example shows why layered controls should be tested together on the actual workflow. The remaining ten errors are residual failures, not evidence that the controls have eliminated the risk.

Illustrative case study

Fictional case: Cedar Agents adds a confirmation screen before external actions. Testing shows that users approve confusing requests without understanding what will be sent. The team improves the action summary, restricts destinations, and tests whether users can identify inappropriate requests. It also examines whether some actions should be blocked outright. This invented case illustrates that a human-in-the-loop control needs usable information and a meaningful decision, not simply an extra button.

Common misconceptions and mistakes

A control is not effective just because it exists in a policy or product feature list. A detector without a response owner may only generate ignored alerts. Two controls can share a failure mode, and a fallback can introduce different risks. High effectiveness on known tests does not guarantee protection against every new scenario. Risk acceptance is also not the same as risk reduction: documenting that a risk is accepted does not lower its likelihood. Controls should be assessed against specific objectives rather than counted as if a larger number automatically meant greater safety.

Business use

Maintain a link from each significant risk to its controls, owners, evidence, and residual limitations. Define what successful operation looks like and how failure will be detected. Include staff capacity, training, and escalation in the design of procedural controls. Test controls after relevant changes and investigate whether they shift burdens to users or other teams. Retire obsolete measures and avoid accumulating alerts that nobody can act on. Where evidence shows a use cannot be adequately controlled, reconsider its scope or deployment rather than relying on increasingly vague assurances.

People also ask

Is a risk control always technical?

No. It can involve procedures, training, authority, access arrangements, or organizational decisions.

Does adding a control eliminate the risk?

Usually not. Its effectiveness and remaining limitations must be assessed.

Can a control create another problem?

Yes. For example, a strict filter may block legitimate tasks or a review queue may introduce harmful delays.

Check your understanding

Two checks each detect many errors in separate tests. Can their detection rates be combined as if their failures were independent?

Answer: Not without evidence. Their errors may overlap or depend on one another. Test the combined workflow and state the assumptions.

Related terms

• Least privilege
• Incident response
• risk mitigation
• safeguard
• preventive control
• detective control
• residual risk
• control effectiveness
• human oversight
• risk acceptance

Learn next

Write one risk scenario for an AI workflow and identify its causal steps. Place a proposed control at a specific step. Define how to test whether it works, what it could miss, and what new burden or failure it might introduce.

More in Governance, ethics and safety

• Responsible AI - Responsible AI is the practice of developing, deploying, using, and governing AI with attention to its effects on people, society, and the environment.
• Retention limit - A retention limit is a defined boundary on how long data or records are kept for a particular purpose.
• Retraining - Retraining updates a previously trained machine-learning model through another training process using different training data.
• Risk-averse - Risk-averse describes a decision maker who prefers a certain outcome to an uncertain prospect with the same expected outcome value.
• Risk-neutral - Risk-neutral describes a decision maker who compares uncertain prospects by their expected value in a specified outcome measure, without an additional...
• Risk-seeking - Risk-seeking describes a decision maker who prefers an uncertain prospect to a certain outcome with the same expected value.

Sources

NIST AI RMF Core, governance and management of AI risks: https://airc.nist.gov/airmf-resources/airmf/5-sec-core/ NIST CSRC, control definitions and source contexts: https://csrc.nist.gov/glossary/term/control NIST AI RMF Playbook, management practices: https://airc.nist.gov/airmf-resources/playbook/manage/

Previous: Risk-averse Next: Risk-neutral