AI Glossary / Governance, ethics and safety / Intermediate

Relative risk or risk ratio

Definition

Relative risk, or risk ratio, compares the probability of a specified event in one group or condition with its probability in another. It is calculated by dividing the first risk by the reference risk. A ratio of 1 means equal risks for that event; a ratio above 1 means higher risk in the numerator group, and a ratio below 1 means lower risk.

What it means

The ratio answers a relative question: how many times as likely is the event under one condition compared with the reference? It does not say how common the event is in absolute terms. A doubling from 1% to 2% and a doubling from 20% to 40% both produce a risk ratio of 2, but their absolute differences are very different. The event, time horizon, groups, and reference direction must be stated. In AI evaluation, the same arithmetic can compare probabilities of a defined failure or outcome, provided the measurement design supports those probabilities.

How it works

Estimate the event probability in each relevant group using suitable denominators and comparable observation conditions. Divide the numerator group's probability by the reference group's probability. For a simple complete follow-up dataset, these probabilities can be event counts divided by group sizes. More complex designs, censoring, or confounding may require appropriate statistical methods. A risk ratio describes an association unless the design and assumptions support a causal interpretation. It is distinct from an odds ratio, which compares odds, and a hazard ratio, which concerns instantaneous event rates among those still at risk.

Three practical examples

• An AI service evaluation compares the probability of an incorrect extraction for two document formats under the same scoring rule.
• A controlled workflow study compares the probability of a defined task failure with and without an assistance feature.
• A reliability review compares the probability of at least one outage during a fixed period for two operating configurations. These examples require consistent event definitions and comparable observation, not merely two convenient percentages.

Think of it this way

Think of comparing two jars by the fraction of red tokens inside each. If one jar has twice the red-token fraction, its relative risk for drawing red is 2. The analogy is limited because real groups may differ in many ways, data can be incomplete, and event probabilities can change over time. A ratio alone does not establish why the difference exists or what would happen after an intervention.

Worked example

Suppose a hypothetical evaluation observes errors in 40 of 200 cases for Configuration A and 20 of 200 for Configuration B. The risks are 20% and 10%, so A relative to B has a risk ratio of 0.20/0.10 = 2. The absolute risk difference is 10 percentage points, and the relative increase is 100%. The odds ratio is different: (40/160)/(20/180) = 2.25. If the event is changed to success, the probabilities become 80% and 90%, giving a success risk ratio of 0.80/0.90, approximately 0.889. That is not the reciprocal of the error risk ratio. Reversing the groups for the same error event would instead give 0.5. These distinctions prevent several common reporting mistakes.

Illustrative case study

Fictional case: Fern Documents announces that a new capture process halves extraction-error risk. Reviewers ask for the underlying numbers and learn that the illustrative pilot changed from 2% to 1%. The report is revised to include the one-percentage-point absolute difference, sample sizes, uncertainty, and the scope of the test. This invented case illustrates clearer communication of a relative result. It does not imply that small absolute differences are unimportant; their value depends on frequency, consequences, and cost.

Common misconceptions and mistakes

A risk ratio is not a percentage-point difference, and an odds ratio should not automatically be described as a risk ratio. A ratio of 2 does not mean that everyone has a 200% probability. Probabilities cannot exceed 1, so a constant multiplicative ratio cannot apply across arbitrary baseline risks; doubling a baseline of 70% would be impossible. If the reference event risk is zero, the simple ratio is undefined rather than an ordinary finite estimate. Sampling uncertainty matters, especially with rare events or small groups, and a large ratio can coexist with a very small absolute difference.

Business use

Report both group risks and the ratio, along with event counts, observation period, and uncertainty. Explain whether the comparison is adjusted and what assumptions support it. Use absolute differences to help decision-makers understand scale. Check that changing the event from failure to success has not silently altered the measure's meaning. Avoid causal language for an observational comparison without appropriate support. When comparing AI systems, also examine severity and distribution of failures: a lower overall event probability may conceal more serious or concentrated errors.

People also ask

Does a risk ratio of 1 prove the groups are identical?

No. It indicates equal measured risks for the specified event, not equality on every outcome or certainty about the population.

Is a risk ratio the same as an odds ratio?

No. They compare different quantities and can differ substantially.

Can a risk ratio remain constant at every baseline risk?

Not for arbitrary baselines when the ratio exceeds 1, because the resulting probability cannot exceed 100%.

Check your understanding

Error risk decreases from 4% to 2%. What are the new-to-old risk ratio and absolute decrease?

Answer: The ratio is 0.02/0.04 = 0.5, and the absolute decrease is 2 percentage points. This is a 50% relative reduction, not a 50-point reduction.

Related terms

• Odds ratio
• Hazard ratio
• Confidence interval
• Confounding
• Causal inference
• absolute risk
• risk difference
• baseline risk
• event probability

Learn next

Calculate the risk ratio and risk difference for two pairs of probabilities with the same relative change but different baselines. Reverse the groups, then redefine the event as its complement. Explain which quantities change and why the reporting labels matter.

More in Governance, ethics and safety

• Reference class - A reference class is a group of cases used as the comparison basis for a judgment, estimate, or probability about a particular case.
• Referential transparency - Referential transparency concerns whether an expression can be replaced by an equivalent expression or value without changing the meaning or relevant...
• Regularized empirical risk minimization - Regularized empirical risk minimization, or RERM, is a model-training principle that balances average loss on training data with a penalty expressing a...
• Reliability - Reliability is the ability of a system to perform its required function without failure over a specified period and under stated conditions.
• Reporting bias - Reporting bias is a systematic difference between what people record or describe and what actually occurs in the world.
• Representation Bias - Representation bias occurs when the data used to develop or evaluate a system inadequately represent the population or conditions to which the system is...

Sources

Frank Harrell, Glossary of Statistical Terms, relative risk entry: https://hbiostat.org/glossary/ CDC, Principles of Epidemiology, measures of association: https://archive.cdc.gov/www_cdc_gov/csels/dsepd/ss1978/lesson3/section5.html CDC, analyzing and interpreting field-study data: https://www.cdc.gov/field-epi-manual/php/chapters/analyze-interpret-data.html

Previous: Relative pose Next: Relative standard error