AI Glossary / Governance, ethics and safety / Intermediate

Post-hoc explanation

Definition

A post-hoc explanation is an explanation produced by analyzing an already developed AI model or its behavior. It aims to make an output or pattern understandable without necessarily making the model's original decision process inherently transparent.

What it means

The phrase means that explanation is added through a separate analysis, rather than guaranteed by the model's structure. A method might highlight influential input features, build a simpler approximation or show how an output changes when selected inputs change.

Some explanations concern one prediction; others describe broader behavior across many examples. A useful explanation must state which question it answers and the range over which its claims apply. A local approximation of one classification should not be presented as a complete description of the model.

Post-hoc does not automatically mean inaccurate or deceptive. It does mean that the explanation needs its own evaluation. A plausible story can be easy to understand while failing to reflect the system. Conversely, technically detailed evidence may be accurate but unhelpful to the intended audience.

How it works

Choose an explanation method according to the model, available access and the user's question. A model-agnostic method can query outputs while changing inputs; a model-specific method can use details of the architecture or internal computations. Different methods can answer different questions even when they produce similar-looking charts.

For example, a local surrogate method samples variations around an input, observes the model's responses and fits a simpler model to approximate nearby behavior. Its usefulness depends on the variations being meaningful and the approximation fitting the relevant neighborhood. Feature dependence and the chosen reference or baseline can affect the result.

Evaluate fidelity, scope and usefulness separately. Check whether the explanation tracks the model's behavior, whether small irrelevant changes make it unstable and whether the intended audience interprets it correctly. An association in the model is not automatically a real-world cause or a practical action someone can take. Explanations should also disclose important uncertainty and limitations.

Three practical examples

• An analyst uses a local explanation to investigate which words influence a support-ticket classifier for a particular message.
• A quality team examines feature contributions across many examples to investigate whether a forecasting model relies on an unexpected input.
• A service designer tests whether an explanation helps users distinguish a model estimate from a guaranteed outcome.

Think of it this way

Think of drawing a simplified map after studying a complicated transport network. The map can help answer a particular question, such as how to reach one station, while omitting many details of the actual network.

The analogy does not imply the map necessarily records the exact route taken on a particular journey. A post-hoc explanation must be checked against the behavior it claims to describe.

Worked example

A convincing approximation with a limited scope

In an invented example, a classifier assigns a support message an urgency score of 0.78. A local surrogate assigns the same message 0.76 and identifies a deadline phrase as an important positive contribution.

The difference at that one input is 0.02. That close agreement is useful but insufficient: a surrogate could match one point while behaving differently nearby. The analyst therefore checks a separate set of relevant variations and records the approximation errors, including cases where the models disagree about urgency.

The finding supports an investigation into the deadline phrase's role in this local context. It does not prove that the phrase causes real urgency, that removing it is an appropriate user action or that the same explanation applies to every message.

Illustrative case study

Fictional case: Fern Logistics buys a forecasting tool that supplies an attractive feature-importance chart. Managers initially interpret the chart as showing which business changes would improve demand.

A review establishes that the chart describes predictive associations under a particular explanation method. Some features are correlated, and the chart does not evaluate interventions. The team revises its presentation, adds the method's scope and uses further analysis for questions about business actions.

The chart remains useful for examining model behavior. The fictional lesson is to match the explanation to the question, rather than turning a diagnostic display into an unsupported causal recommendation.

Common misconceptions and mistakes

• A readable explanation must be faithful. Clarity and fidelity are different properties and both require attention.
• Feature importance identifies real-world causes. It describes relationships under a method and does not by itself establish causal effects.
• One explanation captures everything. Local, global, example-based and counterfactual approaches provide different perspectives.

Business use

Businesses should ask explanation providers what the output represents, how it is computed and what evidence supports its accuracy for the intended use. Record the model version, relevant settings and data context so that a later reviewer can interpret the result.

Test explanations with the people expected to use them. A developer investigating a failure may need different evidence from someone trying to understand an affected service decision. Avoid using a persuasive explanation as a substitute for correction, appeal routes or broader evaluation of the system's consequences.

People also ask

Does post-hoc mean after every individual prediction?

Not necessarily. The analysis may explain a prediction or study the trained model more broadly.

Can two methods disagree?

Yes. They may use different definitions, baselines or approximations, so the disagreement needs interpretation.

Is a model-generated verbal reason automatically a faithful explanation?

No. Its relationship to the actual computation or evidence must be established rather than assumed.

Check your understanding

A local explanation closely matches one prediction but is advertised as proving how the model always works. What is wrong?

Answer: The claim exceeds the evidence. Agreement for one case does not establish fidelity across other inputs or a complete account of the model.

Related terms

• Explainable AI
• Interpretability
• LIME
• SHAP
• local surrogate
• explanation fidelity

Learn next

Read LIME and explanation fidelity. Take one explanation your team uses and write the question it answers, its scope, and one test that could reveal a misleading interpretation.

More in Governance, ethics and safety

• Personal data - Personal data is information relating to an identified or identifiable person.
• Popularity Bias - Popularity bias is a tendency for already popular items to receive disproportionate exposure while less popular items receive little representation.
• Population Bias - Population bias is a systematic mismatch between the people represented in a dataset and the population about which conclusions or predictions are intended.
• Post-processing - Post-processing is the adjustment or transformation of a model's output after inference.
• Post-processing algorithm - A post-processing algorithm is a defined procedure that transforms a model's outputs after prediction.
• Practical significance - Practical significance is the importance of an observed or estimated effect in its real-world context.

Sources

NISTIR 8312, Four Principles of Explainable Artificial Intelligence: https://nvlpubs.nist.gov/nistpubs/ir/2021/NIST.IR.8312.pdf. Relevant returned material distinguishes understandable explanations from explanation accuracy. Ribeiro, Singh and Guestrin, Why Should I Trust You?: https://arxiv.org/abs/1602.04938. The original LIME paper supports the local-surrogate discussion. NIST AI RMF Playbook: https://airc.nist.gov/airmf-resources/playbook/measure/. Examples and calculations are original illustrations.

Previous: Post hoc analysis Next: Post-processing