Is an overlay always added after prediction?

No. Broad usage can include input or parameter changes, while post-model adjustments specifically concern outputs.

Can an overlay be negative?

Yes. Its sign should follow the supported effect and the defined quantity, rather than an assumption that all adjustments increase estimates.

When should an overlay be removed?

When its justification no longer applies, its effect is adequately incorporated elsewhere or evidence supports a different response, following documented review.

AI Glossary / Statistics and probability / Intermediate

Model overlay

Definition

A model overlay is a documented adjustment applied to a model's inputs, parameters or outputs to address a limitation, missing risk or circumstance that the underlying model does not adequately represent. In model-risk management, the term often refers to a judgment-based addition or correction alongside the model's ordinary calculation. A post-model adjustment is a narrower expression commonly used for an output-level overlay. Terminology varies by organization, so an overlay record should identify exactly what is changed, why it is changed and how the adjusted result is used.

What it means

A model is necessarily an approximation. Its historical data may omit a new event, its variables may miss a relevant factor or its structure may respond too slowly to changed conditions. An overlay can make that gap explicit while the organization investigates a more durable solution. However, an adjustment introduces its own assumptions and uncertainty. Calling something an overlay does not make it correct, conservative or independently validated. The unadjusted model result and the adjustment should remain distinguishable so reviewers can understand the contribution of each.

How it works

Identify the specific limitation and gather evidence about its likely effect. Determine whether the proposed adjustment belongs in inputs, parameters or outputs, and examine what the model already captures. Specify the calculation, its rationale, ownership and review conditions. Test alternative assumptions and show results before and after the overlay. Record approval and implementation details, including how to prevent duplicate application. Monitor whether the adjustment improves decisions and whether its original justification remains relevant. Establish conditions for reducing, replacing or retiring it when the model or operating environment changes.

Three practical examples

Demand planning: incorporate a known local event absent from a forecasting model's historical features.
Scenario analysis: supplement a model when a specified stress scenario contains a mechanism the existing equations omit.
Operational forecasting: adjust an estimate for a documented temporary capacity constraint while a revised model is being developed. Each example requires an explicit account of overlap with effects already represented in the model.

Think of it this way

Think of annotating a map with a verified road closure. The map remains useful, but the annotation changes the route you choose. If the map is later updated, keeping the same annotation without checking it could mislead you. An overlay serves a similar purpose: it addresses a known gap while remaining visible, reviewable and removable.

Worked example

Suppose an original demand model forecasts 1,000 units. Analysts estimate that a special event adds 120 units, but determine that 40 of those units are already reflected in an existing seasonal feature. Their net overlay is 120 - 40 = 80 units, producing an adjusted forecast of 1,080. If observed demand is subsequently 1,060, the original absolute error is 60 units and the adjusted error is 20. This single outcome does not establish general effectiveness. Later, a revised model independently incorporates the entire event effect and forecasts 1,080 without an overlay. Keeping the old 80-unit adjustment would produce 1,160 and double-count the residual event effect. This original example assumes that the overlap assessment and the revised model's coverage are known; in practice, both require evidence and sensitivity analysis.

Illustrative case study

In a fictional case, Juniper Supplies uses a demand forecast to plan staffing for a seasonal distribution center. A newly announced exhibition is absent from its historical data. Analysts propose an overlay, explain its overlap with the calendar features and show a range of plausible effects. The planning manager approves a temporary adjustment with a review date. After the event, the team compares the forecast components with observed demand and records what remained uncertain. When the forecasting pipeline gains an event feature, the team reviews and retires the temporary overlay instead of carrying it forward automatically.

Common misconceptions and mistakes

An overlay is not an ensemble merely because it changes a prediction. An unexplained manual override is not equivalent to a controlled adjustment. Adding a conservative amount does not automatically reduce overall risk, particularly when interacting effects or operational costs matter. Adjustments can double-count existing model effects, conceal poor model performance or accumulate long after their justification has expired. Agreement with one observed outcome does not validate the rationale. A model limitation should not become an indefinite excuse for avoiding repair of the underlying data or model.

Business use

Businesses should make overlays traceable from the limitation to the adjusted decision. Useful records include the affected model version, evidence, calculation, responsible person, independent challenge, materiality and retirement criteria. Compare the size of adjustments with the original output and report uncertainty in both. Evaluate a portfolio of overlays for inconsistent assumptions and overlapping effects. A useful governance question is whether the adjustment still represents a temporary response to a known limitation or has become a separate model requiring its own maintenance and assessment.

Check your understanding

A revised model already incorporates an effect previously supplied by an overlay. Should the same overlay automatically remain?

Answer: No. Review its coverage and remove or revise the adjustment to avoid counting the same effect twice.

Related terms

Model validation
Sensitivity analysis
Model risk
expert judgment
scenario analysis
forecast adjustment
model monitoring
double counting

Learn next

Learn how model limitations are documented, then study sensitivity analysis and independent validation. Practice separating a baseline forecast, an adjustment and the evidence supporting each, including an explicit rule for reviewing the adjustment.

Sensitivity analysis

Sources

Office of the Comptroller of the Currency, Model Risk Management, Comptroller's Handbook: https://occ.treas.gov/publications-and-resources/publications/comptrollers-handbook/files/model-risk-management/pub-ch-model-risk.pdf Bank of England, Supervisory Statement SS1/23, Model risk management principles for banks, discussion of post-model adjustments: https://www.bankofengland.co.uk/-/media/boe/files/prudential-regulation/supervisory-statement/2023/ss123.pdf

Previous: Model order reduction Next: Model parallelism