Does a positive signal always mean underforecasting?

Only under a sign convention such as actual minus forecast; the opposite convention reverses it.

Is MAD the median absolute deviation here?

No. In this formulation it is the arithmetic mean of absolute forecast errors.

Should every alert force a model replacement?

No. First examine whether the cause is data quality, a temporary event, a changed process or a model limitation.

AI Glossary / Time series and signal processing / Intermediate

Tracking signal

Definition

A tracking signal is a forecast-monitoring statistic highlighting persistent errors that may indicate bias or a changed forecasting environment. One common version divides cumulative signed forecast error by mean absolute error over a defined monitoring period. A large positive or negative value can prompt investigation. Tracking signals have several formulations, so the error sign, denominator, time window and alert rule must be stated.

What it means

An occasional large forecasting error is different from repeatedly forecasting too low or too high. A tracking signal accumulates direction as well as considering error scale. With error defined as actual minus forecast, positive accumulated error means actual values have tended to exceed forecasts. Negative accumulated error points in the opposite direction.

This entry uses the signed cumulative-error-over-MAD convention, where MAD means mean absolute forecast deviation, not median absolute deviation. Some implementations use an absolute signal; smoothed statistics weight history differently. Their thresholds are not interchangeable. The statistic does not establish why forecasts went wrong.

How it works

Store the forecasts actually issued at the decision horizon and compare them with corresponding actual outcomes when available. For each period calculate signed error e = actual minus forecast. Over the selected window, add the errors to obtain a running error total and average their absolute values to obtain MAD. Divide the total by MAD when MAD is positive.

Compare the result with a documented monitoring rule. Investigate sustained deviations alongside absolute errors, data completeness and business changes. If every error is zero, the ratio is undefined as written, so software needs an explicit convention. A rolling window, an expanding window and exponential smoothing have different behavior; choose and record the design before interpreting alerts.

Three practical examples

Replenishment: monitor whether demand forecasts repeatedly underestimate actual demand, while checking stockouts and delayed transactions before attributing the problem to the model.
Staffing: flag persistent underprediction of weekly workload at the scheduling horizon, then examine holidays, service changes and missing records.
Model maintenance: prioritize forecast series for review when directional errors accumulate, using the signal alongside magnitude-based accuracy measures rather than as the sole ranking criterion.

Think of it this way

Think of a compass that records whether successive steps have drifted more to one side than the other. Large left and right steps can cancel, while smaller repeated steps in one direction accumulate. The analogy captures directional imbalance, but the monitoring threshold still requires a statistical and operational design; it is not a universal boundary between good and bad forecasting.

Worked example

Suppose four completed periods have actual-minus-forecast errors of 4, minus 2, 6 and 4 units. The cumulative signed error is 12. The sum of absolute errors is 16, so MAD is 16/4 = 4. The signed tracking signal is 12/4 = 3. This positive value indicates net underforecasting under the declared sign convention.

Now add a fifth error of 8 without resetting the expanding window. Cumulative error becomes 20 and absolute-error total becomes 24. MAD is 24/5 = 4.8, giving a tracking signal of 20/4.8, approximately 4.17. An illustrative policy that investigates absolute signals greater than 4 would trigger here. That threshold is chosen solely to demonstrate the rule; it is not a universal statistical limit or a recommendation for deployment.

For contrast, errors 10, minus 10, 10 and minus 10 have cumulative error zero and MAD 10, so their signal is zero despite large mistakes. A near-zero signal therefore cannot establish accuracy. The example also shows why the monitoring window matters: using only the latest error gives signal 1, a different result from retaining all five observations.

Illustrative case study

A fictional parts distributor sees positive tracking signals for several products after opening a new sales channel. Analysts verify that they are comparing the original purchase-planning forecasts with complete demand records. They find a sustained increase rather than a one-day posting error and test revised models on later periods. They preserve the original alert history and document any reset after the approved change. The signal directs attention to a problem; the investigation and subsequent evaluation determine the response.

Common misconceptions and mistakes

A tracking signal is not a percentage error, a prediction interval or a p-value. A threshold crossing does not prove structural change, and no crossing does not prove that a model is suitable. Alternating errors can cancel, and changing error variability can affect the denominator. Reversing the error definition reverses the interpretation of positive and negative signals.

With the same n errors in both the cumulative numerator and ordinary MAD denominator, the absolute ratio cannot exceed n. Repeated errors of one sign attain that bound. This property highlights why observation count and window design matter. A four-period signal and a year-long expanding signal should not be compared without understanding their construction.

Business use

Use tracking signals to organize forecast review around a documented objective. Keep the forecast horizon consistent and distinguish issued forecasts from later revisions. Specify the alert threshold, required persistence, minimum history, reset procedure and handling of missing actuals. Evaluate the rule on historical periods that include ordinary variation and known disruptions.

Record false alarms and missed changes. Inspect error magnitude, service consequences and data quality before replacing a model. Thresholds should reflect the chosen statistic and acceptable monitoring behavior rather than a number copied from an unrelated system.

Check your understanding

Can a tracking signal of zero coexist with poor forecasts?

Answer: Yes. Equal positive and negative errors can cancel in the numerator. The alternating errors in the worked example have signal zero but mean absolute error 10, so directional balance does not imply small errors.

Related terms

Forecast bias
Mean absolute deviation (m.a.d.)
Mean Absolute Error
Control limits
cumulative forecast error
mean absolute deviation
structural change
monitoring window
control limit
forecast revision
drift detection
Trigg tracking signal

Learn next

Calculate signed and absolute errors for a short forecast history. Compare expanding and rolling monitoring windows, then examine what happens after a one-off outlier or sustained bias. Study statistical process monitoring before selecting thresholds, and keep separate measures of error magnitude and operational consequences.

Sources

Hyndman, Glossary of Forecasting Terms, tracking signal entry: https://robjhyndman.com/mwh3/FG4.pdf SAP, monitoring the selected forecast model, error-total/MAD formulation: https://help.sap.com/docs/SAP_ERP_SPV/42ad0c855a03441abde4d5db2fef5a65/a66db6531de6b64ce10000000a174cb4.html SAP, Trigg's Tracking Signal, a distinct smoothed monitoring implementation: https://help.sap.com/saphelp_SCM700_ehp02/helpdata/en/47/c4c9f558a53020e10000000a42189d/content.htm Source scope: glossary and Trigg documentation read; SAP basic-ratio indexed excerpt consulted because its opened page returned no readable body. The signed teaching convention and illustrative threshold are explicitly defined here.

Previous: Traceability Next: Tractability