Is a Hampel filter the same as a median filter?

No. A Hampel filter typically replaces only flagged points, whereas a standard median filter replaces each point with its local median.

Does the method require a normal distribution to calculate a flag?

No, but the scale interpretation and threshold behavior depend on the data distribution.

Use a documented handling rule; do not divide by zero or silently invent a spread.

AI Glossary / Statistics and probability / Intermediate

Hampel test

Definition

A Hampel test, commonly called the Hampel identifier, flags observations that lie unusually far from a median relative to a robust estimate of spread based on the median absolute deviation. A sliding-window version compares each time-series observation with its local neighborhood. Replacing flagged observations by the local median produces a Hampel filter. The identifier is an outlier-detection rule; its usual threshold is not automatically an exact hypothesis test with a specified significance level.

What it means

An extreme value can pull the mean and inflate the standard deviation, making itself appear less unusual under a conventional standardized score. The median and median absolute deviation are less sensitive to isolated extreme values, which makes them useful for detecting spikes.

The comparison can use an entire sample or a local window. The choice changes the question: a global rule asks whether a value differs from the overall distribution, while a local rule asks whether it differs from nearby observations.

A flag indicates inconsistency with the chosen neighborhood and threshold. It does not identify the cause or prove that a measurement should be altered.

How it works

Calculate the median m of the reference observations and the raw median absolute deviation MAD = median(|x_j − m|). A common normal-consistent scale estimate is s = 1.4826*MAD. Flag x_i when |x_i − m| exceeds a chosen multiplier times s, often three.

For a moving-window implementation, repeat these calculations around each target observation. Specify the number of neighbors, whether the target itself is included, the boundary rule, missing-value handling, and whether future observations are available. A centered offline window is not automatically appropriate for real-time detection.

The scaling constant calibrates MAD to normal-distribution standard deviation asymptotically; it does not make every local window normal. If MAD is zero, a ratio-based score is undefined and a threshold rule can become excessively sensitive. An implementation needs an explicit policy for this case.

Three practical examples

A sensor engineer flags isolated spikes for review before fitting a trend model.
A data-quality analyst uses a median-based identifier to locate unusually large deviations within a comparable batch of measurements.
An AI operations team evaluates local anomaly flags while preserving genuine step changes that may signal a new operating regime.

Think of it this way

Think of asking a group of nearby observers for their typical reading rather than letting the loudest observer determine both the average and what counts as normal variation. The median provides a resistant center, and MAD describes typical distance from it. The analogy assumes the nearby observers are meaningfully comparable; a mixed group can still give a misleading reference.

Worked example

Take a seven-observation window in time order: 9, 10, 11, 25, 10, 9, and 11. The target is its middle observation, 25. The window median is 10. Absolute deviations from that median are 1, 0, 1, 15, 0, 1, and 1, whose median is 1.

The scaled spread is therefore 1.4826. With a threshold multiplier of three, the permitted deviation is 4.4478. The target's deviation is |25 − 10| = 15, so the identifier flags it. A median-replacement filter would output 10 at this position, while a flag-only workflow would retain 25 and record the flag.

Independent calculation confirms the median, MAD, and decision. For comparison, this same window has mean 12.1429 and sample standard deviation 5.7280; the spike inflates both. This example computes one centered window only. A full moving-window filter would recalculate its reference values at each position and handle endpoints according to its specified rule.

Illustrative case study

In a fictional warehouse, a temperature stream contains occasional isolated electronic spikes and genuine increases when equipment is restarted. An analyst initially applies a Hampel filter to every flagged point.

Review against maintenance logs shows that some replacements erase meaningful restart behavior. The team changes the workflow to preserve raw readings, attach anomaly flags, and replace values only under a documented sensor-error policy. It also evaluates window widths against known events and checks boundary behavior. The resulting dataset records both raw and processed values so downstream model evaluations remain reproducible. The scenario illustrates the difference between detection and correction, not proof that a particular threshold is optimal.

Common misconceptions and mistakes

Do not confuse median absolute deviation with mean absolute deviation. Do not multiply by 1.4826 twice when a software function already returns a normal-scaled MAD. A threshold of three robust scale units is not a universal guarantee of a particular false-alarm rate.

Clusters of anomalous values can move the local reference and escape detection. A long window can mix different operating regimes, while a short window can react strongly to ordinary local variation. Skewness, quantization, ties, and a zero MAD require special attention. Repeated screening across many times and channels changes the overall burden of false flags.

Business use

Hampel identification is useful when an inexpensive and understandable screening rule fits the measurement process. Choose settings through representative validation and the costs of missed events and unnecessary reviews. Document the reference window, scaling convention, threshold, and treatment of flags.

For predictive modeling, apply preprocessing consistently and avoid using future data in a real-time evaluation. Keep raw observations available because an apparent outlier may be a valuable failure signal. Compare model performance on meaningful events before and after any replacement step rather than judging success solely by a smoother graph.

Check your understanding

What robust threshold is used in the example?

Answer: Three times 1.4826 times the raw MAD of one, giving 4.4478.

Does the flag prove that 25 is erroneous?

Answer: No; it identifies a large local deviation requiring context.

Related terms

Learn next

Next, implement a flag-only identifier on a small sequence and inspect every decision. Then compare window choices on spikes, sustained changes, and tied data before considering automated replacement.

Sources

Sources: MathWorks, hampel documentation and Hampel identifier equations, https://www.mathworks.com/help/signal/ref/hampel.html; SciPy, median_abs_deviation documentation, https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.median_abs_deviation.html. Official materials were opened and the scaling, local rule, and filtering distinction checked. The original example independently verifies one window; it is not presented as the output of a complete filtering run.

Previous: Hamming loss Next: Handwriting recognition