AI Glossary / Time series and signal processing / Intermediate
Filter
Definition
A filter is a rule or system that transforms a sequence of observations to retain, suppress, or reshape particular features. In time-series and signal-processing work, a filter may smooth fluctuations, emphasize changes, isolate frequency ranges, or estimate an underlying state. Its behavior depends on its design and assumptions, not simply on being called a filter.
What it means
Filtering changes what information is emphasized. A moving average can make a slowly changing level easier to see by reducing some rapid variation. A differencing filter instead emphasizes change and removes a constant level. Neither operation discovers a uniquely true pattern without assumptions about what matters.
In another common sense, a probabilistic filter updates knowledge of a hidden current state from observations available so far. These meanings are related through sequential estimation, but not every filter is a weighted average or a probability calculation.
How it works
A finite linear filter forms each output from a weighted combination of input samples. For a causal three-point average, output yt = (xt + x(t−1) + x(t−2))/3. Causal means that the calculation does not require future observations. Other designs also use previous outputs, producing recursive filters.
For a linear time-invariant filter, the weights or impulse response determine its frequency response. Some frequencies are attenuated more than others, and phase changes can shift features in time. Nonlinear filters, such as a median filter, behave differently and cannot generally be described by the same fixed frequency-response rule. Implementation also requires decisions about startup and endpoints.
Three practical examples
- A demand dashboard applies a trailing average to daily orders so managers can inspect the recent level. The raw series remains available for unusual-event analysis.
- A vibration-monitoring system uses a band-pass filter to focus attention on a specified frequency region, after verifying that relevant fault signals fall in that region.
- A location estimator combines movement predictions with noisy position readings. Its state-estimation filter uses uncertainty assumptions rather than simply removing selected frequencies.
Think of it this way
Think of a tinted lens that makes some visual features easier to see while diminishing others. The lens does not certify that everything it hides is irrelevant. Similarly, a filter can improve a particular view of data while removing a short event or altering its apparent timing. The intended decision determines whether that tradeoff is acceptable.
Worked example
Consider an original sequence of five daily readings: 10, 12, 8, 14, and 16. Apply a causal three-day average and report outputs only after three actual observations exist. The outputs for days three, four, and five are (10 + 12 + 8)/3 = 10, (12 + 8 + 14)/3 = 11.3333, and (8 + 14 + 16)/3 = 12.6667.
Day five's raw reading is 16, while its filtered value is about 12.67. That difference is not automatically measurement error; it may reflect genuine recent growth that the average follows with delay.
Now impose a separate theoretical assumption: each reading contains an independent zero-mean noise term with variance 9. The filtered noise variance is 9 × [(1/3) squared + (1/3) squared + (1/3) squared] = 3. Noise has been reduced under this assumption, not eliminated. Adjacent filtered outputs share two noise terms, so their noise covariance is 2 and correlation is 2/3. Thus smoothing can introduce dependence even when input noise is independent. These variance results concern the assumed noise component, not the total variability of a changing signal.
Illustrative case study
Illustrative fictional case: Willow Service Centre smooths daily complaint counts to help identify sustained increases. A manager notices that the filtered chart looks reassuring during a brief but serious outage. The team restores a raw-count alert alongside the trend display.
It labels the smoothing window and avoids presenting the delayed average as the current instantaneous count. Analysts assess whether the filter supports weekly planning, while incident staff use a separate view designed for rapid detection. The fictional lesson is that one filter need not serve every operational purpose.
Common misconceptions and mistakes
A filter does not necessarily remove randomness optimally, and random variation is not restricted to high frequencies. Useful signals and unwanted noise may occupy overlapping frequency ranges. A smooth output can therefore be misleadingly persuasive.
Do not treat a centered average as available in real time if it uses later observations. Nor should filtered samples automatically be analyzed as independent. Endpoint padding, initial conditions, and missing-value choices can create artifacts. Applying a filter repeatedly also changes its effective response, even if the same settings are used each time.
Business use
A business should specify the filter's purpose before choosing coefficients: monitoring a slow level, detecting an abrupt event, preparing inputs for a model, or estimating a hidden state are different tasks. Evaluate the resulting latency and lost detail alongside any reduction in variability.
Document the sampling interval, causal or retrospective use, startup policy, and parameter selection. Validate the whole downstream decision process because a preprocessing improvement judged by visual appearance can still worsen detection or forecasting performance.
People also ask
Does filtering always mean smoothing?
No. High-pass and differencing filters can emphasize rapid changes.
Can a filter use future values?
Retrospective filters can, but such outputs must not be described as available earlier.
Is a Kalman filter just a moving average?
No. It updates a state estimate and uncertainty using a model and observations, although linear cases also combine information through weights.
Check your understanding
Which observations produce the example's day-four output?
Answer: Days two, three, and four.
Why is the filtered noise variance not zero?
Answer: Averaging independent noise reduces its variance but leaves residual uncertainty.
Why might a short, important incident disappear from a filtered chart?
Answer: Its contribution can be diluted by neighboring values, depending on the filter and display scale.
Related terms
- Filtering
- Moving average
- Smoothing
- Differencing
- Low-pass filter
- Kalman filter
- impulse response
- frequency response
- causal filter
- phase delay
Learn next
compare raw values with both trailing and centered averages, explicitly marking when each result becomes available. Then examine a filter's impulse and step responses to understand its delay and treatment of sudden changes.
More in Time series and signal processing
- Exponential smoothing - Exponential smoothing is a family of methods that updates estimates of a time series as new observations arrive.
- Extended Kalman filter - An extended Kalman filter, or EKF, estimates the evolving hidden state of a system whose dynamics or measurements are nonlinear.
- Fast Fourier transform - A fast Fourier transform, or FFT, is an algorithm for computing a discrete Fourier transform efficiently.
- Filter Design - Filter design is the process of choosing a filter structure and parameters to meet specified requirements for transforming a signal.
- First difference - A first difference is the change between a time-series observation and the immediately preceding observation: Δyt = yt − y(t−1).
- Fixed-lag smoothing - Fixed-lag smoothing estimates a hidden state from a fixed number of time steps in the past using observations available through the present.
Sources
Sources: Glossary of Forecasting Terms, Filter entry: https://robjhyndman.com/mwh3/FG4.pdf . SciPy, official Signal Processing tutorial, filtering and filter-design sections: https://docs.scipy.org/doc/scipy/tutorial/signal.html . Primary sources consulted. The glossary's broad language about eliminating randomness is qualified here; the numerical example and fictional case are original.
Previous: Filled pause Next: Filter Design