AI Glossary / MLOps and deployment / Intermediate
Nyquist - Shannon sampling theorem
Definition
The Nyquist - Shannon sampling theorem describes when a continuous, band-limited signal can be reconstructed from uniformly spaced samples. In its familiar low-pass form, sampling faster than twice the highest frequency present avoids overlap between spectral copies. The result explains why sampling rate, signal bandwidth, and filtering must be considered together.
What it means
Sampling records values at discrete positions or times: over time for audio, across space for images. Many different curves could pass through those values unless the original signal is constrained.
Band limitation supplies that constraint: the signal has no components above a specified highest frequency. Under ideal conditions, sufficiently close samples then determine the signal rather than merely suggesting a plausible curve through it.
The Nyquist frequency of a sampling system is half its sampling rate. The Nyquist rate associated with a low-pass signal is twice its highest frequency. These related terms describe different quantities. Equality at the boundary requires care with assumptions, so practical designs do not rely on a sinusoid exactly at half the sampling rate being unambiguously captured for every phase.
How it works
Uniform sampling creates repeated copies of a signal's spectrum, spaced by the sampling frequency. When the original spectrum is appropriately band-limited, the copies remain separated and an ideal reconstruction filter can isolate the original.
In the time domain, ideal reconstruction can be expressed as a weighted sum of sinc functions centered at the sample positions. This is not the same as simply drawing straight lines between adjacent samples.
If frequencies outside the allowable band reach the sampler, different continuous signals can produce identical sampled sequences. That ambiguity is aliasing. An anti-aliasing filter suppresses unwanted high-frequency content before sampling or before reducing an existing sample rate.
The mathematical theorem assumes ideal conditions. Real systems have finite records, imperfect filters, quantization, noise, and timing errors. A design must allow for those limitations rather than treating the theorem as a promise of perfect reconstruction from any finite file.
Three practical examples
1. An audio engineer chooses a sample rate with room for the desired sound bandwidth and a realizable anti-aliasing transition. The rate alone does not establish the microphone's noise performance or the recording's bit depth.
2. A machine-monitoring team samples a vibration sensor. It checks whether higher-frequency vibration or electrical noise could fold into the frequencies used to diagnose a fault.
3. An image-processing tool reduces a photograph's dimensions. Appropriate low-pass filtering before downsampling helps prevent fine repetitive patterns from becoming false larger patterns in the smaller image.
Think of it this way
Think of observing a repeating motion at regular moments. If observations are too far apart, several different motions can fit exactly what you saw. Knowing that the motion cannot change beyond a certain frequency helps resolve that ambiguity.
The analogy is limited because the theorem concerns a defined signal model and complete sampling conditions. A few snapshots of an unrestricted real-world motion do not automatically satisfy band limitation or provide enough information for exact reconstruction.
Worked example
Consider sampling a cosine at 1,000 samples per second. A 100-hertz cosine produces samples cos(2π×100×n/1000), where n is an integer sample index.
A 900-hertz cosine produces cos(2π×900×n/1000). Because 900/1000 equals 1−0.1, these values equal cos(2πn−2π×0.1n), which equals the 100-hertz cosine samples. The sampler cannot distinguish these two input cosines from their values alone.
The system's Nyquist frequency is 500 hertz. If a suitable pre-sampling filter ensures that the input contains only the intended band below that boundary, the 900-hertz alternative is excluded by the model. Filtering after it has already aliased cannot generally identify and undo the ambiguity.
At the exact boundary, a 500-hertz sine sampled at the same rate with zero phase gives sin(πn)=0 at every sample. This illustrates why “exactly two samples per cycle always works” is an unsafe practical slogan. The example assumes ideal timing and does not model real filter attenuation or sensor noise.
Illustrative case study
Fictional case: RotorWatch adds a vibration sensor to a small test rig. An apparent low-frequency oscillation appears in its recorded data, but a higher-rate reference measurement suggests a higher-frequency source.
The team reviews its analog filter and sampling configuration, then repeats the measurement with a properly specified passband and transition region. It also checks whether the sensor itself responds to frequencies beyond the intended diagnostic band.
This hypothetical investigation does not explain every unexpected peak. Other causes, including mechanical behavior and measurement errors, remain possible and require evidence.
Common misconceptions and mistakes
Sampling rate and bit depth solve different problems. Increasing the number of bits per sample improves amplitude representation under suitable conditions; it does not prevent frequency aliasing caused by an inadequate sampling arrangement.
Nor does collecting a longer recording raise the Nyquist frequency. A longer observation can improve frequency discrimination in analysis, but the sample spacing still sets the relevant sampling boundary.
The usual twice-the-highest-frequency statement is the familiar low-pass case. Bandpass sampling and other specialized settings need their own constraints. Applying one slogan to every spectrum can lead to incorrect designs, especially when the occupied band does not start near zero.
Business use
The theorem helps businesses specify audio, imaging, telemetry, and monitoring systems without confusing stored data volume with usable signal information. The relevant question is which physical variations must be preserved and what unwanted components reach the sampling stage.
Document the sample rate together with the passband, filtering, timing assumptions, and measurement units. Validate the whole acquisition chain using representative inputs, including signals near and beyond the intended band. For stored data that will later be downsampled, define the filtering step before discarding samples.
People also ask
Can software fix aliasing after acquisition?
Not generally from the samples alone, because different original signals may have produced exactly the same values. Additional information or assumptions would be required.
Does a higher sample rate always improve every aspect of quality?
No. It changes sampling capacity, while noise, dynamic range, sensor response, and other factors also matter.
Does the theorem apply to images?
Yes, sampling principles extend to spatial signals, with the relevant dimensions and frequency support considered explicitly.
Check your understanding
Why can 100-hertz and 900-hertz cosines produce identical samples at 1,000 samples per second?
Answer: Their phase increments differ by an integer full cycle and a sign that cosine cannot distinguish at those sample times, producing aliasing.
Related terms
- Aliasing (factorial experiments)
- Nyquist frequency
- aliasing
- sample rate
- band-limited signal
- anti-aliasing filter
- sinc interpolation
- quantization
Learn next
Calculate a few samples of two aliased sinusoids, then add a stated band-limit assumption. Separate what the sample values reveal from what the acquisition model allows you to infer.
More in MLOps and deployment
- Multi-class multi-output
- Multi-output - Multi-output describes a prediction problem in which each input example has more than one target value, such as several numerical measurements or...
- Negafibonacci coding - Negafibonacci coding represents nonzero signed integers using Fibonacci numbers with negative indices, then turns that representation into a self-delimiting...
- Offline Store - An offline store is the part of a feature-store architecture used to retain and retrieve historical feature values for work such as model training and batch...
- Online Store - An online store is the part of a feature-store architecture designed to retrieve feature values quickly for prediction requests, usually using one or more...
- OR-parallelism - OR-parallelism explores alternative solutions concurrently.
Sources
Primary reference: Claude E. Shannon, Communication in the Presence of Noise, original 1949 paper reproduced by IEEE, https://webusers.imj-prg.fr/~antoine.chambert-loir/enseignement/2020-21/shannon/shannon1949.pdf . The sinusoid calculations and fictional monitoring example are original teaching illustrations.
Previous: Nyquist sampling rate Next: NYSIIS phonetic code