AI Glossary / MLOps and deployment / Advanced

Motion compensation

Definition

Motion compensation uses motion information to form a prediction of image content from one or more reference images. In video compression, it commonly reconstructs a predicted block by sampling a displaced region of a reference picture. The remaining difference is represented as a residual. Motion estimation finds or selects the motion information; motion compensation applies it to construct the prediction.

What it means

Successive video frames often contain related content at different positions. A coder can reuse that content instead of describing the new frame independently. The prediction is not necessarily perfect: lighting changes, newly visible areas and model limitations can leave differences. A residual accounts for those differences within the coding system. Motion compensation therefore should not be described as transmitting only movement while ignoring everything else, nor should a coding motion vector automatically be interpreted as an exact physical measurement of an object's movement.

How it works

A decoder uses the chosen reference picture, motion vector and applicable sampling rules to form a prediction. Fractional-position vectors require interpolation because samples at those exact locations are not directly stored. The encoder forms a corresponding residual, and the decoder adds its reconstructed version to the prediction. In a lossy codec, quantization can change that residual. Block size, reference selection and motion-vector information all contribute to the coding cost, so a smaller prediction error alone does not establish the smallest complete encoded representation.

Three practical examples

• A video-coding lesson predicts a moving image region from a displaced position in an earlier reference picture.
• An encoder experiment compares a coarse motion partition with smaller partitions while counting both residual and motion-information bits.
• A decoder test verifies that interpolation and reference-picture handling match the format's specified reconstruction rules.

Think of it this way

Imagine updating a map by shifting a previously drawn feature into its new position and then recording the details that still differ. The shift can save work, but it does not describe new features or changes in appearance by itself. The correction remains important. Motion compensation uses a similarly structured prediction, although a video codec's vectors describe sampling relationships under its rules rather than necessarily tracing real-world objects with geographical accuracy.

Worked example

Use an original one-dimensional reference row R = [10, 20, 30, 40, 50], with zero-based positions. Suppose a three-sample current block at positions zero through two is C = [21, 29, 42]. Under the explicitly defined sampling rule P[i] = R[i + 1], its prediction is [20, 30, 40]. The residual C − P is [1, −1, 2]. Adding that exact residual to P reconstructs C. If a toy lossy step changes the residual to [0, 0, 2], reconstruction becomes [20, 30, 42]. The example fixes the displacement sign convention and uses integer positions; it does not specify a real codec's motion-vector bitstream or interpolation filters.

Illustrative case study

A fictional video team compares two motion-search settings. One setting produces a lower residual error but requires many more small partitions and motion descriptions. The team measures total bitrate and decoded quality rather than choosing solely on residual energy. It also tests a scene with newly revealed background, where a previous reference does not contain all the current content. The case illustrates why prediction quality, side-information cost and reconstruction quality must be assessed together. It does not imply that smaller partitions are always better or that motion compensation removes the need to encode newly visible content.

Common misconceptions and mistakes

Motion estimation and motion compensation are related but different operations. A vector is not necessarily physical optical flow or an object trajectory; a coding decision can favor an economical prediction. Another mistake is assuming that a good motion match eliminates all residual information. Appearance changes and imperfect models can still matter. Fractional motion vectors also do not mean that fractional samples were originally captured; the decoder generates prediction values through interpolation. Finally, exact prediction arithmetic does not make the complete codec lossless if later quantization or other processing changes the reconstructed data.

Business use

The term helps explain video bitrate, quality and decoder-complexity tradeoffs. When comparing settings or implementations, inspect representative motion, texture, occlusion and scene-change cases. Measure complete encoded size and decoded quality, including the cost of motion descriptions. For systems using motion vectors as AI features, evaluate whether those coding vectors serve the downstream task rather than assuming they measure physical motion accurately. In interoperability work, reference handling and interpolation must match the specification, because small prediction mismatches can affect subsequent reconstruction.

People also ask

Is motion compensation the search for a vector?

The search is motion estimation; compensation applies motion information to form a prediction.

Why is a residual needed?

The prediction can differ from the current content.

Are coding vectors exact object velocities?

Not necessarily. They describe a prediction relationship chosen within a coding system.

Check your understanding

Answer: What prediction did the shifted reference produce in the toy example? [20, 30, 40]. What residual reconstructed the current block exactly? [1, −1, 2].

Why did the altered residual change the first two reconstructed samples?

Answer: The correction no longer contained the original differences at those positions.

Related terms

• Motion estimation
• Motion vector
• inter prediction
• reference picture
• prediction residual
• subpixel interpolation

Learn next

Construct a small reference array and state a displacement convention explicitly. Calculate the prediction and residual, then reconstruct the current values. Extend the example with an unmatched region and discuss its correction cost before comparing complete codec settings on actual video.

More in MLOps and deployment

• Modified discrete cosine transform - The modified discrete cosine transform, or MDCT, is a lapped time-frequency transform used in audio coding.
• Modified Huffman coding - Modified Huffman coding, or MH, is a one-dimensional lossless run-length coding method associated with bilevel facsimile images.
• Modulo-N code - Modulo-N coding represents a value by its remainder after division by N and uses correlated information at the decoder to help recover the intended value.
• Move-to-front transform - The move-to-front transform, or MTF, is a reversible transformation that replaces each input symbol with its current position in an ordered list, then moves...
• Multi-class multi-output
• Multi-output - Multi-output describes a prediction problem in which each input example has more than one target value, such as several numerical measurements or...

Sources

Iain Richardson, H.264/AVC Inter Prediction: https://www.vcodex.com/h264avc-inter-prediction The five-sample reference calculation, map analogy and video-team case are original educational illustrations. The source provides a concrete block-based coding context; its particular partition and interpolation rules are not generalized to every motion-compensation method.

Previous: Motion coding Next: Motion estimation