AI Glossary / Computer vision and multimodal AI / Intermediate
B-frame
Definition
A B-frame is a video picture that supports bidirectional or bi-predictive coding, using reference information to predict image content efficiently. A familiar arrangement predicts a picture from references before and after it in display order, though modern codecs allow more flexible reference structures.
What it means
Consecutive video frames often contain similar objects in slightly different positions. Rather than storing every frame independently, a codec can describe predictions from other decoded pictures and encode the remaining differences.
A B-frame provides flexible prediction choices that can reduce the amount of information needed for a sequence. It does not mean the frame is missing, generated without source evidence or merely displayed halfway between two other frames. The decoder reconstructs a coded picture from the prediction and any required residual information.
The letter is traditionally associated with bidirectional prediction. In standards such as H.264, prediction types are specified at slice and block levels, so the everyday phrase B-frame simplifies a more detailed coding structure.
How it works
An encoder searches for useful reference regions and represents motion or prediction relationships. A bi-predicted block can combine two reference predictions, while other blocks in the same broad coding context may use different permitted modes.
When a picture depends on a reference that appears later in display order, that reference must be decoded first. The stream and decoder therefore distinguish decoding order from presentation order. Buffering stores reconstructed pictures until they are needed for reference or display.
The exact rules depend on the codec and configuration. Modern designs can use some B pictures as references for other pictures, and their reference choices need not always be one earlier and one later display-time picture. The compression benefit, buffering and latency implications therefore come from the complete prediction structure, not the label B alone.
Three practical examples
- An on-demand video service uses a prediction structure with B pictures to improve compression efficiency for stored programs.
- An editor examines decode and presentation timestamps when troubleshooting a transcoded clip that appears out of sequence.
- A live-video engineer compares low-delay settings with configurations that use more reordering, measuring the effect on the full pipeline.
Think of it this way
Think of describing the middle drawing in a flipbook using drawings on either side, plus notes explaining what differs. To reconstruct the middle drawing, the reader must already have the reference drawings available. The analogy is limited: codecs work with blocks, motion and coded residuals, and modern reference relationships can be more complex than two neighboring pages.
Worked example
Consider a simple display sequence I0, B1, B2, P3. Suppose B1 and B2 both use I0 and P3 as references. The viewer should still see the pictures in the order 0, 1, 2, 3.
A valid illustrative decoding order is I0, P3, B1, B2. P3 is reconstructed before B1 because B1 needs it as a reference. The decoder buffers pictures and releases them according to presentation timing rather than immediately showing each picture as it is decoded.
This is one simple arrangement, not a universal rule for every codec or B-picture hierarchy. The example demonstrates dependency ordering. It does not establish an exact end-to-end delay, which also includes capture, encoding, transport, decoding and display scheduling.
Illustrative case study
Fictional example: Streamhaven Training prepares recorded lessons and an interactive live classroom. The recorded lessons can tolerate more processing delay, while the classroom needs responsive conversation.
Engineers test separate encoding configurations using representative content. They compare visual quality at a chosen bitrate and measure actual latency rather than assuming that a single B-frame setting determines the result. Playback is checked on the intended devices, including seeking and recovery after interruptions.
The team documents the prediction structure and delivery settings for each use. This fictional case illustrates a compression-and-latency trade-off; it does not claim a measured bitrate saving or prescribe one configuration for every service.
Common misconceptions and mistakes
B-frames are not always the smallest frames in a stream. Size depends on scene content, prediction quality, quantization and encoder decisions. A scene change or difficult motion can alter the expected pattern.
It is also incorrect to say that B pictures can never be references; some modern coding structures use them that way. Bidirectional prediction does not require every block to average exactly two adjacent frames. Finally, dropping a coded picture without considering dependencies can break later decoding even when that picture is not intended as a random-access point.
Business use
Understanding B pictures helps teams make informed choices about compression efficiency, latency, editing and delivery compatibility. The right configuration depends on whether the application prioritizes stored-video quality, interactive response or convenient access and editing.
Use representative motion and scene changes in evaluations. Measure complete pipeline delay and playback behavior, not just encoder speed. Keep track of reference dependencies and timestamps when transcoding or packaging. Changes to a codec setting should be validated on actual receiving devices because nominal support does not describe every operational constraint.
People also ask
Are B-frames fake intermediate frames?
No. They are coded pictures reconstructed using prediction and residual information. Frame interpolation is a different operation.
Must decoding order match viewing order?
No. A later-displayed reference may need to be decoded earlier, with buffering restoring the intended presentation sequence.
Do B-frames always add the same delay?
No. Delay depends on the prediction structure and the entire pipeline. Some arrangements require more reordering than others.
Check your understanding
Why is P3 decoded before B1 in the worked example even though P3 is displayed later?
Answer: B1 depends on P3 as a reference. The decoder must reconstruct that reference first, then present the completed pictures in their intended display order.
Related terms
- I-frame
- P-frame
- Motion compensation
- inter prediction
- residual coding
- group of pictures
- decoding order
- presentation timestamp
- reference picture
Learn next
Draw the reference arrows for the four-picture example and list which pictures must be available before each decoding step. Then compare a real encoder's documented low-delay and stored-video configurations. Measure playback delay and quality separately, and avoid treating frame-type counts as a complete description of codec behavior.
More in Computer vision and multimodal AI
- Aspect ratio - Aspect ratio is the proportional relationship between width and height.
- Automatic white balance (AWB) - Automatic white balance, or AWB, estimates the color influence of scene illumination and adjusts an image's color balance accordingly.
- Autoregressive image model - An autoregressive image model represents an image as an ordered sequence and predicts each element using previously available elements.
- B-rep - B-rep, short for boundary representation, describes a three-dimensional shape through its boundary elements and how they connect.
- Background subtraction - Background subtraction is a computer-vision technique that identifies image regions differing from an estimated background scene.
- Banding - Banding is the appearance of visible steps or stripes in a region that should change smoothly in brightness or color.
Sources
ITU-T H.264, Advanced video coding for generic audiovisual services, normative codec reference: https://www.itu.int/rec/T-REC-H.264 ITU-T H.264 definition and decoding provisions, accessible edition: https://www.itu.int/rec/dologin_pub.asp?id=T-REC-H.264-202108-S%21%21PDF-E&lang=e&type=items The picture sequence and training-service case are original illustrations; no full codec-conformance audit is claimed.