Is a visibility mask the same as an occlusion mask?

They often encode complementary ideas, but conventions differ. One may use one for visible and another one for occluded. Check the definition before using either.

Can visibility be estimated from optical flow?

Yes, flow consistency can provide useful evidence. It is not infallible because motion estimates themselves can be wrong.

Does a weight of 0.8 mean an 80% chance of visibility?

Not necessarily. It may simply be a learned or heuristic blending weight without probability calibration.

AI Glossary / Computer vision and multimodal AI / Intermediate

Visibility mask

Definition

A visibility mask records which image locations provide visible, usable evidence for a particular view or synthesis step. In frame interpolation, it commonly assigns a binary value or soft weight to each source contribution so that pixels hidden by occlusion receive less or no influence in the synthesized frame.

What it means

When an object moves, it can uncover background that was hidden in an earlier frame. At that background location, blindly blending both frames can mix the newly visible background with the foreground object that previously covered it. A visibility mask helps select the source that actually contains the relevant surface.

Visibility is relative to a source view, target view and time. It is not a permanent property of a pixel coordinate. The same wall region may be visible in one frame and hidden in another.

The term is broader in graphics and vision, but this entry follows the supplied glossary's frame-interpolation meaning. It differs from object-category segmentation and transparency, despite related uses.

How it works

An interpolation system first estimates how source content maps to the desired intermediate frame. It then evaluates or predicts the visibility of each warped source contribution at target locations.

Some approaches use consistency between forward and backward motion estimates. A point that moves forward and back inconsistently may be occluded, though bad flow can produce the same warning. Learned systems can predict soft visibility weights alongside refined motion.

The synthesis stage combines source samples using both temporal and visibility weights, normally with normalization. A source assigned zero visibility contributes nothing at that location. If neither source provides usable evidence, the system needs a separate policy, such as additional frames or a synthesis method; division by a zero weight sum is not a valid solution.

A predicted soft mask is an estimate. Its numerical values should not automatically be interpreted as calibrated probabilities of visibility.

Three practical examples

A slow-motion generator reduces the contribution of a frame in which a moving hand hides the background needed for an intermediate view.
A multi-view synthesis experiment selects image evidence from cameras that can see a surface rather than blending every camera equally.
A debugging tool displays predicted visibility weights near moving boundaries to help distinguish occlusion problems from incorrect motion estimates.

Think of it this way

Think of asking two observers what is written on a sign. If one observer's view is blocked, averaging their unrelated visual information with the clear observer's account makes little sense.

The analogy is limited because a mask does not understand the sign or guarantee that the supposedly clear source is correct. It controls the contribution of numerical evidence whose alignment and quality still need checking.

Worked example

Combining visibility and temporal weights

Suppose an intermediate frame is required at t = 0.25 between two source frames. At one target location, their warped grayscale samples are A = 40 and B = 100. Let visibility weights be V0 = 1 and V1 = 0.2.

Using weights (1 − t)V0 and tV1 gives 0.75 and 0.05. The normalized output is (0.75 × 40 + 0.05 × 100)/(0.75 + 0.05) = 35/0.8 = 43.75.

Without visibility weighting, the ordinary temporal blend would be 0.75 × 40 + 0.25 × 100 = 55. The mask reduces the influence of the less-trusted second contribution. If V1 becomes zero, the result is 40; the first source supplies the value completely.

If both visibility weights are zero, the denominator vanishes. Returning zero brightness would not mean that the hidden surface is black. It would confuse missing evidence with observed content. The implementation must recognize the unsupported location and use an explicitly defined fallback.

The numbers illustrate synthesis, not model calibration.

Illustrative case study

Illustrative case study - fictional

A fictional tutorial producer generates intermediate frames of a demonstrator moving a tool across a tabletop. A simple blend produces a faint duplicate along the tool's edge because one source shows the tool where the other shows the table.

The team tests an interpolation workflow with visibility weighting. It inspects both the synthesized frames and the source footage, paying particular attention to the newly exposed tabletop. Some errors remain where the estimated motion is wrong, so mask changes alone do not resolve them.

The producer keeps the original-speed segment where the interpolation is unreliable. Masks do not recover every hidden detail.

Common misconceptions and mistakes

A visibility mask is not necessarily binary. Soft weights are common, but their meaning depends on the method.
Occlusion and incorrect correspondence can produce similar symptoms. A mask is not a substitute for reliable alignment.
A zero weight means “do not use this contribution”, not “the true pixel is black”.
A mask cannot directly reveal a surface that no input image observed.
Segmentation, transparency and visibility answer different questions even when their representations look like similar grayscale images.

Business use

Businesses using frame interpolation, view synthesis or compositing can benefit from understanding how hidden regions are handled. The practical concern is whether the output mixes incompatible source content or invents unsupported detail.

Review moving boundaries, newly uncovered regions and thin structures rather than only smooth backgrounds. Keep intermediate masks available during technical evaluation when possible, and define how unsupported areas are treated. For instructional or documentary material, avoid presenting a synthesized detail as reliable evidence merely because the overall video looks smooth.

Check your understanding

Both source visibility weights are zero at a target location. Does the normalized blending equation establish a black output there?

Answer: No. The equation is undefined because its denominator is zero. The system has missing support and needs a separate fallback.

Related terms

Learn next

Compare an ordinary two-frame blend with a visibility-weighted blend around a moving foreground object. Mark where each source contains the desired background. Inspect unsupported pixels separately and document the convention used for zero and one.

Sources

Sources: Jiang and colleagues, Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation: https://arxiv.org/abs/1712.00080 ; MIT Computational Photography, Frame interpolation and slow-motion synthesis: https://people.csail.mit.edu/fredo/comp-photo-book/12-video-06-frame-interpolation-and-slow-motion-synthesis.html . The research abstract and relevant visibility-weighting explanation were reviewed; full model implementation was not audited.

Previous: Virtual support vector machine Next: Visible differences predictor (VDP)