Does it remove the background from a single photograph?

That is a different common meaning of background removal. Video background subtraction usually compares observations with a scene model to detect departures.

Can it work with a moving camera?

Some systems compensate for camera motion or use more complex models, but a simple fixed-background method is easily disrupted by viewpoint changes.

Why can a stopped object disappear from the foreground mask?

An adaptive model may eventually treat its persistent appearance as part of the background, depending on the update rules.

AI Glossary / Computer vision and multimodal AI / Intermediate

Background subtraction

Definition

Background subtraction is a computer-vision technique that identifies image regions differing from an estimated background scene. It commonly produces a foreground mask from video and is often used as an early step in detecting activity viewed by a stationary camera.

What it means

A fixed camera repeatedly observes much of the same scene. If the system learns what normally appears at each location, a new object can create a difference from that background model.

The word subtraction is broader than a single arithmetic operation. A basic method compares the current frame with a reference image, while adaptive methods maintain statistical models that change over time. The output identifies departures from the model, not necessarily named objects.

A foreground mask is therefore not a complete interpretation of the scene. A moving shadow, swaying tree, lighting change or camera vibration can also differ from the estimated background. Conversely, an object that remains still long enough may become part of an adaptive background model.

How it works

The system first establishes a background estimate from one or more observations. For each new frame, it compares pixel values or features with that estimate and marks sufficiently unusual regions as foreground.

Adaptive methods update their models to accommodate gradual changes. A mixture model can represent several recurring appearances at one location, while other methods use collections of recent samples or nearest-neighbor comparisons. Parameters control how quickly new observations influence the background and how much difference counts as foreground.

The raw mask often needs further processing to remove isolated noise, fill small gaps or form connected regions. Tracking and object recognition can then interpret those regions. Shadow handling, camera-motion compensation and scene-specific evaluation remain separate concerns; a foreground mask alone does not solve them automatically.

Three practical examples

A loading-bay camera proposes regions of activity for a later vehicle or package detector.
A laboratory camera identifies an object entering a controlled tabletop scene before measuring its movement.
A wildlife-video workflow flags frames with foreground changes so reviewers can focus on candidate activity rather than watch every frame.

Think of it this way

Think of comparing a room with a mental picture of how it usually looks. A newly placed box stands out, but so can a changing shadow or a chair that has been moved. The analogy is limited: the algorithm follows numerical update rules and does not inherently know whether a change is an object, an illumination effect or something important to the user.

Worked example

Suppose a simple grayscale background estimate at a pixel is 100, and the foreground threshold is an absolute difference greater than 20. A new value of 108 differs by 8 and remains background. A value of 145 differs by 45 and is marked foreground.

Now consider an illustrative background update Bnew = 0.9Bold + 0.1I. If the current value is 110, the estimate moves from 100 to 101. Repeated observations near 110 gradually shift the model towards that appearance.

This update is not necessarily applied to every detected foreground pixel; practical systems often control updates to avoid absorbing new objects too quickly. The example demonstrates the trade-off: adaptation handles gradual change but can eventually incorporate persistent objects into the background.

Illustrative case study

Fictional example: Parcelwatch Operations analyses a fixed view of a sorting area. Its initial foreground mask responds to moving parcels, but also to shadows from workers and a brief camera shake.

The team reviews annotated clips covering ordinary motion, empty periods and changing light. It adds appropriate mask cleanup and sends candidate regions to an object detector before counting parcels. Update settings are tested with a parcel that pauses on the belt so that the system's behavior is understood.

The team reports missed and false activity separately. This fictional workflow illustrates why background subtraction is an intermediate signal, not an automatic guarantee of accurate counting.

Common misconceptions and mistakes

Foreground does not necessarily mean moving object, and background does not necessarily mean unimportant. A newly stationary item may initially be foreground and later be absorbed by the model.

Simple frame differencing is related but not identical to a maintained background model: comparing adjacent frames can miss the interior of a moving object or fail to represent the stable scene. A camera pan can make most of the image look changed. Thresholds that work indoors may fail outdoors under moving foliage, reflections or rapidly changing illumination.

Business use

Background subtraction can reduce the area or time a downstream system must inspect, especially with stable cameras and predictable scenes. Its value lies in useful activity proposals rather than semantic understanding by itself.

Define the target events and evaluate representative quiet periods as well as activity. Record camera placement, update rate, shadow settings and mask-processing steps. Monitor changes after the camera moves or the environment is rearranged. If the output supports a count or alert, validate that final result instead of treating a visually clean mask as sufficient evidence.

Check your understanding

A foreground mask activates when sunlight suddenly changes across an empty room. Does that prove an object entered?

Answer: No. The image changed relative to the background model, but the cause may be illumination. Additional interpretation is needed.

Related terms

Gaussian mixture model
Foreground mask
frame differencing
adaptive background model
shadow detection
connected components
object tracking
motion detection

Learn next

Test a simple background-subtraction method on a short fixed-camera clip. Include an empty interval, a moving object, a stopped object and a lighting change. Inspect the mask before and after cleanup, then measure whether it helps the actual counting, tracking or review task.

Sources

OpenCV, How to Use Background Subtraction Methods, official tutorial covering MOG2 and KNN: https://docs.opencv.org/4.10.0/d1/dc5/tutorial_background_subtraction.html OpenCV, BackgroundSubtractor interface: https://docs.opencv.org/4.x/d7/df6/classcv_1_1BackgroundSubtractor.html The threshold arithmetic and parcel case are original examples, not reported performance measurements.

Previous: Background knowledge Next: Backjumping