Can merging recover a highlight clipped in every exposure?

Not from those measurements alone. At least one suitable measurement must preserve relevant information, or the result relies on additional assumptions.

Why normalize by exposure?

Different exposure times scale the recorded signal, so corresponding values need a common basis before combination.

Is tone mapping part of the merge?

It is often a later presentation step, although software may package both into one workflow.

AI Glossary / Computer vision and multimodal AI / Intermediate

HDR merging

Definition

HDR merging combines multiple measurements of the same scene to produce an image representation with useful information across a wider brightness range. A common approach merges photographs taken at different exposure times, using each where its measurements are reliable and accounting for their exposure differences.

What it means

A short exposure can preserve bright highlights that saturate in a long exposure. A longer exposure can record darker areas with a stronger signal. Merging aims to combine those complementary observations into a consistent estimate rather than simply averaging their displayed pixel values. The process requires correspondence between scene points, an appropriate response model and handling of unreliable measurements. Some multi-frame camera pipelines use related merging ideas with exposures that are not a conventional bracket. The term should therefore be tied to the actual capture method and desired output representation.

How it works

For exposure-bracketed merging, align images so corresponding pixels refer to the same scene content. Convert recorded values into a suitable linear or calibrated response domain and normalize for exposure. Weight or select measurements according to reliability, reducing the influence of saturation and very noisy values. Combine the valid estimates into an HDR representation.
Moving subjects, changing light and camera motion can violate the assumption that each image measures the same scene. Deghosting attempts to manage those differences, sometimes sacrificing the information available in some exposures. The merged data may then be tone-mapped for display. Merging and tone mapping are separate steps, and a pleasing output does not alone establish accurate radiometric reconstruction.

Three practical examples

A real-estate photographer merges a static room bracket to retain both interior detail and bright exterior features.
A visual-effects team captures an environment map with exposures that preserve very bright light sources for later rendering work.
An imaging researcher compares weighting and deghosting strategies on scenes containing both stationary structures and moving people.

Think of it this way

Think of measuring the same object with instruments set to different sensitivity ranges. One handles strong signals without overload; another resolves weaker signals. Combining them requires knowing the scale of each reading. The analogy is limited because photographs also need spatial alignment, and parts of the scene may move or change between measurements.

Worked example

Assume an ideal linear sensor with no offset. One scene point produces value 20 at exposure time 1 unit and value 80 at exposure time 4 units. Dividing by exposure gives the same relative scene estimate: 20/1=20 and 80/4=20. Averaging the raw values to get 50 would mix incompatible exposure scales.
Now suppose a brighter point would produce 300 in the longer exposure, but the sensor clips at 255. The clipped value divided by 4 is not a valid estimate of the original signal. A reliable merge should avoid treating that saturated measurement as ordinary evidence and use a suitable unsaturated exposure if available. Real cameras also require noise and response calibration.

Illustrative case study

Fictional example: Linden Media brackets a lobby scene containing a slowly moving visitor. Static walls merge cleanly, but the visitor appears in several positions. The editor uses a deghosting option that relies on one exposure for that region and checks whether its highlights and shadows remain acceptable. The team retains the component images so the choice can be revised. The case illustrates that motion handling can trade away some multi-exposure benefits rather than magically reconciling inconsistent observations.

Common misconceptions and mistakes

HDR merging is not simply averaging JPEG values. Exposure normalization matters, and nonlinear encoding must be handled appropriately. Saturated values do not preserve the missing intensity above the clipping limit. Alignment does not solve every moving-object problem. Deghosting can introduce artifacts or choose one exposure's limited information. Exposure fusion that directly creates a pleasing display image is not necessarily the same as reconstructing a calibrated radiance map. Finally, more frames are not always better if they add motion, lighting changes or redundant poor measurements.

Business use

Businesses should plan capture around the required output and scene behavior. Use stable framing and record exposure information, then inspect difficult regions rather than only the overall preview. Keep source frames and document merging settings. For measurement or rendering applications, distinguish relative radiance estimates from aesthetic blends and verify calibration requirements. For high-volume photography, include capture failures and manual deghosting effort in workflow estimates. A reliable process provides a fallback when the scene changes too much for a trustworthy merge.

Check your understanding

A linear pixel reads 30 at exposure 1 and 120 at exposure 4. Are these contradictory measurements of the scene?

Answer: No. Both imply the same relative value of 30 after dividing by exposure, assuming no clipping, offset or scene change.

Related terms

HDR (high dynamic range)
Exposure bracketing
Deghosting
Tone mapping
HDR
radiance map
camera response function
image alignment

Learn next

Learn linear exposure response, clipping and noise, then normalize a small bracket manually. Study alignment and motion failures before comparing radiance reconstruction with display-oriented exposure fusion.

Exposure fusion

Sources

Debevec and Malik, Recovering High Dynamic Range Radiance Maps from Photographs: https://www.pauldebevec.com/Research/HDR/debevec-siggraph97.pdf Authors' project: https://www.pauldebevec.com/Research/HDR Primary research excerpts consulted; full algorithm-specific and editorial review remain pending. The sensor arithmetic is an original idealized example.

Previous: HDR (high dynamic range) Next: HDR-VDP