Is assimilation just averaging?

No. A scalar weighted average is a special case; realistic methods account for dynamics, indirect measurements, dependencies, and constraints.

In weather and climate work, it applies a consistent assimilation system retrospectively to historical observations to estimate past states.

Can AI replace the physical model?

Some approaches use learned models or learned components, but they still require appropriate observations, uncertainty treatment, and validation for the intended application.

AI Glossary / Statistics and probability / Advanced

Data assimilation

Definition

Data assimilation is the process of combining observations with a model-based estimate to infer the state of a changing system. It accounts for uncertainty in both sources and often updates the starting conditions for a subsequent forecast. It is widely used in weather prediction and other applications where observations are incomplete, noisy, or collected at different places and times.

What it means

The state is a description of the system at a particular time, such as temperatures across a region or the position and velocity of a moving object. A forecast model predicts how that state evolves. Observations supply partial evidence about what actually happened.

Assimilation reconciles those sources rather than simply replacing the forecast with every new measurement. A sensor may be noisy, biased, delayed, or measuring something indirectly related to the model state. The updated estimate is often called an analysis, and its uncertainty matters as much as its central value.

This is a statistical estimation process, not merely importing records into a database.

How it works

Begin with a background estimate and its uncertainty. Use an observation operator to translate the model state into quantities comparable with the measurements. The difference between observed and predicted measurements is an innovation or departure.

An assimilation method uses that discrepancy, error assumptions, and system constraints to update the estimate. Sequential approaches include Kalman-filter variants and particle filters. Variational methods optimize a cost function, sometimes across a time window, to find a state or trajectory consistent with background information and observations.

Propagate the updated state through the dynamical model before incorporating later evidence. Errors can be correlated across locations, variables, and observations, so realistic systems need more than a separate weighted average for each measurement. Bias correction, quality control, and appropriate model-error treatment are essential parts of the workflow.

Three practical examples

A weather-prediction system combines observations with a short-range forecast to estimate initial conditions for the next forecast.
An industrial monitoring system reconciles sensor readings with a process model to estimate internal conditions that cannot all be measured directly.
A vehicle-tracking system combines motion predictions with noisy position measurements, updating its estimated state and uncertainty as observations arrive.

Think of it this way

Think of navigating with both a map-based estimate of your progress and an imperfect location reading. The map and motion model preserve continuity; the reading provides fresh evidence. A sensible update considers how trustworthy each is. Blindly following either source can be worse than combining them carefully, especially when one temporarily becomes unreliable.

Worked example

Consider an original scalar Gaussian example. A background estimate for a temperature is 20 degrees, with error variance P = 4 squared degrees. A direct sensor observation is 26 degrees, with error variance R = 9. Assume unbiased errors, independence between background error and observation noise, and a direct observation of the same state.

The scalar update weight on the observation discrepancy is K = P/(P+R) = 4/13, approximately 0.30769. The innovation is 26 - 20 = 6 degrees. The updated estimate is 20 + (4/13) times 6, approximately 21.84615 degrees.

Under these assumptions, the updated error variance is (1-K)P = 36/13, approximately 2.76923 squared degrees. The observation receives less weight because its error variance is larger than the background variance. A simple average of 23 degrees would ignore that difference.

This is one assimilation update, not a complete time-evolving filter. A subsequent forecast must propagate both the state and uncertainty through a model and account for process noise. Correlated errors or sensor bias would invalidate the simple weighting assumptions.

Illustrative case study

In this fictional case, Harbor Cooling estimates temperatures inside a storage facility using a thermal model and a sparse sensor network. Some sensors are close to doors, so their readings do not directly represent the volume-average temperature the model tracks.

Engineers define how each measurement relates to the model state and assess sensor uncertainty. They compare updated estimates against independent reference measurements and inspect persistent innovations that could indicate calibration problems or model error.

An AI component helps model a difficult correction, but the team still validates the combined estimation system. This invented case illustrates assimilation design; it does not claim measured energy savings or establish a safe control policy.

Common misconceptions and mistakes

More observations do not automatically improve an estimate. Duplicated information, correlated errors, systematic bias, or a wrong observation operator can make updates overconfident or inaccurate.

Data assimilation is not identical to training a machine-learning model. State estimation and parameter learning are distinct tasks, although modern systems can combine them and share optimization or Bayesian methods.

Do not call the analysis ground truth. It is an estimate influenced by the model and assimilated observations. Evaluating it only against the same observations used in the update can give an overly favorable impression. Independent checks and forecasts beyond the assimilation window provide additional evidence.

Business use

Businesses can benefit when decisions depend on a changing, partially observed system: logistics, energy operations, equipment monitoring, or environmental services. The objective is a useful state estimate and forecast, not simply a larger stream of sensor data.

Define the operational state, observation latency, acceptable uncertainty, and consequences of incorrect updates. Monitor sensor quality and innovations, record model versions, and provide a fallback for missing or suspect observations. Compare the complete workflow with simpler baselines using evidence relevant to the decision, including performance during unusual conditions rather than only routine operation.

Check your understanding

Check your understanding: Why does the updated temperature stay closer to 20 than to 26?

Answer: The background has the smaller stipulated error variance.

Does a variance of 2.76923 mean the estimate is certainly correct?

Answer: No; it quantifies uncertainty under the assumptions.

Why is an observation operator needed?

Answer: Measurements may represent a different quantity or scale from the model's state variables.

Related terms

Kalman filter
State estimation
observation operator
innovation
background error
ensemble filter
variational assimilation
reanalysis
sensor fusion

Learn next

Extend the scalar update to a two-variable system with correlated errors. Then compare filtering, which uses evidence available up to a time, with smoothing, which can also use later observations to estimate earlier states.

Sources

Sources: ECMWF, Earth system data assimilation fact sheet, https://www.ecmwf.int/en/about/media-centre/focus/2025/fact-sheet-earth-system-data-assimilation. Bonavita and colleagues, Data assimilation or machine learning?, https://www.ecmwf.int/en/newsletter/167/meteorology/data-assimilation-or-machine-learning. The scalar Gaussian calculation and fictional case are original teaching examples, with assumptions explicitly stated.

Previous: Data architecture Next: Data auditing