Does a linear inverse problem always have one solution?

No. It can have no exact solution, one solution, or many, depending on the measurements and model.

Why add regularization?

It can stabilize estimation or select among plausible solutions by expressing assumptions about the unknown.

Can the reconstruction algorithm be nonlinear?

Yes. Linear describes the forward model; nonlinear priors, constraints, or learned methods can be used to estimate the unknown.

AI Glossary / Computer vision and multimodal AI / Intermediate

Linear inverse problem

Definition

A linear inverse problem estimates an unknown quantity from measurements related to it by a linear forward model. A common form is b = Ax + noise, where x is the unknown, A describes measurement formation, and b contains the observations. Imaging examples include deblurring and reconstruction from incomplete measurements under suitable linear assumptions.

What it means

The forward question asks what measurements a known object would produce. The inverse question works backward: which object could have produced the observed measurements? Linear refers to the forward relationship, not to whether the reconstruction is easy or whether its algorithm is itself linear.

Some measurement systems discard information or mix it so strongly that different unknowns produce the same or very similar data. Noise can make the distinction harder. A problem may therefore lack a unique solution or be unstable even when its forward model is linear.

Assumptions such as smoothness, sparsity, or nonnegativity can guide reconstruction. They contribute information beyond the measurements and are not facts established by the data alone.

How it works

Define a forward operator that predicts measurements from a candidate reconstruction. It may be represented as a matrix or implemented as a function when explicitly storing the matrix would be impractical.

Compare predicted measurements with observed ones using a data-fidelity term suited to the noise model. Least squares is a common choice when squared residuals are appropriate.

Add a regularization term or constraints to express useful prior assumptions. A typical objective balances measurement mismatch against a penalty, with a parameter controlling their relative influence.

Solve the resulting optimization problem and evaluate both the residuals and the reconstructed content. A low residual does not prove that the recovered object is correct, particularly when many candidates fit the data. Check conditioning, calibration, and sensitivity to noise and regularization choices.

Three practical examples

1. Deblurring: A system models a captured image as a known linear blur applied to a sharper image, then estimates the sharper image while controlling noise amplification.

2. Compressive imaging: A camera records weighted combinations of scene samples and reconstructs an image using assumptions that compensate for having fewer measurements than unknowns.

3. Missing-data recovery: A research pipeline estimates unobserved image values under a sampling model and an explicit prior, then evaluates how reliably important details are retained.

Think of it this way

Think of receiving the totals from several shopping baskets and trying to infer the price of each item. Enough independent totals may determine the prices, but repeated or incomplete combinations leave ambiguity. The analogy is limited because imaging measurements also contain noise and can involve millions of coupled unknowns, requiring numerical methods and modeling assumptions.

Worked example

Suppose the unknown vector contains two values, x1 and x2. The first measurement gives their sum: x1 + x2 = 10. By itself, this permits many solutions, including (4, 6), (5, 5), and (8, 2).

A second independent measurement gives x1 − x2 = 2. Adding the equations yields 2x1 = 12, so x1 = 6 and x2 = 4. In this ideal noiseless example, the two independent measurements determine the unknowns uniquely.

Now return to only the sum measurement. Choosing the solution with the smallest sum of squared values selects (5, 5), because equal division minimizes x1 squared plus x2 squared subject to the fixed sum. That selection is a consequence of the minimum-norm criterion. It does not prove that the original values were equal.

If the real pair was (6, 4), the minimum-norm reconstruction still reproduces the observed sum exactly while getting each component wrong. This illustrates why measurement fit and reconstruction truth are distinct, and why an appropriate prior matters when information is missing.

Illustrative case study

Fictional case study: An imaging startup reconstructs a stationary object from a small set of coded measurements. Its initial method produces a smooth image that matches the recorded data closely.

When tested on a target containing a thin stripe, the reconstruction weakens that feature. The team finds that the chosen smoothness preference discourages the stripe even though several candidate images fit the measurements.

It evaluates alternative priors and additional measurement patterns using targets with known structure. The report distinguishes data consistency from recovery of fine features and includes examples where the reconstruction is ambiguous. The scenario illustrates model evaluation rather than a claim that a particular reconstruction method guarantees missing-detail recovery.

Common misconceptions and mistakes

Linear does not mean invertible. A matrix can be rectangular, rank-deficient, or so poorly conditioned that small measurement errors cause large reconstruction changes.

Directly computing a matrix inverse is not the universal solution. Many problems need least-squares methods, regularization, constraints, or iterative solvers.

Regularization does not add new measurements. It expresses preferences that can stabilize an estimate while also introducing bias.

A learned reconstruction method can solve a problem with a linear forward model, but its output may depend strongly on training data. The physical model's linearity does not eliminate the need to evaluate learned assumptions.

A visually convincing reconstruction can fit the measurements yet contain incorrect details, especially when the problem is underdetermined.

Business use

Linear inverse models provide a useful framework for designing imaging systems and explaining reconstruction tradeoffs. They make it possible to separate the measurement process, noise assumptions, and reconstruction preferences.

Evaluate with known test objects, realistic noise, and calibration perturbations. Report task-relevant errors alongside residuals and visual quality. For important decisions, examine how plausible reconstructions vary with the prior or regularization strength. If the result changes substantially while fitting the data equally well, that ambiguity should inform how the reconstructed image is used.

Check your understanding

If the only measurement is x1 + x2 = 12, does a reconstructed pair (6, 6) prove that both original values were 6?

Answer: No. Many pairs fit the same measurement. Selecting (6, 6) requires an additional criterion or assumption.

Related terms

Least squares
Regularization
Compressive sensing
Forward model
ill-posed problem
null space
deconvolution

Learn next

Solve small linear systems with unique and nonunique solutions, then add noise and compare unregularized and regularized estimates. Track measurement residual and reconstruction error separately to see why one does not determine the other.

Sources

Gordon Wetzstein, Stanford Computational Imaging, Notes on Solving Regularized Inverse Problems: https://stanford.edu/class/ee367/reading/ee367_notes_inverse_problems.pdf

The primary course notes were opened and the linear measurement model and regularization formulation checked. Two-variable calculations and the startup scenario are original teaching illustrations.

Previous: Linear interpolation smoothing Next: Linear least squares