What does Jensen's inequality state?

For a curved function, the function of the average and the average of the function stand in a fixed order, one bounding the other, with equality only in special cases.

What is variational inference?

Approximating a hard posterior by optimizing a tractable substitute against a Jensen-derived bound, the main use of this machinery.

Why is the direction of the bound useful?

Because knowing your estimate is, say, always below the truth lets you reason safely about worst cases.

AI Glossary / AI and machine learning / Advanced

Jensen bound integration

Definition

Jensen bound integration is a technique for approximating an average that is hard to compute exactly: it uses Jensen's inequality, which relates the average of a curved function to the function of the average, to bound or approximate the quantity of interest.

What it means

Many quantities in probabilistic modeling are averages over uncertainty that cannot be computed in closed form: the expected behavior over all plausible parameter values, or over all possible hidden states.

Jensen's inequality says that for a curved function, the function of the average and the average of the function are related in a fixed direction, one bounding the other. That bound turns an impossible integral into a tractable, if approximate, target.

The trick anchors a family of approximation methods, most famously variational inference, where a hard posterior average is replaced by an easier optimization against a bound that is pushed as close to the truth as possible.

The output is therefore an approximation with a known direction: you know whether your estimate sits above or below the true value, which is often enough to make sound decisions and always better than an unknowable number.

For a business, this is deep machinery inside Bayesian tools and some modern generative models. The usable takeaway is that principled approximations exist with known error directions, and vendors using them should be able to say what is being bounded and how tight the bound is.

How it works

Identify the intractable average: an integral over parameters or hidden states that has no closed form.

Choose a tractable family of substitute distributions over which the average can be computed.

Apply Jensen's inequality to obtain a bound: an objective that under- or over-estimates the true quantity in a known direction.

Optimize the bound over the substitute family, bringing it as close to the true value as the family allows.

Report the approximation with its bound direction and, where possible, a check of tightness, so users know how much to trust it.

Three practical examples

A Bayesian demand model whose posterior averages are computed by variational inference built on a Jensen bound.
A topic model fitted through a bound-optimization loop rather than exact integration, making millions of documents tractable.
A research prototype reporting that its likelihood estimate is a lower bound, with the tightness measured on small cases where exact answers exist.

Think of it this way

Think of estimating the average depth of a lake you cannot drain. You cannot sound every point, but you can bound the answer: no point is deeper than the deepest sounding, none shallower than the shore. Clever placement of soundings tightens the bound. Jensen-style methods are clever sounding, with a guarantee about which side the answer lies on.

The limit: a bound is not the value. If the substitute family cannot reach the truth's neighborhood, the bound stays loose, and decisions needing the exact number need more computation or a different method.

Worked example

tightening the estimate

Suppose a pricing science team needs the expected revenue under a Bayesian model with no closed form. The figures are illustrative.

A variational approximation built on a Jensen bound computes in minutes what Monte Carlo sampling estimates in hours.

On small products where exact computation is feasible, the bound proves tight to within 2 per cent, so the team trusts it at scale.

The direction of the bound, a lower bound on likelihood, is documented in the model card so reviewers know the approximation's bias.

Illustrative case study

A fictional analytics vendor claimed exact Bayesian inference at interactive speeds, and a client's statistician recognized the claim as impossible for the model class in question.

Pressed, the vendor admitted the numbers came from a bound approximation, perfectly legitimate, but sold as exact, and the gap between the words cost them the renewal.

The invented lesson: an honest bound beats a dishonest exact. Approximation with a known direction is engineering; hiding the approximation is marketing.

Common misconceptions and mistakes

It is a hack with no guarantees. The inequality gives a guaranteed direction for the error, which is more honesty than many exact-looking approximations offer.
Bounded means wrong. Every computable answer to an intractable integral is an approximation; the bound tells you which side yours errs on and how hard you pushed it.
This matters only to theorists. It is the engine inside production Bayesian and generative tools; the practical question for buyers is what is being bounded and how tight it is.

Business use

Ask vendors of probabilistic models what is approximated and in which direction, since the answer reveals both honesty and competence.

Prefer methods that can demonstrate tightness on checkable cases over methods that ask for faith.

Keep the approximation details in the model documentation, because the next reviewer will need to know which numbers are bounds.

Check your understanding

Why is a bound with a known direction more useful than an unprincipled approximation?

Answer: Because the direction tells you which side of the truth your number sits on, so decisions can be made conservatively in the right direction instead of trusting a number with unknown bias.

Related terms

Uncertainty
Monte Carlo
Bayesian methods
variational inference
expectation
approximation

Learn next

AI for Business Owners covers how to question the precision of vendor numbers that look too good to be exact: https://aiforbusinessandlife.com/academy/ai-for-business-owners. [C2]

Sources

[S13] Minka, Statistical Learning/Pattern Recognition Glossary. Full URLs are in the source ledger. Verified 11 October 2026.

Previous: Jenks natural breaks optimization Next: Job-shop scheduling problem