Is InfoVAE a single architecture?

No. It is primarily an objective family that can be used with suitable encoder and decoder architectures.

Is MMD-VAE the whole family?

No. It is a particular member using maximum mean discrepancy for distribution matching.

Does it guarantee disentangled semantic factors?

No. Encouraging latent information does not by itself specify which factors should be separated or what meaning each coordinate should have.

AI Glossary / Probabilistic machine learning / Advanced

InfoVAE

Definition

InfoVAE is a family of variational-autoencoder training objectives designed to give more explicit control over data modeling, latent-variable information and approximate inference. It adjusts the balance between reconstruction, prior matching and the information carried by the latent code.

What it means

A variational autoencoder learns an encoder distribution q(z|x) and a decoder p(x|z). A flexible decoder may model data well while making little use of z. If the purpose includes learning informative features, that behavior can be undesirable even when a likelihood-related objective looks good.

InfoVAE makes some of these trade-offs explicit. The name refers to the framework introduced by Zhao, Song and Ermon, not to every VAE that uses an information-related term.

One important distinction is between matching the prior for each individual input and matching the aggregated latent distribution across the data. Those are different requirements. An aggregate can match the prior while individual encodings still retain substantial information about their inputs.

How it works

The aggregated posterior is q(z) = integral p_data(x)q(z|x) dx. It describes latent samples produced by first drawing an input from the data distribution and then encoding it.

InfoVAE objectives adjust how strongly this aggregate is matched to the prior and how much dependence between x and z is encouraged. Equivalent formulations make the objectives trainable without directly evaluating every apparently difficult information quantity.

MMD-VAE is one member of the family, using a kernel-based maximum mean discrepancy penalty to compare aggregated latent samples with prior samples. Its kernel, weighting and estimation procedure matter. InfoVAE is broader than MMD-VAE, and changing the objective does not automatically ensure successful optimization or useful semantic features.

Three practical examples

A representation-learning experiment tests whether an autoencoder's latent code retains useful distinctions among inputs.
A generative-modeling study compares reconstruction, prior-sampled outputs and inference quality under several objective weights.
A research team uses an MMD-based aggregate-matching term while separately evaluating the learned features on a downstream task.

Think of it this way

Think of a filing system whose overall distribution of labels follows an agreed pattern while individual files still receive informative labels. Requiring every file to receive a randomly chosen label from the same distribution would be a much stronger and less useful restriction.

The analogy illustrates aggregate versus individual matching. It does not imply that a neural model will discover an intuitive filing scheme without supervision or evaluation.

Worked example

Matching an aggregate without losing all information

Consider two equally likely inputs, A and B, and a latent code z that can be 0 or 1. Let the desired prior assign probability 0.5 to each code.

Encoder one always maps A to 0 and B to 1. Across both inputs, its aggregated q(z) is exactly (0.5, 0.5), matching the prior. The code identifies the input perfectly, so its mutual information with the input is 1 bit.

Encoder two produces 0 or 1 with equal probability regardless of the input. Its aggregate is also (0.5, 0.5), but the code reveals nothing about whether the input was A or B. Its mutual information is zero.

For encoder one, the KL divergence between each individual encoding and the uniform prior is log(2), about 0.6931 nats. For encoder two it is zero. Yet the aggregate-to-prior KL is zero for both.

This discrete teaching example exposes the distinction between regularization targets. It is not a complete InfoVAE training run, and it does not prove that an aggregate-matching objective alone always selects the informative encoder. Reconstruction, information preferences, model capacity and optimization also matter.

Illustrative case study

Fictional case: exploring useful image features

A fictional design-research team trains a VAE on an internal image collection. Generated images appear plausible, but changing the inferred latent code has little influence on a powerful decoder's output.

The team tests an InfoVAE-family objective and measures several outcomes separately: reconstruction, latent dependence, prior-sampled outputs and performance on a specified image-retrieval evaluation. It documents objective weights and the estimator used for aggregate matching.

The team avoids declaring the representation meaningful merely because an information statistic rises. It inspects whether the retained distinctions serve the retrieval task. This fictional case illustrates a research comparison without inventing performance gains.

Common misconceptions and mistakes

InfoVAE is not the same as the InfoNCE contrastive objective. Both involve information-related ideas, but their constructions and roles differ.

Matching the aggregate posterior to a prior does not force every input's posterior approximation to equal that prior. The worked example shows why this difference can matter.

More information is not automatically more useful information. A latent code might preserve irrelevant image details or record-specific artifacts. Nor should an adjusted objective automatically be reported as the standard ELBO; its mathematical interpretation depends on the chosen coefficients and divergence terms.

Business use

Use InfoVAE as a research option when the project needs both generative modeling and informative latent features. Define what useful means before selecting an objective: retrieval, controlled reconstruction or another measurable purpose.

Compare with a conventional VAE using matched data and an appropriate computing budget. Report each evaluation dimension, because a better reconstruction score need not imply better prior sampling or better inference. For an MMD-based implementation, record the kernel and estimator alongside the objective weights so results can be interpreted and reproduced.

Check your understanding

Two encoders have the same aggregated latent distribution. Must they carry the same information about their inputs?

Answer: No. In the worked example, one encoding identifies the input while the other is independent of it, even though both aggregates match the same uniform prior.

Related terms

Variational autoencoder
Mutual information
Representation learning
aggregated posterior
maximum mean discrepancy
posterior collapse
evidence lower bound

Learn next

Compare individual posterior-to-prior matching with aggregate matching using a small discrete example. Then evaluate whether a learned code supports a specific downstream task rather than assuming that latent information equals semantic usefulness.

Sources

Primary paper: Zhao, Song and Ermon, InfoVAE: Balancing Learning and Inference in Variational Autoencoders: https://arxiv.org/pdf/1706.02262

Author tutorial explaining the MMD-VAE member: https://ermongroup.github.io/blog/a-tutorial-on-mmd-variational-autoencoders/

The two-encoder calculation and design-research case are original teaching examples.

Previous: Informative vector machine Next: Infrastructure as a service