Is diffusion the same as a GAN?

No. GANs use an adversarial training setup; common diffusion models learn a denoising-related objective and use a different generation process.

Why can generation take several steps?

The sampler progressively updates a noisy representation using the learned model, although accelerated and distilled variants can reduce the number of steps.

What is latent diffusion?

Diffusion performed in a learned latent representation rather than directly in full-resolution pixel space.

AI Glossary / Generative AI and language / Intermediate

Diffusion model

Definition

A diffusion model is a generative model that learns to produce data through a process related to reversing progressive corruption. In a common image-generation setup, it starts with random noise and repeatedly uses a trained network to move toward a structured image.

What it means

The name comes from a forward process that gradually adds noise to training examples. As corruption increases, recognizable structure is obscured. The model learns information useful for the reverse direction, allowing generation to begin from a simple noise distribution rather than an existing photograph.

This is a model family, not one product or a synonym for every image generator. Diffusion methods can be used with different data types, architectures, training targets and sampling procedures. Text prompts are an optional conditioning mechanism, not an essential ingredient of the basic concept.

Generation does not mean recovering a particular hidden original image from pure noise. The system samples a result shaped by learned patterns and any conditioning inputs. It can produce plausible content without establishing that the depicted objects, relationships or events exist.

How it works

During training, a common denoising diffusion method selects a clean example, a noise level and a random noise sample. It creates the corresponding corrupted input and trains a network, informed of the noise level, to predict a target such as the added noise. Other parameterizations predict clean data or related quantities.

During generation, a sampler uses the network's predictions and a specified schedule to update an initially noisy representation. The update is more structured than simply subtracting the predicted noise repeatedly. It depends on the mathematical parameterization, noise level and sampling algorithm, and may include additional randomness.

Pixel-space models work directly with image representations. Latent diffusion instead performs the process in a learned compressed space and uses a decoder to turn the final latent into an image. Conditioning can enter through text embeddings, class labels or other signals; cross-attention is one way to connect text and visual features.

Training noise steps and generation steps need not be identical. Different samplers and acceleration methods change the quality - cost tradeoff. More steps are not an unconditional guarantee of better output.

Three practical examples

A design team generates alternative scene concepts from text descriptions using a conditioned diffusion system.
An image-editing workflow fills a selected region while using the surrounding image as a constraint.
A research system generates synthetic examples from a learned data distribution for evaluation or experimentation, with separate checks on their usefulness.

Think of it this way

Think of learning how structure emerges when a very noisy sketch is progressively clarified. Each stage uses experience with patterns at that level of uncertainty.

The analogy does not mean an intact original is always waiting underneath. Starting from newly sampled noise creates a new sample, and the learned clarification process can invent details. A realistic-looking result is not evidence of historical or physical truth.

Worked example

Understanding one training corruption

For a one-number teaching example, let the clean value be x₀ = 0.5. Choose a cumulative signal coefficient of 0.64, so its square root is 0.8 and the corresponding noise coefficient is √(1 − 0.64) = 0.6. Draw noise ε = −1.

The corrupted value is xₜ = 0.8 × 0.5 + 0.6 × (−1) = −0.2. A noise-prediction training objective asks the network to predict ε from the corrupted input and noise-level information.

If an idealized prediction were exactly −1, the clean-value estimate from this relation would be [−0.2 − 0.6 × (−1)] / 0.8 = 0.5. In reality, one noisy observation does not uniquely reveal its original, and a learned estimate can be wrong. This scalar calculation illustrates a training relation, not the complete multi-step image-generation algorithm.

Illustrative case study

A fictional creative studio, Birch Scenes, uses a diffusion tool to propose product backgrounds. Some outputs look convincing but alter the shape of a supplied product or invent text on its label.

The studio separates visual concept approval from product-accuracy review. It retains controlled product imagery where exact geometry and labels matter, and checks generated surroundings for unwanted objects or misleading implications. It records the model and generation settings used for approved drafts.

The case illustrates a useful boundary: diffusion can support visual exploration, but aesthetic quality and faithful representation are different criteria. A polished image can fail the task even when it looks natural.

Common misconceptions and mistakes

Diffusion generation only removes sensor noise from an existing image. Generative sampling can begin from newly drawn noise.
Every diffusion model requires a text prompt. Unconditional and other conditioned forms exist.
The network must always be a U-Net. Different architectures can implement the learned component.
A fixed seed guarantees identical output everywhere. Reproducibility also depends on the model, sampler, settings and execution environment.
Generated examples automatically provide unbiased training data. Their distribution reflects the model and conditioning process.

Business use

Business value depends on the application: concept generation, controlled editing and synthetic-data experiments require different evaluation criteria. Measure task suitability, consistency, turnaround time and the review effort needed to make outputs usable.

For workflows requiring exact products, quantities or textual details, assess those requirements directly instead of relying on visual attractiveness. Also document the generation configuration. A changed checkpoint, prompt encoder or sampler can alter behavior even when the visible interface looks the same.

Check your understanding

A generated scene contains a plausible-looking machine part. Does the model's denoising process establish that the part has a physically workable design?

Answer: No. Plausibility under learned patterns is different from engineering validity. The relevant geometry and physical constraints require separate checks.

Related terms

Learn next

Compare the forward corruption process with the reverse generation process on a diagram. Label what is known during training and what is estimated during generation. Then read latent diffusion and sampling schedules to understand how representation and sampler choices affect speed and output behavior.

Latent diffusion

Sources

Sources: Ho, Jain and Abbeel, Denoising Diffusion Probabilistic Models: https://arxiv.org/abs/2006.11239 Song, Meng and Ermon, Denoising Diffusion Implicit Models: https://arxiv.org/abs/2010.02502 Rombach and colleagues, High-Resolution Image Synthesis with Latent Diffusion Models: https://arxiv.org/abs/2112.10752 MIT computational photography glossary, diffusion model: https://people.csail.mit.edu/fredo/comp-photo-book/glossary.html

Previous: Differentially private stochastic gradient descent Next: Digidot chart