Diffusion Model
The AI behind image generation: learning to reverse the process of adding noise.
Diffusion models are the technology behind DALL·E, Midjourney, Stable Diffusion, and most image/video generation systems. They work by learning to reverse a noise-adding process.
Training: Take a real image. Add random noise to it step by step until it's pure static. Train a neural network to predict and remove the noise at each step.
Inference (generation): Start with pure random noise. Apply the trained denoising network repeatedly, guided by a text prompt. After 20–50 steps, noise becomes a coherent image that matches the description.
Key components:
- U-Net / DiT — the denoising neural network
- CLIP / text encoder — converts your prompt into the vector that guides denoising
- VAE — compresses images to a latent space (Latent Diffusion Models, including Stable Diffusion)
- Classifier-free guidance (CFG) — controls how strongly the image follows the prompt vs. creative freedom
Newer architectures: Flow Matching (used in Stable Diffusion 3, Flux) is faster and higher-quality than traditional DDPM diffusion. Video diffusion (Sora, Kling, Gen-3) extends the same idea to temporal sequences.
In plain terms
A sculptor starting with a random block of marble, using a blueprint (your prompt) to progressively chip away noise until the intended figure emerges.