Step by Step
V
VAE — encoding to a smooth, interpolatable space
A Variational Autoencoder encodes an image into a probability distribution (mean and variance), samples from it, and decodes back into an image — producing a smooth latent space that enables meaningful interpolation between images.
Example: smoothly interpolating between two face images by moving through the VAE's latent space, producing a gradual, meaningful morphing sequence between them.
G
GAN — two networks competing adversarially
A Generative Adversarial Network trains a Generator against a Discriminator adversarially, producing sharp, realistic results, but suffering from known challenges: mode collapse (limited output diversity) and general training instability.
Example: a GAN's Generator producing increasingly convincing fake face images while its Discriminator tries to get better at telling real from fake, with both networks pushing each other to improve.
D
Diffusion — reversing gradual noise addition
Diffusion models add noise to an image over many steps (T steps) during training, then train a network to reverse this process by denoising step by step, generating new images by starting from pure noise and progressively removing it.
Example: Stable Diffusion generating an image by starting with pure random static and gradually refining it through many denoising steps into a coherent picture.
⭐
Why diffusion now dominates — stability, diversity, text conditioning
Diffusion models (used by Stable Diffusion, DALL-E 3, and Midjourney) now dominate image generation specifically because they offer more stable training than GANs, better output diversity, and easier text-conditioning for generating images from text prompts.
Example: modern text-to-image tools overwhelmingly using diffusion rather than GANs or VAEs, specifically because of these stability, diversity, and text-conditioning advantages.
Applied Walkthrough
1
A team building a new text-to-image generation tool needs to choose among VAE, GAN, and diffusion approaches.
2
They consider GANs' well-known challenges — mode collapse (limited output diversity) and general training instability — both of which can make GANs difficult to train reliably.
3
Choosing diffusion instead, they get a more stable training process, better output diversity, and easier text-conditioning for generating images from text prompts.
4
This is exactly why modern leading text-to-image tools like Stable Diffusion, DALL-E 3, and Midjourney all use diffusion rather than the GAN or VAE approaches that were prominent in earlier eras of image generation.
Exam Application
Exams test whether you can describe all three generative approaches' core mechanisms (VAE: encode-sample-decode with smooth latent space; GAN: adversarial generator vs. discriminator; diffusion: noise addition and reversal) and whether you know diffusion has become the dominant approach for modern image generation specifically due to greater stability, diversity, and easier text conditioning.
⚠ Common Trap
The most common trap is assuming GANs are still the dominant image generation approach, since they were historically the most famous generative technique. In practice, diffusion models have now overtaken GANs (and VAEs) as the dominant approach for image generation specifically because of their training stability and output diversity advantages.
✓ Quick Self-Check
1. What does a VAE's smooth latent space enable?
Meaningful interpolation between images, by smoothly moving through the encoded distribution space.
Tap to reveal / hide
2. What are the two networks in a GAN, and how are they trained?
The Generator and Discriminator, trained adversarially against each other.
Tap to reveal / hide
3. What two known challenges do GANs commonly face?
Mode collapse (limited output diversity) and general training instability.
Tap to reveal / hide
4. How does a diffusion model generate an image?
By starting from pure random noise and progressively denoising it, using a learned reverse process.
Tap to reveal / hide
5. Why do diffusion models now dominate image generation over GANs?
More stable training, better output diversity, and easier text conditioning.
Tap to reveal / hide