GANs: generator versus discriminator
Understand how adversarial training works, why it is unstable and where GANs still make sense
A taste of a lesson
My GAN's discriminator loss drops to almost zero and the generated images stop improving. What is happening?
Your discriminator has become too strong. It rejects every fake with near total confidence, so the gradients it sends back to the generator carry little useful information about how to improve. First, confirm you use the non saturating generator loss. Then rebalance: lower the discriminator's learning rate, update it less often, add noise or label smoothing to its inputs, or use spectral normalisation or a gradient penalty to keep it smoother. Check sample images every few hundred steps rather than trusting loss values. Quick check: if the discriminator were too weak instead, what would you expect the generated images to look like?
Written by the teacher as an example. In your lesson the tutor answers your own questions, and like any AI it can be wrong.
What you will be able to do
- Explain the generator and discriminator game and the losses each player optimises
- Run through the alternating GAN training loop and its practical settings
- Recognise mode collapse and instability and describe common remedies
- Interpret image generation metrics with their limitations in mind
- Judge when a GAN or a diffusion model fits a generation task better
Lesson plan
- 1 The forger and the inspector Set up the generator and discriminator and the idea of adversarial training. Start
- 2 Losses and the training loop Write the GAN objectives and run through one round of alternating updates. Start
- 3 Mode collapse and instability Recognise the classic GAN failures in outputs and loss curves. Start
- 4 Stabilising GANs Understand Wasserstein style critics, normalisation and other common fixes. Start
- 5 Conditional GANs and image to image tasks Add labels or input images to control what the generator makes. Start
- 6 Evaluation and where GANs stand today Evaluate generated images carefully and compare GANs with diffusion models. Start
Try asking
About this tutor
For learners comfortable with neural network training who want to understand generative adversarial networks properly. You will set up the two player game between a generator and a discriminator, write the losses in words and symbols, and walk through the alternating training loop. Then you study the classic problems, mode collapse and unstable training, and the fixes people developed, including the non saturating generator loss, Wasserstein style critics and conditional GANs. You also learn how generated images are evaluated, with honest caveats about popular metrics, and why diffusion models have largely replaced GANs for image generation while GANs remain useful for fast single step tasks.
Reviews
4.0
3 ratingsSample
- Ravi N.Sample
The forger and inspector framing made the training loop easy to remember, and the stabilisation lesson fixed my collapsing face generator.
- Thomas B.Sample
Clear, but I was hoping for more hands on code. It is concept focused, which the description does say.
- Elif Y.Sample
Honest about GANs versus diffusion, which I appreciated. The evaluation section on FID's blind spots was the most useful part for my thesis.
About the teacher
Generative models and representation learning, explained with intuition first and maths second
9 tutors 335 lessons taught Sample
I teach how models learn useful representations and how they generate new data: autoencoders, GANs, diffusion models, self supervised learning and language model pretraining. I came to this through research engineering work where we had to decide which kind of model was worth the compute, so I teach with trade offs in mind. Each topic starts with a picture or...
See Mateo's profile and tutorsMore like this
Other tutors on the same or nearby topics.