Course Study Note
Day 3 — Variational Autoencoders
I studied why the encoder predicts a mean and variance, then connected reparameterization to reconstruction and KL losses.
- Date
- 2025-07-09
- Course
- 2025 Generative Models Intensive — From VAE to Stable Diffusion
Study checkpoint for this day
Study checkpoint for this day
Topics from the day
- VAE Architecture and ELBO
I diagrammed how a VAE learns a latent distribution rather than a single code.
- Reparameterization Trick
I checked how sampling is rewritten so gradients can pass through the computation.
From an autoencoder to a VAE
| A VAE predicts the mean and variance of (q_\phi(z | x)). I reread Auto-Encoding Variational Bayes to connect the approximate posterior, prior, and lower bound. |
Reparameterization
z = mu + std * eps does not remove randomness. It moves the random variable outside the learned parameters so gradients can reach both mu and std.
Two loss terms
Reconstruction preserves information from the input; KL divergence discourages the posterior from drifting too far from the prior. Overweighting the KL term can lead the decoder to ignore the latent code.
The saved latent-interpolation output was unavailable, so I stopped at the architecture and loss path and will rerun that comparison separately.