AI · Deep Learning · Computer Vision · Generative AI
VAE ELBO and Reparameterization
2026-08-01 · updated 2026-08-01 · Hyeongrok Ryu
Notes on why the encoder predicts a mean and variance and how reconstruction and KL losses interact.
- Type / level
- study-note · intermediate
- Tools
- Python, PyTorch
A checkpoint in the study sequence for this note.
A checkpoint in the study sequence for this note.
A checkpoint in the study sequence for this note.
A checkpoint in the study sequence for this note.
From a point to a distribution
| The VAE encoder predicts the mean and log variance of (q_\phi(z | x)), not one fixed latent code. |
Reparameterization
z = mu + std * eps moves randomness outside the learned parameters so gradients can reach mu and std.
Two loss terms
Reconstruction preserves input information; KL divergence keeps the posterior near the prior. Poor weighting can produce weak reconstructions or posterior collapse.
Shape checks
I verified that mu, logvar, eps, and z all use (batch, latent_dim) and made the KL reductions explicit.
Next rerun
The latent-interpolation output was not saved. I will vary β and plot reconstruction against latent continuity in a new run.
Sources used
- Auto-Encoding Variational Bayes — paper; ELBO and reparameterization