AI · Deep Learning · Computer Vision · Generative AI

VAE ELBO and Reparameterization

2026-08-01 · updated 2026-08-01 · Hyeongrok Ryu

Notes on why the encoder predicts a mean and variance and how reconstruction and KL losses interact.

Series
2025 Generative Models Intensive · 3
Type / level
study-note · intermediate
Tools
Python, PyTorch
01From a point to a distribution

A checkpoint in the study sequence for this note.

02Reparameterization

A checkpoint in the study sequence for this note.

03Two loss terms

A checkpoint in the study sequence for this note.

04Shape checks

A checkpoint in the study sequence for this note.

A compact concept path generated from this post's table of contents.

From a point to a distribution

The VAE encoder predicts the mean and log variance of (q_\phi(z x)), not one fixed latent code.
VAE input, encoder, mean, variance, latent code, and decoder
I separated learned parameters from the random sample.

Reparameterization

z = mu + std * eps moves randomness outside the learned parameters so gradients can reach mu and std.

Two loss terms

Reconstruction preserves input information; KL divergence keeps the posterior near the prior. Poor weighting can produce weak reconstructions or posterior collapse.

Shape checks

I verified that mu, logvar, eps, and z all use (batch, latent_dim) and made the KL reductions explicit.

Next rerun

The latent-interpolation output was not saved. I will vary β and plot reconstruction against latent continuity in a new run.

Sources used

Publication first-page preview