Theoretical Foundations of Score-Based Modeling and a Case Study on Stochastic Localization
OPEN ACCESS
Loading...
Author / Producer
Date
2025
Publication Type
Student Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Diffusion models have emerged as a powerful framework for generative modeling, achieving state-of-the-art results in tasks such as image synthesis and density estimation. These models are typically formulated in a probabilistic setting, where the goal is to learn a generative process that, given i.i.d. samples from an unknown target distribution μ, can generate new samples from the same distribution. Diffusion models achieve this by learning the score function, i.e., the gradient of the log-likelihood.
After briefly comparing diffusion models with other approaches used in generative modeling, we introduce methods for learning the score function, such as score matching [3] and denoising score matching [4]. Once the score function is learned, we explore various sampling algorithms, including annealed Langevin dynamics [6], denoising diffusion probabilistic models (DDPM) [7], score-based generative modeling via SDEs [8], in particular with the OU process, and stochastic localization [9].
We then analyze key results from the paper by Montanari [9], which provides further insights into the empirical properties of these methods. In the end, following the approach of Shah et al. [10], we study the convergence properties of stochastic localization when applied to a two-component Gaussian mixture model.
All code used to produce the results and figures in this work is available at the author's Github.
Permanent link
Publication status
published
External links
Editor
Contributors
Examiner : Chen, Yuansi
Book title
Journal / series
Volume
Pages / Article No.
Publisher
ETH Zurich
Event
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Score-Based Generative Modeling, Diffusion Models, Denoising Score Matching, Stochastic Localization, Gaussian Mixture Models
Organisational unit
09819 - Chen, Yuansi / Chen, Yuansi