AMS Acheson J. Duncan Lecture Series | Youssef Marzouk

Location: Krieger 205

When: October 2nd at 1:30 p.m.

Title: From gradient flows to particle systems for sampling and quantization

Abstract: Sampling or otherwise summarizing complex probability distributions is a central task in applied mathematics, statistics, and machine learning, with myriad applications. Many modern algorithms for this task introduce dynamics in the space of probability measures and design these dynamics to achieve efficiency, scalability, and other desirable features.

In this vein, we will discuss two new interacting particle systems (IPS) for sampling and quantization. First is the problem of sampling from a target distribution whose unnormalized density is available. We introduce a mean-field ODE and a corresponding IPS that approximate a Fisher–Rao gradient flow from the reference to the target, and that can be integrated without access to gradients of the target density. This sampler is an instance of a broader class of kernelized samplers, useful in Bayesian inference and data assimilation, and we will discuss systematic approaches to designing such schemes.

Second is the problem of weighted quantization, i.e., summarizing a complex distribution with a small set of weighted Dirac measures. We study this problem from the perspective of minimizing maximum mean discrepancy via gradient flow in the Wasserstein–Fisher–Rao (WFR) geometry. This gradient flow yields an ODE system from which we further derive a fixed-point algorithm called mean shift interacting particles (MSIP). We show that MSIP extends the classical mean shift algorithm, used for identifying modes in kernel density estimates, and that it outperforms state-of-the-art methods for quantization.

One lesson from these efforts is that gradient flows offer a unifying perspective on how to design sampling dynamics, but that it can be useful to depart from canonical gradient flows in a principled way.

Zoom link: https://wse.zoom.us/j/93600407710?pwd=JBL8VsObRxX6MkhdjAUxCadqJDoZrZ.1