Join the DSI Foundations of Data Science Center on Friday, May 2 for a day-long event featuring two tutorial-style talks from leading experts. The program will explore the statistical and algorithmic tradeoffs of score-based losses in generative modeling, as well as fresh perspectives on classic problems in learning theory. The tutorials will share how rethinking problem formulations can yield new approaches for tractable and efficient learning.

Register

Speakers & Tutorial Abstracts

Speaker: Andrej Risteski, Assistant Professor, Machine Learning Department,  Carnegie Mellon University

Title: Towards Understanding the Statistical Landscape of Score-based Losses

Abstract: Score-based losses have emerged as a more computationally appealing alternative to maximum likelihood for fitting (probabilistic) generative models with an intractable likelihood (for example, energy-based models and diffusion models). What is gained by foregoing maximum likelihood is a tractable gradient-based training algorithm. What is lost is less clear: in particular, since maximum likelihood is asymptotically optimal in terms of statistical efficiency, how suboptimal are score-based losses? 

I will survey a recent connection relating the statistical efficiency of broad families of generalized score losses, to the algorithmic efficiency of a natural inference-time algorithm: namely, the mixing time of a suitable diffusion using the score that can be used to draw samples from the model. This “dictionary” allows us to elucidate the design space for score losses with good statistical behavior, by “translating” techniques for speeding up Markov chain convergence (e.g., preconditioning and lifting). I will also touch upon a parallel story for learning discrete probability distributions, in which the “analogue” of score-based losses is played by masked-prediction-like losses. Finally, time-permitting, I will speculate on co-designing pre-training and inference time procedures in foundation models in light of recent interest in inference-time algorithms.  

Speaker: Ankur Moitra, Norbert Wiener Professor of Mathematics, MIT

Title: Vignettes in Learning Theory

Abstract: In this tutorial I will revisit two classic learning problems in a new light:

(1) Learning sequence models. Can we hope for algorithms that work in greater generality when we are given access to a conditional sampling oracle?

(2) Learning graphical models. Is learning from trajectories of the Glauber dynamics actually computationally easier than learning from iid samples?

Both are examples of what I hope is a more general theme, that new and arguably more natural and modern problem formulations can help us overcome intransigent barriers.

Event Details & Agenda

Friday, May 2, 2025 (9:30 AM – 3:00 PM ET)

Location: School of Social Work Room – C03
Address: 1255 Amsterdam Ave, New York, NY 10027

9:00 AM: Guests are welcome to arrive early for check-In and coffee

9:30 AM – 11:30 AM: Tutorial 1

  • Andrej Risteski, Carnegie Mellon University
  • Title: Towards Understanding the Statistical Landscape of Score-based Losses

11:30 AM – 1:00 PM: Lunch & Networking Break (1 hour)

1:00 PM – 3:00 PM: Tutorial 2

  • Ankur Moitra, MIT
  • Title: Vignettes in Learning Theory

3:00 PM: End

REGISTRATION DEADLINE: The Columbia Morningside campus is open to the Columbia community. If you do not have an active CUID, the deadline to register is at 12:00 PM the day before the event.