In an era where machine learning drives everything from recommendation engines to scientific discovery, a deeper question shapes the field from behind the scenes: When is learning even possible? For Moïse Blanchard, a Postdoctoral Fellow at Columbia University’s Data Science Institute, this question isn’t philosophical – it’s mathematical. Working under the guidance of Vineet Goyal, Associate Professor of Industrial Engineering and Operations Research and Samory Kpotufe, Associate Professor of Statistics, Blanchard is exploring key questions in statistical learning theory, seeking to understand the fundamental conditions under which algorithms can reliably learn from data.
“Learning can be defined as extracting information from previous data for decision-making,” Blanchard explains. “My work examines the statistical and physical limitations on what is learnable. Once we understand if learning is possible in a specific scenario, the goal is to create an algorithm that optimally learns in that context.”
“Statistical learning theory provides an elegant and systematic way of understanding fundamentals about how machine learning works, which I am very drawn to,” Blanchard says. At the heart of statistical learning theory is the concept of generalization – the idea that an algorithm trained on a sample of data can make accurate predictions on new, unseen examples. Blanchard studies the parameters of successful generalization: under what conditions does a learning algorithm perform well? Under what conditions will it not be able to learn – no matter how much data is available? What resources – time, data and computational power – are necessary to help algorithms achieve success? His work draws on tools from optimization, probability, and computational complexity to address these questions.
Much of Blanchard’s research centers on sequential and online learning, where algorithms must make decisions over time, often under uncertainty, and adapt as they go. This is the foundation behind systems like recommendation engines, online decision-making tools, and even aspects of robotics and autonomous systems. In this context, Blanchard’s research focuses on theoretical guarantees: not only whether learning is possible in a given setting, but also how well it can be done, how much data is required, and what computational trade-offs are involved. A major goal is to create a universal or meta “optimistic” algorithm – a single learning rule that assumes a learnable scenario and performs well. Blanchard explored this idea in a recent paper, presented at the Conference on Learning Theory (COLT), proposing an “optimistically universal” method that can automatically adapt to a wide range of learning problems, guaranteeing success whenever learning is theoretically possible.
An additional layer to Blanchard’s work incorporates the constraint of “memory” into the optimization algorithms. For example, an algorithm might be able to compute quickly but struggle with limited memory – a common issue in real-world systems working with enormous data sets. In a paper presented at the 2024 IEEE Symposium on Foundations of Computer Science (FOCS), Blanchard examines how algorithms can be designed to make the best possible trade-offs between computational efficiency and memory usage. The paper helps explain why popular algorithms like gradient descent can be theoretically optimal – mathematically proven to perform as well as possible – even in resource-constrained environments, and lays the foundation for designing other memory-efficient methods.
Blanchard’s work is grounded in real-world challenges and influenced by the vibrant, interdisciplinary environment of Columbia. “I am grateful for the collaborative culture at the Institute” Blanchard says. “In fact, interactions at workshops and seminars have led to interdisciplinary research – like a new project on game theory. You certainly don’t get bored here,” he concludes with a smile.