While data science research has opened many promising avenues to help humanity address and adapt to climate change, data centers – the structures that house servers  – are energy-hungry themselves, consuming approximately 2 percent of U.S. energy supplies according to the Department of Energy, and their appetite is growing swiftly. 

To stave off the looming energy crisis, longstanding collaborators IBM and Columbia, are partnering on four separate research projects to make data centers more efficient. Tamar Eilam, IBM’s Chief Scientist for Sustainable Computing, who is Principal Investigator on the Columbia-IBM Sustainable Computing Initiative, and Clifford Stein, Interim Director of the Data Science Institute and Wai T. Chang Professor of Industrial Engineering and Operations Research and Professor of Computer Science, discuss the issue and how they have partnered to address it.

Some estimate that by 2030, U.S. data centers’ energy demands could double their 2022 needs, and many experts say AI is driving that growth. Why is that the case?

Clifford Stein: Powerful computing has a great cost in terms of energy consumed. At this point, what’s possible in AI is enormous. The limiting factor is actually energy and access to computational resources.  

Tamar Eilam: The rate of growth in energy consumed is outpacing the rate of energy production.  It’s getting worse due to a combination of data explosion, energy-hungry workloads like AI, and the flattening of Dennard Scaling, which means we are approaching the physical limits of what we can achieve with energy efficiency in general-purpose chip design.

How did IBM and Columbia come together to address this issue?

Tamar Eilam

Tamar Eilam: I had very early conversations with several professors at Columbia, including Professor Stein. We realized that we have very similar interests. There is a perfect match between the interests, skills, and experience that Columbia University brings and IBM’s aims to create a strong agenda to address sustainable computing. It’s necessary for industry and academia to come together to create standards and collaborate on innovation in this space.

Given the complexity of data centers, there are many avenues to address energy consumption. What areas are IBM and Columbia focusing on, and what work has been done before?

Tamar Eilam: We are working to address the electricity consumed as well as the carbon associated with manufacturing–the whole ecosystem. 

Right now, we are pursuing four avenues: 

  1. First is modeling how AI behaves on different hardware with respect to energy consumption, based on differing parameters. Optimization algorithms can then find the best energy configuration for a system. Professor Martha Kim leads this work.
  2. Second is designing lower-power chips, particularly for the new high-demand uses, Professor Luca Carloni is leading this initiative. 
  3. A third, headed by Professor Asaf Cidon, aims to address software bloat to gain the necessary benefits of new memory and storage technology. 
  4. Finally, Professor Stein is leading the group that looks at optimizing the operations and scheduling of components and systems.

Why has this not been addressed before? And where are sustainable computing efforts headed?

Clifford Stein

Clifford Stein: First, there has long been work to make computing more efficient, particularly on the hardware side. Many of the issues we are working to address in the IBM-Columbia collaboration are on the software side – how do you make efficient use of the entire system?

We must continue to improve because the pace of the energy consumption for computing is growing faster than energy production overall. 

What makes IBM and Columbia good partners?

Tamar Eilam: IBM believes that if you want to address sustainable computing, we need to address the entire life cycle, as we are trying to do with the unique areas of expertise Professors Stein, Cidon, Carloni, and Kim bring to the four areas we focus on. It is important for industry and academia to come together to create standards and collaborate on an open source project. 

Ultimately, there is a perfect match between the skills and experience that Columbia University brings and IBM’s aims in sustainable computing.  

Clifford Stein: From the perspective of the Data Science Institute, we find IBM to be a great partner that allows us to apply our methodological expertise. Having access to computing systems and real data is critically  important for our progress. 

The access IBM has given us is extensive, and we have developed a substantial and directed partnership. It’s really been exciting to see the progress we’ve made.

What are your individual goals for this work?

Clifford Stein: As a researcher, I’ve been working on the optimization side and thinking about data center scheduling.

Ideally there would be a system and an algorithm that would be invisible to the user, but would deliver all the performance that they expect while really understanding that energy profile, for significant savings on the energy used.

Something else we are working to address is how, with specialized chips, you have to pay attention to the details of exactly how they perform at different speeds and different amounts of power. We want to develop methodology so that one day you can change a setting in your algorithm to take advantage of new efficiencies. We’re trying to design general principles for management.

Tamar Eilam: I very much have a systems view. I’d like to see us doing the same computation with less energy. Let’s take, for example, AI inference.  To meaningfully understand the efficiency of AI inference jobs, you need to factor in everything that happened that led to the creation of the model that is now serving these inference jobs. That includes the training phase, the fine tuning phase. I call it life cycle efficiency analysis.

How will you measure success?

Tamar Eilam: We want to develop groundbreaking technology that provably leads to the reduction of carbon associated with computing. 

How does this project reflect the values of your institution?

Tamar Eilam: It is exciting that we are in a position to focus the research on real problems, and to leverage IBM’s own AI platform to explore applications that are of great interest to our customers. 

For example, our clients are interested in open and heterogeneous platforms that allow them to use multiple different types of accelerators. They want optimization of cost and energy consumption. They also want to understand their computing environments to enhance efficiency and performance.

Clifford Stein: The Data Science Institute has a tagline of  “Data for Good,” and this project reflects that approach to data science.  Sustainability is a challenge that requires intervention at a scale beyond what most people can do as environmentally conscious individuals. As an individual, there are not a lot of ways you can make a real impact on climate change. Even if you do everything to reduce your personal carbon footprint you still don’t make a dent. But what you can do when you have certain skills, you have the opportunity to figure out what you can do at a larger scale. As a data scientist, we have the ability to see things at a larger scale, have insights, and make recommendations.  We can influence how industry operates. We can work with other faculty to make improvements in their fields – data is in everything now. That’s why I really like doing work like this.