DSI Alumni Download: Rahul Agarwal, Senior Data Scientist at IBM

July 28, 2022

Rahul Agarwal completed his undergraduate degree in information and communication technology at Manipal Institute of Technology in India and worked as an applications developer for Oracle before coming to Columbia University for the M.S. in Data Science program.

Here, the 2022 alumnus shares a few gems from his experience at Columbia, including his internships, capstone project, and his exciting new endeavors.

Why did you choose to come to Columbia for your M.S. in Data Science?

The main reason I chose Columbia was the curriculum. I was not sure whether or not I wanted to go deeper into the technical details of data science or develop myself more on the business and client side of things. The curriculum had topics from both areas. There was also space for electives that helped me explore and dive deeper into the topics I wanted to learn more about. Another reason Columbia was my top priority was its location in New York City. The opportunities the city has to offer are unlimited in my opinion.

What was your favorite data science course at Columbia?

One of my favorites was Machine Learning for Data Science taught by John Paisley. Professor Paisley would explain the intuition behind the different algorithms and then explain the math so we can understand the execution of it. That was very insightful. The assignments required us to perform and implement the algorithms from scratch. This helped me better understand the inner workings of the modules with machine learning algorithms.

Tell us about your capstone project with Ralph Lauren.

Shaurya Malik, Keertan Krishnan, Rahul Subramaniam, and I designed and developed Market Basket Analysis Framework. We detected customer patterns and behavior changes in 9GB transactional data stored as a stream over 12 months for 30 stores, split into five test-control groups using AWS SageMaker. We presented an analysis report and data-driven recommendations with actionable insights to the business and management team.

Did you complete an internship during your Columbia studies?

I interned at Goose Hollow Capital Management. Trying to develop trade signals to buy and sell currencies using historical data was extremely satisfying. I learned about the financial markets, how a hedge fund leverages data, what are the key indicators to look at in an industrial setting, and when a data-based solution will or will not work. Answers to these questions helped me understand the power of data. Interacting with my colleagues showed me the reality of why many companies are not able to leverage it to its full potential.

I also interned at Orenda, which is a startup aimed at leveraging data science to optimize energy storage solutions. My experience at Orenda was a short one, limited to building and designing visualizations to interpret and identify the inconsistencies in the electrical grid. I learned a lot about the utility industry and how stakeholders look at a data solution. I also learned how to create the key performance indicators that are able to highlight the data solution.

How did the COVID-19 pandemic impact your experience?

I intended to start classes in the Fall 2020, but I had to delay my studies by one semester due to visa issues. In Spring 2021, all classes were virtual except one hybrid course from the math department. The coursework and the assignments were not affected as much and the virtual experience was good, although I do believe that some classes would have been better in person. My experience was not as bad mainly due to the friends I made through the student council.

Ultimately, what did you gain during the M.S. in Data Science program?

I definitely learned a ton through my coursework and my projects. I can confidently address and solve any typical data science problem today from start to end. Another important thing I learned is how to manage time and prioritize activities. With so many courses, project work, internship and job search, interviews, social events, tech events, webinars, assignments, etc., it became essential to prioritize and manage the productive hours efficiently.

What are you currently working on and willing to share with us?

I am working on a reinforcement learning-based text summarizer. The current version outperforms the state-of-the-art REINFORCE, and we intend to take it further before finalizing. I am also working on an algorithmic trading company. It is currently incorporated in India and the platform supports work on the Indian market, but the algorithms are versatile for any trading script.

This interview has been edited and condensed for clarity.