Yading Yuan Frames the Discussion
As Yading Yuan, the session moderator who opened the session, put it, “images are everywhere.” In “Making the Most of an Image” presenters looked at some of the ways that data science is revolutionizing a critical means by which we understand and thrive in the world. Their talks covered a range of areas, from new kinds of image capture, to image analysis, ways to add additional information for new insights, even how image analysis techniques can improve other areas of experience, like understanding music.
Yading Yuan himself does powerful work with images. A radiologist by training, he uses machine learning and computational analysis to improve accuracy in assessing medical images of tumors, work that might lead to improved diagnosis, prognosis, and treatment planning for cancer patients.
Shree Nayar: “Future Cameras”
Imaging and vision are at the heart of how we understand the world, both as people and with modern computing systems, and they are changing in important ways. While cameras have long been designed to capture images based on the workings of the human eye, this approach mimicked the limitations of human vision as well as its capabilities. A new approach called computational imaging uses optical coding to record visual information and algorithmic decoding to produce images that are richer in information, immersive, and even interactive. This technology has been commercialized and is today used in the cameras found in popular smartphones. Professor Shree Nayar showed the use of computational image to capture an unprecedented ranges of brightness and color that reveal details in bright skies and dark shadows, and produce billion-pixel images capable of examining the stipples on a dollar bill from across the room. “Computational imaging will break the limits of the traditional camera,” he said, adding, “It will allow us to explore new visual worlds that we haven’t seen before.”
Shree K. Nayar is T. C. Chang Professor in the Department of Computer Science, Columbia Engineering
Arian Maleki: “Image Acquisition Challenges in the Presence of Speckle Noise: Theoretical and Practical Insights”
In traditional pinhole cameras, there exists a direct relation between the desired image and the measurements used to produce it. However, this relationship is often not applicable in imaging systems designed to capture images of objects beyond the capability of our visual system, such as internal organs or atomic structures. While many of these imaging systems have existed for approximately half a century, recent advancements in mathematics, statistics, and artificial intelligence offer new opportunities to significantly enhance the quality, speed, and resolution of such systems.
During the presentation, Arian Maleki elaborated on how the mathematical comprehension of various imaging challenges, coupled with the capabilities of neural networks in image modeling and synthesis, not only suggest innovative approaches to imaging but also enable the reconstruction of images from acquired measurements. The application of these ideas to coherent imaging systems was also discussed during the talk.
Arian Maleki is Associate Professor in the Department of Statistics at Columbia’s Faculty of Arts and Sciences
Hortense Fong: “A Theory-Based Explainable Deep Learning Architecture for Music Emotion”
While visual imaging has a long and sophisticated history in engineering and computer science, the representation and analysis of sound, in particular the emotional experience of music, is relatively less studied. Might some techniques of AI-based image analysis be deployed in ways that make music’s effects on people more analytic and explainable? In her talk, Hortense Fong described how spectrographic representations of music’s frequency, loudness, and time could be analyzed with a CNN classifier to determine the likely impression a piece of music would have on a listener. In particular, the relative consonance and dissonance of sounds, whether there seems to be compatibility or clash in the music, provides structure to help predict and explain whether music elicits positive or negative and soothing or arousing emotion. “By imposing theory onto the design of the model,” she said, “we’ve gotten a model that is not only accurate, but explainable.” That is important to build trust in future systems.
Hortense Fong is Assistant Professor in the Marketing Division of Columbia Business School
Despina Kontos: “Radiomics, Radiogenomics, and AI: The Emerging Role of Imaging Biomarkers in Precision Cancer Care”
Imaging plays a crucial role in cancer care, encompassing screening, prevention, diagnosis, prognosis, and treatment monitoring. In her talk, Professor Despina Kontos, a computer scientist, emphasized how AI-enhanced image analysis in breast cancer can extend beyond traditional assessments of tumor density. She discussed the integration of additional data such as genetics, family history, and the relationships among heterogeneous tumors to provide a more comprehensive understanding of cancer. This approach includes using heat map analysis to gain insights into tumor morphology and behavior, which can significantly improve oncological outcomes.
Professor Kontos highlighted the vast repository of imaging data in medical centers. “Imaging is one of the most untapped sources of data in a medical center,” she said, noting that just at Columbia over 470,000 scans are processed annually, each containing valuable information beyond the initial diagnostic reports. These data, when pooled and analyzed, represent a promising resource for advancing cancer care. The upcoming launch of the Center for Innovation in Imaging Biomarkers and Integrated Diagnostics, an interdisciplinary center launching in fall 2024 under Professor Kontos’ leadership, will offer a vibrant ecosystem to pursue this work and advance the frontiers of cancer care.
Despina Kontos is Professor in the Department of Radiology at Columbia’s Vagelos College of Physicians and Surgeons