Interpretable multimodal deep learning with application to brain imaging and genomics data fusion

Speaker: Prof. Yu-Ping Wang, PhD
Department of Biomedical Engineering, Biostatistics and Data Sciences, Computer Science and Neurosciences
Tulane University

Abstract:
Deep network-based data fusion models have been developed to integrate complementary information from multi-modal datasets while capture their complex relationships. This is particularly useful in biomedical domain, where multi-modal data such as imaging and multi-omics are ubiquitous and the integration of these heterogenous data can lead to novel biological findings. However, deep learning models are often difficult to interpret, bringing about challenges for uncovering biological mechanisms using these models. In this work, we develop an interpretable multimodal deep learning-based fusion model to perform automated disease diagnosis and result interpretation simultaneously. We name it Grad-CAM guided convolutional collaborative learning (gCAM-CCL), which is achieved by combining intermediate feature maps with gradient-based weights in a multi-modal convolution network. The gCAM-CCL model can generate interpretable activation maps to quantify pixel-level contributions of the input fMRI imaging features. Moreover, the estimated activation maps are class-specific, which can therefore facilitate the identification of imaging biomarkers underlying different populations such as age, gender and cognitive groups. Finally, we apply and validate the gCAM-CCL model in the study of brain development with integrative analysis of multi-modal brain imaging and genomics data. We demonstrate its successful application to both the classification of cognitive function groups and the discovery of underlying genetic mechanisms.

Biography: Dr. Yu-Ping Wang received the BS degree in applied mathematics from Tianjin University, China, in 1990, and the MS degree in computational mathematics and the PhD degree in communications and electronic systems from Xi’an Jiaotong University, China, in 1993 and 1996, respectively. After his graduation, he had visiting positions at the Center for Wavelets, Approximation and Information Processing of the National University of Singapore and Washington University Medical School in St. Louis. From 2000 to 2003, he worked as a senior research engineer at Perceptive Scientific Instruments, Inc., and then Advanced Digital Imaging Research, LLC, Houston, Texas. In the fall of 2003, he returned to academia as an assistant professor of computer science and electrical engineering at the University of Missouri-Kansas City. He is currently a Professor of Biomedical Engineering, Computer Sciences, Neurosciences, and Biostatistics & Data Sciences at Tulane University. Dr. Wang’s recent effort has been bridging the gap between biomedical imaging and genomics, where has over 300 peer reviewed publications and received a “Convergence Award” given by Tulane University in 2022. Dr. Wang is a fellow of AIMBE and has served for numerous program committees and NSF and NIH review panels. He is currently an associate editor for J. Neuroscience Methods, IEEE/ACM Trans. Computational Biology and Bioinformatics (TCBB) and IEEE Trans. Medical Imaging (TMI). More about his research can be found at his website: http://www.tulane.edu/~wyp/

This seminar is hosted by Machine Intelligence for Medical Engineering Team.