Tensor Learning Team Seminar (Talk by Grigorios Chrysos, University of Wisconsin-Madison, USA).

This is an online seminar. Registration is required.
【Tensor Learning Team】
【Date】2024/June/18(Tuesday) 15:00-16:00 (JST)
*【Speaker】Grigorios Chrysos, University of Wisconsin-Madison, USA *

Title:  Are activation functions required for learning in all deep networks?

Abstract: Activation functions play a pivotal role in deep neural networks, enabling them to tackle complex tasks like image recognition. However, activation functions also introduce significant challenges for deep learning theory, network dynamics analysis, and properties such as interpretability and privacy preservation. In this talk, we revisit the necessity of activation functions across various scenarios. Specifically, we explore expressing network outputs through high-order interactions among input elements using multilinear algebra. This approach allows us to attain the necessary expressivity via these high-order interactions. Yet, the question remains: Is this expressivity alone sufficient for effective learning? Our recent research, presented at ICLR’24, unveils networks that achieve strong performance even in demanding tasks, such as ImageNet image recognition.

Bio:
Grigorios Chrysos is an Assistant Professor in the University of Wisconsin-Madison. Before that, Grigorios was a postdoctoral fellow at EPFL following the completion of his PhD at Imperial College London. Previously, he graduated from National Technical University of Athens with a Diploma/MEng in Electrical and Computer Engineering. He has co-organized workshops in top conferences (CVPR, ICCV). He also organized tutorials on polynomial nets (CVPR'22, AAAI'23) and deep learning theory (CVPR'23, ISIT'24). His research interests lie in generative models, tensor decompositions and modelling high dimensional distributions. Grigorios has been recognized as an outstanding reviewer in top-tier conferences (ICML'21, ICLR'22, ICML'22, NeurIPS'22, NeurIPS'23) and is currently an Action Editor for TMLR and an Area Chair for NeurIPS.