CNA colloquium: Joan Bruna (NYU)

Small steps towards a theory of deep feature learning Abstract: A defining characteristic of modern AI systems is their ability to learn useful representations out of complex high-dimensional data. While an idealised picture using low-index functions has emerged over the past years, it focuses on shallow neural networks, leaving the role of depth as an unexplained challenge. In this talk, we will address this gap by first focusing on multi-scale single-index models, originally introduced by Oymak and Soltanolkotabi in the context of tensor decompositions. We will argue that these models display (i) necessity of depth, in the sense that they probably require deep architectures, and (ii) sufficiency of gradient-dynamics, in the sense that they can be learnt by back-propagation methods. Finally, we will relate the depth question with transport-based generative models, discussing (sub)-optimality of score-based diffusion. The event will be available both by Zoom and in Wean Hall 7218.