Conference abstracts

Session C1 - Computational Harmonic Analysis and Compressive Sensing

July 18, 16:00 ~ 16:25 - Room B3

What Can Deep Learning Learn from Compressive Sensing?

Benjamin Recht

University of California, Berkeley, USA   -   brecht@berkeley.edu

Recent successes in neural networks have demonstrated that models with an excessive numbers parameters can achieve tremendous improvements in pattern recognition. Moreover, empirical evidence demonstrates that such performance is achievable without any obvious forms of regularization or capacity control. These findings at first glance suggest that traditional learning theory fails to explain why large neural networks generalize. In this talk, I will discuss possible mechanisms to explain generalization in such large models, appealing to insights from linear predictors. I will discuss how many of the observations can be understood by direct analogies to the linear case. I will close by proposing some possible directions of future research that connect a decade of work in compressive sensing with the contemporary phenomenology in large-scale machine learning.

Joint work with Samy Bengio (Google Brain), Moritz Hardt (Google Brain/University of California Berkeley), Oriol Vinyals (DeepMind), Chiyuan Zhang (Massachusetts Institute of Technology).

View abstract PDF



FoCM 2017, based on a nodethirtythree design.