Summer Seminar Speaker

Talk Title: Understanding training of depth 2 nets under minimal distributional assumptions and data poisoning attacks

Abstract:

The question of provable training of neural nets is mathematically extremely challenging and vastly open. Over the last year, we have investigated certain special cases at depth 2 and have given provable guarantees in regimes hitherto unexplored. Our results probe the particularly challenging trifecta of having finitely large nets, while not tying the data to any specific distribution and while having an adversarial attack.

We will try to cover 3 of our results in this talk : (a) analysis of a simple stochastic algorithm which near-optimally provably trains a single ReLU gate in the realizable setting as well as under a data poisoning attack using only mild distributional assumptions, (b) showing that noise assisted gradient descent on a ReLU gate can be proven to be "diffusive" in nature without having to make any distributional assumptions beyond realizability and lastly (c) we demonstrate a class of iterative non-gradient algorithms ("Neuro-Tron") which can train a class of "singe filter" depth 2 nets under a data poisoning attack on true realizable labels s.t the error in the recovered weights scales inversely with the width above a certain finite threshold. This provably brings to light the advantage of having a large width when defending against an adversarial attack.

This talk will is based on our recently released two part series of works,

• Part 1: https://arxiv.org/abs/2005.01699
• Part 2: https://arxiv.org/abs/2005.04211

Bio:

Anirbit, is a (final) year Ph.D. student in applied mathematics at the Johns Hopkins University advised by Prof. Amitabh Basu. He specializes in deep-learning theory and has been awarded 2 fellowships from JHU for this research, "Walter L.Robb Fellowship" and the inaugural "Mathematical Institute for Data Science Fellowship." Earlier, he was a researcher in Quantum Field Theory, while doing his undergrad in physics at the Chennai Mathematical Institute (CMI) and masters in theoretical physics at the Tata Institute of Fundamental Research (TIFR). For more details, you can visit his website.