Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan
Title: Implicit Regularization in Matrix Factorization
Abstract: This talk will explore ideas on "implicit regularization" in under-determined problems where the optimization objective has multiple global minima. We specifically study optimization of a quadratic loss over a matrix with gradient descent on the factorized space. We conjecture and provide empirical and theoretical evidence that with small enough step sizes and initialization close enough to the origin, gradient descent on a full dimensional factorization converges to the minimum nuclear norm solution.
Public events of RIKEN Center for Advanced Intelligence Project (AIP)Join community