Doorkeeper

Talk by Prof. Michael I. Jordan (University of California, Berkeley, USA)

Fri, 20 Oct 2017 10:30 - 12:00 JST

RIKEN Center for Advanced Intelligence Project (AIP)

Nihonbashi 1-chome Mitsui Building, 15th floor, 1-4-1 Nihonbashi, Chuo-ku, Tokyo 103-0027, Japan

Register

Registration is closed

Get invited to future events

Free admission

Description

Speaker: Prof. Michael I. Jordan (University of California, Berkeley, USA)
https://people.eecs.berkeley.edu/~jordan/

Title:
Variational, Hamiltonian and Symplectic Perspectives on Acceleration

Abstract:
Accelerated gradient methods play a central role in optimization, achieving
optimal rates in many settings. While many generalizations and extensions of
Nesterov's original acceleration method have been proposed, it is not yet
clear what is the natural scope of the acceleration concept. We study
accelerated methods from a continuous-time perspective. We show that there
is a Lagrangian functional that we call the "Bregman Lagrangian" which generates
a large class of accelerated methods in continuous time, including (but not
limited to) accelerated gradient descent, its non-Euclidean extension, and
accelerated higher-order gradient methods. We show that the continuous-time
limit of all of these methods correspond to traveling the same curve in
spacetime at different speeds. We also describe a "Bregman Hamiltonian"
which generates the accelerated dynamics, we develop a symplectic integrator
for this Hamiltonian and we discuss relations between this symplectic integrator
and classical Nesterov acceleration. I also discuss some of the implications
of this geometric perspective for efficient saddle-point avoidance.

About this community

RIKEN AIP Public

RIKEN AIP Public

Public events of RIKEN Center for Advanced Intelligence Project (AIP)

Join community