Mathematical Science Team Seminar(Talk by Eren Mehmet Kiral, Keio University).

This is an online seminar. Registration is required.
【Team】Mathematical Science Team
【Date】2025/January/21(Tue) 17:00-18:00(JST)
【Speaker】Eren Mehmet Kiral, Keio University

Title: Lie groups in learning distributions

Abstract:Instead of learning a single set of model parameters, learning a distribution over parameters is more robust and better captures the variation in the data distribution. But also, distributions (as opposed to points) provide rich mathematical structures, from whose perspective novel methods are naturally revealed.
We start with a (ubiquitous) setting where a Lie group acts on model parameters. For expected loss over a distribution, or the KL-loss, this action helps to efficiently calculate the best direction of perturbation for the distribution. Lie group Bayesian Learning Rule uses the structure provided by Lie groups to learn distributions over the parameters. I will explain realizations of this update method in various groups, and it's relation to methods such as the Bayesian Learning Rule by Khan&Rue which is defined on exponential families of distributions, as well as certain observed phenomena such as emergent weight sparsity, or potential use in low-precision floating point calculations.