Reinforcement Learning from Scarce Data

This seminar is jointly held with Research Center for Statistical Machine Learning of the Institute of Statistical Mathematics.

Speaker
Dr. Marc Deisenroth (University College London, UK)
https://deisenroth.cc/

Title:
Reinforcement Learning from Very Sparse Data

Abstract:
In many practical applications of machine learning, we face the challenge of data-efficient learning, i.e., learning from sparse data. This includes healthcare, climate science, and autonomous robots. There are many approaches toward learning from sparse data. In this talk, I will discuss a few of them in the context of reinforcement learning. First, I will motivate probabilistic, model-based approaches to reinforcement learning, which allow us to reduce the effect of model errors. Second, I will discuss a meta-learning approach that allows us to generalize knowledge across tasks to enable few-shot learning. Finally, we can also incorporate structural prior knowledge to speed up learning. In this final case, we can exploit Lie group structures to learn predictive models from high-dimensional observations with nearly no data.

Key references:
Marc P. Deisenroth, Dieter Fox, Carl E. Rasmussen, Gaussian Processes for Data-Efficient Learning in Robotics and Control, IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 37, pp. 408–423, 2015
Steindór Sæmundsson, Katja Hofmann, Marc P. Deisenroth, Meta Reinforcement Learning with Latent Variable Gaussian Processes, Proceedings of the International the Conference on Uncertainty in Artificial Intelligence, 2018
Steindór Sæmundsson, Alexander Terenin, Katja Hofmann, Marc P. Deisenroth, Variational Integrator Networks for Physically Meaningful Embeddings, arXiv:1910.09349

Bio:
Marc Deisenroth is the DeepMind Chair in Artificial Intelligence at University College London. He also holds a visiting faculty position at the University of Johannesburg. From 2014 to 2019, Marc was a faculty member in the Department of Computing, Imperial College London. Since September 2016, Marc has also been an advisor to PROWLER.io, a Cambridge-based startup. Marc’s research interests center around data-efficient machine learning, probabilistic modeling and autonomous decision making. Marc was Program Chair of EWRL 2012, Workshops Chair of RSS 2013 and received Best Paper Awards at ICRA 2014 and ICCAS 2016. In 2019, Marc co-organized the Machine Learning Summer School in London with Arthur Gretton. In 2018, Marc has been awarded The President’s Award for Outstanding Early Career Researcher at Imperial College. He is a recipient of a Google Faculty Research Award and a Microsoft PhD Grant. In 2018, Marc spent four months at the African Institute for Mathematical Sciences (Rwanda), where he taught a course on Foundations of Machine Learning as part of the African Masters in Machine Intelligence. He is co-author of the book Mathematics for Machine Learning (Cambridge University Press, 2020).