Talk by Xuedong Shang (Université de Lille)

Title: Bayesian Best-Arm Identification and Beyond.

Abstract:
We consider the following sequential decision-making problem: A learner sequentially chooses options among a given set to evaluate and receives noisy feedback about their qualities in order to find the best one. Imagine that each evaluation is costly, hence a natural goal is to confidently identify the best option with as little evaluations as possible. Such scenarios, often known as (fixed-confidence) best-arm identification (BAI) in a stochastic multi-armed bandit game, serve as an fundamental abstraction for many real-world applications like A/B/C testing, clinical trials, hyper-parameter optimization for machine learning/deep learning algorithms, etc. In this presentation, I will talk about some new insights on BAI from a Bayesian perspective. In particular, we theoretically justify the use of Bayesian-flavoured algorithms for fixed-confidence BAI and propose a variant which disposes of the computational burden of existing Bayesian BAI algorithms. Then I would like to highlight the use of Bayesian BAI algorithms in guiding the design of a dynamic hyper-parameter optimization algorithm. As for future perspectives, I would like to include some discussions about how Bayesian algorithms would perform in a more general case where side information is taken into account (a.k.a. BAI for linear bandits).