Random Subspace Methods for Large-Scale Optimization

This is an online seminar. Registration is required.

【Date】Tuesday, September 30th, 10:00 - 11:00
【Speaker】Pierre-Louis Poirion, AIP Continuous Optimization Team

TITLE:
Random Subspace Methods for Large-Scale Optimization

ABSTRACT:
Modern learning systems optimize models with millions to billions of parameters under tight time and memory budgets. This talk shows how random subspaces make large-scale optimization problems tractable by replacing full-dimensional steps with lower-dimensional computations that still preserve progress toward high-quality solutions. We will first focus on linear problems and the single-shot “sketch-and-solve” setting, where the original problem is compressed once via a randomized embedding and solved in the reduced space, yielding an approximate solution to the full problem. We will then turn to the unconstrained nonconvex setting and develop iterative randomized subspace methods, in which each iteration samples a low-dimensional subspace to cut per-step cost while maintaining a valid descent direction. A key theme will be that randomness can exploit hidden structure, in large optimization problems, to improve runtime without sacrificing solution quality. I will illustrate the theory with numerical examples.