Doorkeeper

Team ABI Seminar by Itay Evron, (Technion) on "Catastrophic forgetting in linear regression"

2022-07-14(木)16:00 - 17:30 JST
オンライン リンクは参加者だけに表示されます。
申し込む

申し込み受付は終了しました

今後イベント情報を受け取る

参加費無料

詳細

Speaker: Itay Evron, Technion
Title: How catastrophic can catastrophic forgetting be in linear regression?
Abstract: To better understand catastrophic forgetting, we study fitting an overparameterized linear model to a sequence of tasks with different input distributions. We analyze how much the model forgets the true labels of earlier tasks after training on subsequent tasks, obtaining exact expressions and bounds. We establish connections between continual learning in the linear setting and two other research areas -- alternating projections and the Kaczmarz method. In specific settings, we highlight differences between forgetting and convergence to the offline solution as studied in those areas. Under a cyclic task ordering, we derive a dimensionality-independent universal bound for the worst case forgetting. This stands in contrast to the convergence to the offline solution, which can be arbitrarily slow according to existing alternating projection results. We further show that a better bound can be achieved when tasks are presented in a random ordering.

コミュニティについて

RIKEN AIP Public

RIKEN AIP Public

Public events of RIKEN Center for Advanced Intelligence Project (AIP)

メンバーになる