Talk by Yanai Elazar (The University of Washington and the Allen Institute for AI in Seattle) "On “Emergent Abilities” and Simple Training Data Statistics"

Speaker: Yanai Elazar
(The University of Washington and the Allen Institute for AI in Seattle)

Time: April 21, 2025, 13:00–14:00 (JST)

Format: AIP Open Space & Zoom

Title: On “Emergent Abilities” and Simple Training Data Statistics

Abstract:
I will present two distinct types of “emergent” abilities: (1) the formation of linear structures within internal hidden representations, and (2) the ability of text-to-image models to imitate specific concepts—for example, generating images in a particular art style. I will then show that simple frequency counts from a model’s training data can account for much of the variance in these abilities.
Finally, I will discuss how measuring such behaviors can help reveal information about a model’s training data, providing much-needed transparency into state-of-the-art generative models.