Talk by Prof. Philippe Esling (IRCAM, Sorbonne université) on June 12 at 4:00 p.m.

Title :
Modeling musical creativity with variational inference and probabilistic generative models

Philippe Esling
ACIDS team at IRCAM, Sorbonne université (Paris, France)

Abstract :
The research project carried by the ACIDS team at IRCAM seeks to model musical creativity by extending variational learning approaches towards the use of multivariate and multimodal time series. Our major object of study lies in the properties and perception of musical orchestration. Orchestration is the subtle art of writing musical pieces for orchestra, by combining the spectral properties of each instrument to achieve a particular sonic goal. In this context, the multivariate analysis of temporal processes is required given the inherent multidimensional nature of instrumental mixtures. Furthermore, time series need to be scrutinized at variable time scales (termed here granularities) as a wealth of time scales co-exist in music (from the identity of single notes up to the structure of entire pieces). Furthermore, orchestration lies at the exact intersection between the symbol (musical writing) and signal (audio recording) representations.

After introducing the general framework and multiple state-of-art creative applications done in the past years, we will focus on various applications of the variational learning framework to disentangle factors of audio variation. Hence, we will detail several recent papers produced by our team, allowing to regularize the topology of the latent space based on perceptual criteria, working with both audio waveforms and spectral transforms and performing timbre style transfer between instruments. Finally, we discuss the development of these approaches as creative tools allowing to increase musical creativity in contemporary music and show case-study of recent pieces played at renowned venues.

We will open the discussion to the question of creative intelligence through the analysis of orchestration and how this could give rise to a whole new category of generic creative learning systems.

Short bio:
Philippe Esling received a B.Sc in mathematics and computer science in 2007, a M.Sc in acoustics and signal processing in 2009 and a PhD on data mining and machine learning in 2012. He was a post-doctoral fellow in the department of Genetics and Evolution at the University of Geneva in 2012. He is now an associate professor with tenure at Ircam laboratory and Sorbonne Université since 2013. In this short time span, he authored and co-authored over 20 peer-reviewed journal papers in prestigious journals such as ACM Computing Surveys, Publications of the National Academy of Science, IEEE TSALP and Nucleic Acids Research. He received a young researcher award for his work in audio querying in 2011, a PhD award for his work in multiobjective time series data mining in 2013 and several best paper awards. In applied research, he developed and released the first computer-aided orchestration software called Orchids, commercialized at fall 2014, which already has a worldwide community of thousands users and led to musical pieces from renowned composers played at international venues. He is the lead investigator of machine learning applied to music generation and orchestration, and directs the recently created Artificial Creative Intelligence and Data Science (ACIDS) team at IRCAM.