Speaker: Dr. Alessio Del Bue (Istituto Italiano di Tecnologia)
Title: Inferring relationships and answering questions on 3D scene geometry
Abstract: In this talk, we look at understanding the relationships
between objects within a 3D scene. Firstly, we present our latest work
on using multi-view information to construct an object level
representation of a 3D scene. In this formalisation, each object is
defined as a 3D ellipsoid encoding the object position, pose and
occupancy. The overall geometry of the scene is then represented as a
graph where each 3D object is a node allowing relationships 'same-set',
'part-of', 'same-plane' and 'support' to be inferred by message passing
over the graph. We build an architecture that can support such geometric
nodes, object nodes and relational nodes merged using an RNN framework.
Finally, we briefly outline how to answer questions about the layout of
a scene using RGBD data. Using a depth branch guided by region
proposals, inferred from the RGB, we show how encoding the relationships
between regions provides the necessary support to improve answer
prediction. We evaluate over new datasets designed for the VQA depth
problem.
Bio: Alessio Del Bue is a Tenure-Track Researcher leading the Visual
Geometry and Modelling (VGM) Lab of the PAVIS Department at the Istituto
Italiano di Tecnologia (IIT). Previously (2006-2009), he was a senior
researcher at the Institute for Systems and Robotics (ISR) at the
Instituto Superior Técnico (IST) in Lisbon, Portugal. Before that, he
obtained his Ph.D. in 2006 under the supervision of Prof. Lourdes
Agapito in the Department of Computer Science at Queen Mary University
of London. His main area of research is computer vision with a
particular focus on dynamic 3D scene understanding and analysis. He is
also active in several interdisciplinary projects at IIT by
scientifically supporting life science, robotic, cultural heritage
research lines on computer vision and image processing related issues.
Public events of RIKEN Center for Advanced Intelligence Project (AIP)
Join community