| KEYWORD |
Humanoid Sensing and Perception @ Istituto Italiano di Tecnologia
Scaling-up Data Collection for Multi-Fingered Robot Learning
Tesi esterna in azienda
Parole chiave ARTIFICIAL INTELLIGENCE, DEEP LEARNING, COMPUTER V, BEHAVIOR CLONING, MULTI-FINGERED MANIPULATION, ROBOT LEARNING, ROBOTICS
Riferimenti RAFFAELLO CAMORIANO
Riferimenti esterni Federico Ceola, Ph. D. (Istituto Italiano di Tecnologia)
Prof. Lorenzo Natale (Istituto Italiano di Tecnologia)
Gruppi di ricerca DAUIN - GR-23 - VANDAL - Visual and Multimodal Applied Learning Lab, Humanoid Sensing and Perception @ Istituto Italiano di Tecnologia
Tipo tesi RESEARCH / EXPERIMENTAL
Descrizione The availability of diverse and heterogeneous datasets has been the key to achieve the latest success in Natural Language Processing (NLP) and Computer Vision (CV).
Collecting datasets that demonstrate robots performing real manipulation tasks to train the equivalent for robotics of a Computer Vision model pre-trained on ImageNet has been a long-standing open problem due to the complexity of collecting real robotic data. The problem has recently been tackled by the Open X-Embodiment project. This project led to the collection of a dataset comprising a large number of smaller datasets showing robots performing a wide number of manipulation tasks. However, while being the largest open dataset for robotic manipulation tasks, datasets in the Open X-Embodiment collaboration generally consider two-fingered grippers and limited sensor multi-modality.
Moreover, collection of training data for robotics is usually performed via tele-operation, which is extremely time consuming and is difficult to deploy in everyday environments, since it would require moving the robot in such settings for data collection.
Universal Manipulation Interface (UMI) overcomes these problems presenting a framework that allows to collect data in-the-wild from human demonstrations collected with a hand-held two-fingered gripper, and zero-shot transfer of policies trained with such data on the robot.
This thesis focuses on the deployment of a new framework for multi-fingered data collection in everyday environments and robot policy learning on such data. The thesis work will investigate methodologies to collect such demonstrations with a hand-held LEAP anthropomorphic hand and deploy the learned policies on a Franka Panda manipulator. The thesis aims at enabling the robot to autonomously perform challenging manipulation tasks, such as sorting objects from a clutter of similar items (e.g. objects in a supermarket basket), or long-horizon tasks as meal preparation.
See attachment for details and required skills.
Vedi anche thesis ideas polito-iit - a.a. 2024-25.pdf
Conoscenze richieste The student is required to carry out the thesis in presence at the IIT Center for Robotics and Intelligent Systems located in Genoa.
Prior knowledge and coursework on Machine Learning and Computer Vision are required.
Good programming skills are required (Python). Knowledge of robotics is a plus.
Expected activities breakdown:
● Literature and study – 20%
● Implementation – 40%
● Experiments – 40%
Note If interested, you can apply via the following form: https://forms.gle/83hfp8uAQQcTpssVA
Contact raffaello.camoriano@polito.it if you have questions.
Scadenza validita proposta 22/11/2025
PROPONI LA TUA CANDIDATURA