| KEYWORD |
Scaling-up Data Collection for Multi-Fingered Robot Learning
Thesis in external company
keywords ARTIFICIAL INTELLIGENCE, DEEP LEARNING, COMPUTER V, BEHAVIOR CLONING, MULTI-FINGERED MANIPULATION, ROBOT LEARNING, ROBOTICS
Reference persons RAFFAELLO CAMORIANO
External reference persons Federico Ceola, Ph. D. (Istituto Italiano di Tecnologia)
Prof. Lorenzo Natale (Istituto Italiano di Tecnologia)
Research Groups DAUIN - GR-23 - VANDAL - Visual and Multimodal Applied Learning Lab, Humanoid Sensing and Perception @ Istituto Italiano di Tecnologia
Thesis type RESEARCH / EXPERIMENTAL
Description The availability of diverse and heterogeneous datasets has been the key to achieve the latest success in Natural Language Processing (NLP) and Computer Vision (CV).
Collecting datasets that demonstrate robots performing real manipulation tasks to train the equivalent for robotics of a Computer Vision model pre-trained on ImageNet has been a long-standing open problem due to the complexity of collecting real robotic data. The problem has recently been tackled by the Open X-Embodiment project. This project led to the collection of a dataset comprising a large number of smaller datasets showing robots performing a wide number of manipulation tasks. However, while being the largest open dataset for robotic manipulation tasks, datasets in the Open X-Embodiment collaboration generally consider two-fingered grippers and limited sensor multi-modality.
Moreover, collection of training data for robotics is usually performed via tele-operation, which is extremely time consuming and is difficult to deploy in everyday environments, since it would require moving the robot in such settings for data collection.
Universal Manipulation Interface (UMI) overcomes these problems presenting a framework that allows to collect data in-the-wild from human demonstrations collected with a hand-held two-fingered gripper, and zero-shot transfer of policies trained with such data on the robot.
This thesis focuses on the deployment of a new framework for multi-fingered data collection in everyday environments and robot policy learning on such data. The thesis work will investigate methodologies to collect such demonstrations with a hand-held LEAP anthropomorphic hand and deploy the learned policies on a Franka Panda manipulator. The thesis aims at enabling the robot to autonomously perform challenging manipulation tasks, such as sorting objects from a clutter of similar items (e.g. objects in a supermarket basket), or long-horizon tasks as meal preparation.
See attachment for details and required skills.
See also thesis ideas polito-iit - a.a. 2024-25.pdf
Required skills The student is required to carry out the thesis in presence at the IIT Center for Robotics and Intelligent Systems located in Genoa.
Prior knowledge and coursework on Machine Learning and Computer Vision are required.
Good programming skills are required (Python). Knowledge of robotics is a plus.
Expected activities breakdown:
● Literature and study – 20%
● Implementation – 40%
● Experiments – 40%
Notes If interested, you can apply via the following form: https://forms.gle/83hfp8uAQQcTpssVA
Contact raffaello.camoriano@polito.it if you have questions.
Deadline 22/11/2025
PROPONI LA TUA CANDIDATURA