Master of science-level of the Bologna process in Ingegneria Informatica (Computer Engineering) - Torino Master of science-level of the Bologna process in Mechatronic Engineering (Ingegneria Meccatronica) - Torino
This course will provide foundational knowledge on robot systems and their control, with the purpose of developing robots with an embodied intelligence, able to learn how to perceive and interact with the surrounding world.
Lectures will focus on theoretical and practical knowledge on machine learning solutions for autonomous systems.
This course will provide foundational knowledge on robot systems and their high-level control, with the purpose of developing robots with an embodied intelligence, able to learn how to perceive and interact with the surrounding world.
Lectures will focus on theoretical and practical knowledge on machine learning solutions for autonomous systems, with specific focus on Reinforcement Learning techniques.
- Knowledge of the main components of robotics applications, from sensing, control and actuation, to high level action decision and planning;
- Knowledge on how to formalize an autonomous learning problem, from the mathematical foundations of the optimization objectives, to the implementation of specific usecases;
- Knowledge of the main characteristics of modern reinforcement learning techniques applied to autonomous robots;
- Basic knowledge on how to deal with research questions, how to organize a project and prepare a scientific conference report.
- Basic Knowledge of the main components of robotics applications, from sensing, control and actuation, to high level action decision and planning;
- Deep knowledge on how to formalize an autonomous learning problem in a reinforcement learning context, from the mathematical foundations of the optimization objectives, to the implementation of specific usecases;
- Knowledge of the main characteristics of modern reinforcement learning techniques applied to autonomous robots and beyond;
- Knowledge of the major challenges of vision-based supervised robot learning;
- Basic knowledge on how to deal with research questions, how to critically read a research paper, how to organize a project and prepare a scientific conference report.
- Linear Algebra
- Probability theory concepts
- Basic concepts of decision theory (model optimization)
- Automatic Control
- C++
- Python: Basic elements
- Linear Algebra
- Probability theory concepts
- Python: Basic elements
- C++ basics
The course is organized in three main modules, plus one additional module for labs.
Robotics survival kit (approx. 15 h)
- Foundations of sensing & actuation
- Briefly on kinematics and dynamics of autonomous systems
- motor control
- introduction to ROS robot programming
- Gaussian Filters: Kalman filter, Extended Kalman filter
- Non-Parametric Filters: Particle filter
Robot learning (approx. 25 h)
- Reinforcement Learning Policy Gradients
- Reinforcement Learning Off-policy RL
- Reinforcement Learning Model-based RL
- Sim2Real
- Imitation Learning (optional)
Applications (approx. 5 h)
- Robot manipulation
- Visual Servoing
- Locomotion
- Autonomous vehicles
Labs. (15 h)
- Manipulator control
- State estimate
- RL
- Sim-to-sim policy transfer
- Personal extensions (students’ contribution)
The course is organized in three main modules, plus one additional module for labs.
Mod 1 -- Robotics survival kit (approx. 15h)
- Foundations of sensing & actuation
- Briefly on kinematics and dynamics of autonomous systems
- motor control
- introduction to ROS robot programming
- Gaussian Filters: Kalman filter, Extended Kalman filter
- Non-Parametric Filters: Particle filter
Mod 2 -- Reinforcement learning (approx. 30h)
- Sup/unsup Learning 4 robotics: Imitation Learning vs Reinforcement Learning
- Reinforcement Learning Value-based 1
- Reinforcement Learning Value-based 2
- Reinforcement Learning Policy Gradients
- Reinforcement Learning Model-based RL
- Sim 2 Real
Mod 3 -- Robot Vision (approx. 15h)
- Recap Deep learning
- Vision-based Robot localization
- Imitation from images/video
- Vision-based manipulation
Labs. (15 h)
- Robot optimal control
- Reward Shaping
- Reinforcement Learning
- Sim-to-sim policy transfer
The course will also host a small number of thematic lectures, where students and teachers will discuss research paper related to the topic of the week.
45 h class lectures + 15 h hands-on labs with group work.
60 h class lectures + 15 h hands-on labs with group work (tutoring hours, not mandatory).
Slides and supplementary material provided by the teachers.
Textbooks:
- Robotics: Modelling, Planning and Control, Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, Giuseppe Oriolo
- Probabilistic Robotics, Sebastian Thrun, Wolfram Burgard, Dieter Fox.
- Reinforcement Learning: An Introduction, Richard S. Sutton, Andrew G. Barto
Extra readings:
- AI at the Edge: Solving Real-World Problems with Embedded Machine Learning, Daniel Situnayake, Jenny Plunkett
- Planning Algorithms, Steven LaValle
- Springer Handbook of Robotics, Bruno Siciliano, Oussama Khatib
- Applied NonLinear Control, Jean-Jacques Slotine, Weiping Li
Useful links:
https://docs.ufpr.br/~danielsantos/ProbabilisticRobotics.pdf
http://www.probabilistic-robotics.org/
http://people.disim.univaq.it/~costanzo.manes/EDU_stuff/Robotics_Modelling,%20Planning%20and%20Control_Sciavicco_extract.pdf
https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf
https://bair.berkeley.edu/blog/2019/12/12/mbpo/
http://www.ioe.nchu.edu.tw/Pic/CourseItem/4497_APPLIED%20NONLINEAR%20CONTROL_slotine_Part1.pdf
Slides and supplementary material provided by the teachers.
Textbooks:
- Robotics: Modelling, Planning and Control, Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, Giuseppe Oriolo
- Probabilistic Robotics, Sebastian Thrun, Wolfram Burgard, Dieter Fox.
- Reinforcement Learning: An Introduction, Richard S. Sutton, Andrew G. Barto
Extra readings:
- AI at the Edge: Solving Real-World Problems with Embedded Machine Learning, Daniel Situnayake, Jenny Plunkett
- Planning Algorithms, Steven LaValle
- Springer Handbook of Robotics, Bruno Siciliano, Oussama Khatib
- Applied NonLinear Control, Jean-Jacques Slotine, Weiping Li
Useful links:
https://docs.ufpr.br/~danielsantos/ProbabilisticRobotics.pdf
http://www.probabilistic-robotics.org/
http://people.disim.univaq.it/~costanzo.manes/EDU_stuff/Robotics_Modelling,%20Planning%20and%20Control_Sciavicco_extract.pdf
https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf
https://bair.berkeley.edu/blog/2019/12/12/mbpo/
http://www.ioe.nchu.edu.tw/Pic/CourseItem/4497_APPLIED%20NONLINEAR%20CONTROL_slotine_Part1.pdf
Slides; Libro di testo; Esercizi; Esercitazioni di laboratorio; Video lezioni dell’anno corrente;
Lecture slides; Text book; Exercises; Lab exercises; Video lectures (current year);
E' possibile sostenere l’esame in anticipo rispetto all’acquisizione della frequenza
You can take this exam before attending the course
Modalità di esame: Prova orale obbligatoria;
Exam: Compulsory oral exam;
...
- final group (up to 3 students) project. This will consists in reproducing and extending the methods developed during in-class Labs. The students will elaborate a pdf report of about 8 pages in IEEE paper format and will present the work in a 15 minutes talk during the oral exam.
- oral exam on project presentation (group) and on the theory topics presented during lectures (individual questions).
Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.
Exam: Compulsory oral exam;
- Final project. This will consists in reproducing and extending the methods developed during in-class Labs. The students will elaborate a pdf report of about 6/8 pages using a template provided by the evaluators, and will present the work in a 15 minutes talk during the oral exam.
- Oral exam on project presentation and on the theory topics presented during lectures.
- The final mark will be obtained considering: i) the verification of completion and submission of bi-weekly labs (up to 3 points); ii) the report on the project extension (up to 12 points); iii) the oral presentation with Q/A (up to 15 points); iv) the active participation in thematic lectures will be considered as a plus (+3 points).
In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.