- Basic Knowledge of the main components of robotics applications, from sensing, control and actuation, to high level action decision and planning; - Deep knowledge on how to formalize an autonomous learning problem in a reinforcement learning context, from the mathematical foundations of the optimization objectives, to the implementation of specific usecases; - Knowledge of the main characteristics of modern reinforcement learning techniques applied to autonomous robots and beyond; - Knowledge of the major challenges of vision-based supervised robot learning; - Basic knowledge on how to deal with research questions, how to critically read a research paper, how to organize a project and prepare a scientific conference report.

- Linear Algebra - Probability theory concepts - Python: Basic elements - C++ basics

The course is organized in three main modules, plus one additional module for labs. Mod 1 -- Robotics survival kit (approx. 15h) - Foundations of sensing & actuation - Briefly on kinematics and dynamics of autonomous systems - motor control - introduction to ROS robot programming - Gaussian Filters: Kalman filter, Extended Kalman filter - Non-Parametric Filters: Particle filter Mod 2 -- Reinforcement learning (approx. 30h) - Sup/unsup Learning 4 robotics: Imitation Learning vs Reinforcement Learning - Reinforcement Learning Value-based 1 - Reinforcement Learning Value-based 2 - Reinforcement Learning Policy Gradients - Reinforcement Learning Model-based RL - Sim 2 Real Mod 3 -- Robot Vision (approx. 15h) - Recap Deep learning - Vision-based Robot localization - Imitation from images/video - Vision-based manipulation Labs. (15 h) - Robot optimal control - Reward Shaping - Reinforcement Learning - Sim-to-sim policy transfer The course will also host a small number of thematic lectures, where students and teachers will discuss research paper related to the topic of the week.

60 h class lectures + 15 h hands-on labs with group work (tutoring hours, not mandatory).

Slides and supplementary material provided by the teachers. Textbooks: - Robotics: Modelling, Planning and Control, Bruno Siciliano, Lorenzo Sciavicco, Luigi Villani, Giuseppe Oriolo - Probabilistic Robotics, Sebastian Thrun, Wolfram Burgard, Dieter Fox. - Reinforcement Learning: An Introduction, Richard S. Sutton, Andrew G. Barto Extra readings: - AI at the Edge: Solving Real-World Problems with Embedded Machine Learning, Daniel Situnayake, Jenny Plunkett - Planning Algorithms, Steven LaValle - Springer Handbook of Robotics, Bruno Siciliano, Oussama Khatib - Applied NonLinear Control, Jean-Jacques Slotine, Weiping Li Useful links: https://docs.ufpr.br/~danielsantos/ProbabilisticRobotics.pdf http://www.probabilistic-robotics.org/ http://people.disim.univaq.it/~costanzo.manes/EDU_stuff/Robotics_Modelling,%20Planning%20and%20Control_Sciavicco_extract.pdf https://web.stanford.edu/class/psych209/Readings/SuttonBartoIPRLBook2ndEd.pdf https://bair.berkeley.edu/blog/2019/12/12/mbpo/ http://www.ioe.nchu.edu.tw/Pic/CourseItem/4497_APPLIED%20NONLINEAR%20CONTROL_slotine_Part1.pdf

Lecture slides; Text book; Exercises; Lab exercises; Video lectures (current year);

You can take this exam before attending the course

Exam: Compulsory oral exam;

- Final project. This will consists in reproducing and extending the methods developed during in-class Labs. The students will elaborate a pdf report of about 6/8 pages using a template provided by the evaluators, and will present the work in a 15 minutes talk during the oral exam. - Oral exam on project presentation and on the theory topics presented during lectures. - The final mark will be obtained considering: i) the verification of completion and submission of bi-weekly labs (up to 3 points); ii) the report on the project extension (up to 12 points); iii) the oral presentation with Q/A (up to 15 points); iv) the active participation in thematic lectures will be considered as a plus (+3 points).

In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.