At the end of the course the students will: - know and understand the basic principles of statistical machine learning applied to pattern recognition and classification; - know the principal techniques for classification, including generative Gaussian models and discriminative approaches based on logistic regression and support vector machines, among others; - understand the theoretical motivations behind different classification approaches, their main properties and domain of application, and their limitations; - be able to implement the different algorithms using wide-spread programming frameworks (Python); - be able to apply different methods to real tasks, to critically evaluate their effectiveness and to analyze which strategies are better suited to different applications; - be able to transfer the acquired knowledge and capabilities to solve novel classification problems, developing novel methods based on the frameworks that will be discussed during classes

The students should have basic knowledge of probability and statistics, linear algebra, calculus and programming.

Introduction - Machine learning and pattern recognition - Probability theory concepts - Python: language, main numerical libraries - Model taxonomy: generative and discriminative approaches - Model optimization, hyperparameter selection, cross-validation Decision theory and model evaluation - Inference and decisions - Classification scores and log-likelihood ratios - Detection Cost Functions and optimal Bayes decisions Dimensionality reduction - Principal Component Analysis (PCA) - Linear Discriminant Analysis (LDA) Generative models - Generative Gaussian classifiers: univariate Gaussian, multivariate Gaussian (MVG), naive Bayes - Tied covariance MVG and LDA - Categorical and Multinomial classifiers Logistic Regression (LR) - Tied MVG and LR - LR as Maximum Likelihood solution for class labels - Binary and multiclass cross-entropy - LR as empirical risk minimization - Overfitting and regularization - MVG and Quadratic LR Support Vector Machines (SVM) - Maximum margin classifier - Soft margin and L2 regularization - Primal and dual SVM formulation - Non linear extension: brief introduction to kernels Density estimation and latent variable models - Gaussian mixture models (GMM) - The Expectation Maximization (EM) algorithm

The course will include 3 hours of lectures and 1,5 hours of laboratory per week. The lectures will focus both on theoretical and practical aspects, and will include open discussions aimed at developing suitable solutions for different problems. The laboratories will allow the students to implement most of the techniques that will be presented during the lectures, and to apply the learned methods to real data.

[1] Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg. [2] Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press. Additional material, including slides and code fragments, will be made available on the course website.

Lecture slides; Text book; Lab exercises; Lab exercises with solutions;

Exam: Written test; Individual project; Group project;

The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems. The exam will consist in two parts: - A project to be developed during the course. The students will be able to choose individual or group projects among a set of possible choices, and will have to deliver a final report on the project. - A written examination, which will include a theory section (max. 24 points) and an assessment of the project, consisting of open and / or closed questions on the project and on the implementation of the applied methods (approximately 150 minutes, no material allowed). The project mark will be based on the report and the written assesment (max. 8 points) The final mark will be the sum of the project and theory written exam marks. The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suitable models based on the topics presented during lectures. Each candidate will have to provide a report detailing the employed methodology and a critical analysis of the obtained results. The report and the project part of the written exam will assess: - The degree of understanding of the theoretical principles of statistical machine learning for pattern recognition - The ability of the student to analyze a specific problem, assessing which approaches, among those that have been presented, are more suited to solve the task - The ability of the student to implement, apply and possibly extend the studied methods to devise suitable classifiers for the specific case study - The ability of the student to critically evaluate the effectiveness of the proposed approaches. The theory part of the written examination will consists of open and / or closed questions covering the topics presented during the lectures, and will assess: - The theoretical understanding of the basic principles of statistical machine learning for pattern recognition - The knowledge and understanding of the different approaches that have been presented during the lectures - The ability of the student to critically analyze and evaluate the different approaches.

In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.