01URTOV

A.A. 2020/21

Course Language

Inglese

Course degree

Master of science-level of the Bologna process in Ingegneria Informatica (Computer Engineering) - Torino

Course structure

Teaching | Hours |
---|---|

Lezioni | 40 |

Esercitazioni in laboratorio | 20 |

Teachers

Teacher | Status | SSD | h.Les | h.Ex | h.Lab | h.Tut | Years teaching |
---|---|---|---|---|---|---|---|

Cumani Sandro | Professore Associato | ING-INF/05 | 40 | 0 | 0 | 0 | 3 |

Teaching assistant

Context

SSD | CFU | Activities | Area context |
---|---|---|---|

ING-INF/05 | 6 | B - Caratterizzanti | Ingegneria informatica |

2020/21

The course aims at providing a solid introduction to machine learning, a branch of artificial intelligence that deals with the development of algorithms able to extract knowledge from data, with a focus on pattern recognition and classification problems. The course will cover the basic concepts of statistical machine learning, both from the frequentist and the Bayesian perspectives, and will be focused on the broad class of generative linear Gaussian models and discriminative classifiers based on logistic regression and support vector machines. The objective of the course is to provide the students with solid theoretical bases that will allow them to select, apply and evaluate different classification methods on real tasks. The students will also acquire the required competencies to devise novel approaches based on the frameworks that will be presented during the classes. The course will include laboratory activities that will allow the students to practice the theoretical notions on real data using modern programming frameworks that are widely employed both by research communities and companies.

The course aims at providing a solid introduction to machine learning, a branch of artificial intelligence that deals with the development of algorithms able to extract knowledge from data, with a focus on pattern recognition and classification problems. The course will cover the basic concepts of statistical machine learning and will concentrate on the broad class of generative linear Gaussian models and discriminative classifiers based on logistic regression and support vector machines. The objective of the course is to provide the students with solid theoretical bases that will allow them to select, apply and evaluate different classification methods on real tasks. The students will also acquire the required competencies to devise novel approaches based on the frameworks that will be presented during the classes. The course will include laboratory activities that will allow the students to practice the theoretical notions on real data using modern programming frameworks that are widely employed both by the research communities and companies.

At the end of the course the students will
- know and understand the basic principles of statistical machine learning applied to pattern recognition and classification;
- know the principal techniques for classification, including generative linear Gaussian models and discriminative approaches based on logistic regression and support vector
machines, among others;
- understand the theoretical motivations behind different classification approaches, their main properties and domain of application, and their limitations;
- be able to implement the different algorithms using wide-spread programming frameworks (Python)
- be able to apply different methods to real tasks, to critically evaluate their effectiveness and to analyze which strategies are better suited to different applications;
- be able to transfer the acquired knowledge and capabilities to solve novel classification problems, developing novel methods based on the frameworks that will be discussed
during classes

At the end of the course the students will:
- know and understand the basic principles of statistical machine learning applied to pattern recognition and classification;
- know the principal techniques for classification, including generative linear Gaussian models and discriminative approaches based on logistic regression and support vector machines, among others;
- understand the theoretical motivations behind different classification approaches, their main properties and domain of application, and their limitations;
- be able to implement the different algorithms using wide-spread programming frameworks (Python);
- be able to apply different methods to real tasks, to critically evaluate their effectiveness and to analyze which strategies are better suited to different applications;
- be able to transfer the acquired knowledge and capabilities to solve novel classification problems, developing novel methods based on the frameworks that will be discussed during classes

The students should have basic knowledge of probability and statistics, linear algebra and calculus.

The students should have basic knowledge of probability and statistics, linear algebra, calculus and programming.

Machine learning and pattern recognition
- Introduction and definitions
Probability theory concepts
- Random Variables
- Estimators
- The Bayesian framework
Introduction to Python
- The language
- Main numerical libraries
Decision Theory
- Inference, expected loss
- Model taxonomy: generative and discriminative approaches
- Model optimization, hyperparameter selection, cross-validation
Model evaluation
- Classification scores and log-likelihood ratios
- Detection Cost Functions and optimal Bayes decisions
Dimensionality reduction
- Principal Component Analysis (PCA)
- Linear Discriminant Analysis (LDA)
Generative Gaussian models
- Generative Gaussian classifiers: univariate Gaussian, Naive Bayes, multivariate Gaussian (MVG)
- Tied covariance MVG and LDA
Logistic Regression (LR)
- From Tied MVG to LR
- LR as ML solution for class labels
- Binary and multiclass cross-entropy
- From MVG to Quadratic LR
- LR as empirical risk minimization
- Overfitting and regularization
Support Vector Machines (SVM)
- Optimal classification hyperplane: the maximum margin definition
- Margin maximization and L2 regularization
- SVM as minimization of classification errors
- Primal and dual SVM formulation
- Non linear extension: brief introduction to kernels
Density estimation and latent variable models
- Gaussian mixture models (GMM)
- The Expectation Maximization algorithm
Continuous latent variable models: Linear-Gaussian Models
- Linear regression
- Linear regression and Tied MVG
- MVG with unknown class means: Probabilistic LDA (PLDA)
- Bayesian MVG
- Factor Analysis: PLDA, Probabilistic PCA
Approximated inference basics
- Variational Bayes

Introduction
- Machine learning and pattern recognition
- Probability theory concepts
- Python: language, main numerical libraries
Decision Theory
- Inference and decisions
- Model taxonomy: generative and discriminative approaches
- Model optimization, hyperparameter selection, cross-validation
Model evaluation
- Classification scores and log-likelihood ratios
- Detection Cost Functions and optimal Bayes decisions
Dimensionality reduction
- Principal Component Analysis (PCA)
- Linear Discriminant Analysis (LDA)
Generative Gaussian models
- Generative Gaussian classifiers: univariate Gaussian, Naive Bayes, multivariate Gaussian (MVG)
- Tied covariance MVG and LDA
Logistic Regression (LR)
- Tied MVG and LR
- LR as Maximum Likelihood solution for class labels
- Binary and multiclass cross-entropy
- LR as empirical risk minimization
- Overfitting and regularization
- MVG and Quadratic LR
Support Vector Machines (SVM)
- Optimal classification hyperplane: the maximum margin definition
- Soft margin and L2 regularization
- Primal and dual SVM formulation
- Non linear extension: brief introduction to kernels
Density estimation and latent variable models
- Gaussian mixture models (GMM)
- The Expectation Maximization (EM) algorithm
Continuous latent variable models: Linear-Gaussian Models
- Linear regression and Tied MVG
- MVG with unknown class means: Probabilistic LDA (PLDA)
- Bayesian MVG: prior, likelihood and predictive distribution
- Factor Analysis (FA): introduction, PLDA as a FA model, Probabilistic PCA (PPCA)
Approximated inference basics
- Laplace approximation
- Variational methods

The course will include 3 hours of lectures and 1,5 hours of laboratory per week. The lectures will focus both on theoretical and practical aspects, and will include open discussions aimed at developping suitable solutions for different problems. The laboratories will allow the students to implement most of the techniques that will be presented during the lectures, and to apply the learned methods to real data.

The course will include 3 hours of lectures and 1,5 hours of laboratory per week. The lectures will focus both on theoretical and practical aspects, and will include open discussions aimed at developing suitable solutions for different problems. The laboratories will allow the students to implement most of the techniques that will be presented during the lectures, and to apply the learned methods to real data.

[1] Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg.
[2] Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press.
Additional material, including slides and code fragments, will be made available on the course website.

[1] Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg.
[2] Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press.
Additional material, including slides and code fragments, will be made available on the course website.

The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems.
The exam will consist in two parts:
- A project to be developed during the course. The students will be able to choose individual or (small) group projects among a set of possible choice.
- An oral examination.
The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suited models based on the topics presented during lectures. Each candidate will have to provide a technical report detailing the employed methodology and a critical analysis of the obtained results.
The oral examination will start from a discussion of the project report and will cover the topics presented during the lectures.

The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems.
The exam will consist in two parts:
- A project to be developed during the course. The students will be able to choose individual or (small) group projects among a set of possible choices.
- An oral examination.
The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suited models based on the topics presented during lectures. Each candidate will have to provide a technical report detailing the employed methodology and a critical analysis of the obtained results.
The oral examination will start from a discussion of the project report and will cover the topics presented during the lectures.

The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems.
The exam will consist in two parts:
- A project to be developed during the course. The students will be able to choose individual or (small) group projects among a set of possible choice.
- An oral examination.
The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suited models based on the topics presented during lectures. Each candidate will have to provide a technical report detailing the employed methodology and a critical analysis of the obtained results.
The oral examination will start from a discussion of the project report and will cover the topics presented during the lectures.

The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems.
The exam will consist in two parts:
- A project to be developed during the course. The students will be able to choose individual or (small) group projects among a set of possible choices.
- An oral examination.
The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suited models based on the topics presented during lectures. Each candidate will have to provide a technical report detailing the employed methodology and a critical analysis of the obtained results.
The oral examination will start from a discussion of the project report and will cover the topics presented during the lectures.

© Politecnico di Torino

Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY

Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY