01HERUU

A.A. 2023/24

Course Language

Inglese

Degree programme(s)

Course structure

Teaching | Hours |
---|

Lecturers

Teacher | Status | SSD | h.Les | h.Ex | h.Lab | h.Tut | Years teaching |
---|

Co-lectuers

Context

SSD | CFU | Activities | Area context |
---|---|---|---|

2023/24

The course aims at providing a solid introduction to machine learning, a branch of artificial intelligence that deals with the development of algorithms able to extract knowledge from data, with a focus on pattern recognition and classification problems. The course will cover the basic concepts of machine learning and will focus on generative Gaussian models, on discriminative approaches such as logistic regression, support vector machines and neural networks, and on clustering methods. The objective of the course is to provide the students with solid theoretical bases that will allow them to select, apply and evaluate different machine learning methods on real tasks. The students will also acquire the required competencies to devise novel approaches based on the frameworks that will be presented during the classes. The course will include laboratory activities that will allow the students to practice the theoretical notions on real data using modern programming frameworks that are widely employed both by the research communities and companies.

The course aims at providing a solid introduction to machine learning, a branch of artificial intelligence that deals with the development of algorithms able to extract knowledge from data, with a focus on pattern recognition and classification problems. The course will cover the basic concepts of machine learning and will focus on generative Gaussian models, on discriminative approaches such as logistic regression, support vector machines and neural networks, and on clustering methods. The objective of the course is to provide the students with solid theoretical bases that will allow them to select, apply and evaluate different machine learning methods on real tasks. The students will also acquire the required competencies to devise novel approaches based on the frameworks that will be presented during the classes. The course will include laboratory activities that will allow the students to practice the theoretical notions on real data using modern programming frameworks that are widely employed both by the research communities and companies.

At the end of the course the students will:
- know and understand the basic principles of machine learning applied to pattern recognition, classification and clustering;
- know the principal techniques for clustering, including K-means and hierarchical methods, and for classification, including generative Gaussian models and discriminative approaches based on logistic regression, support vector machines and neural networks, among others;
- understand the theoretical motivations behind different machine learning approaches, their main properties and domain of application, and their limitations;
- be able to deploy the different algorithms using wide-spread programming frameworks (Python);
- be able to apply different methods to real tasks, to critically evaluate their effectiveness and to analyze which strategies are better suited to different applications;
- be able to transfer the acquired knowledge and capabilities to solve novel machine learning problems, and to develop novel methods based on the frameworks that will be discussed during classes.

At the end of the course the students will:
- know and understand the basic principles of machine learning applied to pattern recognition, classification and clustering;
- know the principal techniques for clustering, including K-means and hierarchical methods, and for classification, including generative Gaussian models and discriminative approaches based on logistic regression, support vector machines and neural networks, among others;
- understand the theoretical motivations behind different machine learning approaches, their main properties and domain of application, and their limitations;
- be able to deploy the different algorithms using wide-spread programming frameworks (Python);
- be able to apply different methods to real tasks, to critically evaluate their effectiveness and to analyze which strategies are better suited to different applications;
- be able to transfer the acquired knowledge and capabilities to solve novel machine learning problems, and to develop novel methods based on the frameworks that will be discussed during classes.

The students should have basic knowledge of probability and statistics, linear algebra, calculus and programming.

The students should have basic knowledge of probability and statistics, linear algebra, calculus and programming.

Introduction
- Machine learning and pattern recognition
- Probability theory concepts
- Python: language, main numerical libraries
Decision Theory and Model Evaluation
- Inference and decisions
- Classification scores and log-likelihood ratios
- Optimal Bayes Decisions
- Evaluation metrics
Clustering
- K-means
- Hierarchical clustering
Dimensionality reduction
- Principal Component Analysis (PCA)
- Linear Discriminant Analysis (LDA)
Generative models
- Generative Gaussian classifiers: univariate Gaussian, Naive Bayes, multivariate Gaussian (MVG)
- Tied covariance MVG and LDA
Logistic Regression (LR)
- The Logistic Regression model
- LR as Maximum Likelihood solution for class labels
- Binary and multiclass cross-entropy
- LR as empirical risk minimization
- Overfitting and regularization
- Quadratic LR
Support Vector Machines (SVM)
- Optimal classification hyperplane: the maximum margin definition
- Soft margin and L2 regularization
- Primal and dual SVM formulation
- Non linear extension: kernels
Neural networks
- Neural networks for non-linear classification and regression
- Feed-forward neural networks and multi-layer perceptron
- Convolutional and deep neural networks

Introduction
- Machine learning and pattern recognition
- Probability theory concepts
- Python: language, main numerical libraries
Decision Theory and Model Evaluation
- Inference and decisions
- Classification scores and log-likelihood ratios
- Optimal Bayes Decisions
- Evaluation metrics
Clustering
- K-means
- Hierarchical clustering
Dimensionality reduction
- Principal Component Analysis (PCA)
- Linear Discriminant Analysis (LDA)
Generative models
- Generative Gaussian classifiers: univariate Gaussian, Naive Bayes, multivariate Gaussian (MVG)
- Tied covariance MVG and LDA
Logistic Regression (LR)
- The Logistic Regression model
- LR as Maximum Likelihood solution for class labels
- Binary and multiclass cross-entropy
- LR as empirical risk minimization
- Overfitting and regularization
- Quadratic LR
Support Vector Machines (SVM)
- Optimal classification hyperplane: the maximum margin definition
- Soft margin and L2 regularization
- Primal and dual SVM formulation
- Non linear extension: kernels
Neural networks
- Neural networks for non-linear classification and regression
- Feed-forward neural networks and multi-layer perceptron
- Convolutional and deep neural networks

The course will include 40 hours of lectures and 20 hours of laboratory. The lectures will focus both on theoretical and practical aspects, and will include open discussions aimed at developing suitable solutions for different problems. The laboratories will allow the students to implement most of the techniques that will be presented during the lectures, and to apply the learned methods to synthetic and real problems.

The course will include 40 hours of lectures and 20 hours of laboratory. The lectures will focus both on theoretical and practical aspects, and will include open discussions aimed at developing suitable solutions for different problems. The laboratories will allow the students to implement most of the techniques that will be presented during the lectures, and to apply the learned methods to synthetic and real problems.

[1] Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg.
[2] Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press.
Additional material, including slides and code fragments, will be made available on the course website.

[1] Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer-Verlag, Berlin, Heidelberg.
[2] Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. The MIT Press.
Additional material, including slides and code fragments, will be made available on the course website.

...
The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems.
The exam will consist in two parts:
- A project to be developed during the course. The students will be able to choose individual or (small) group projects among a set of possible choices (max. 10 points).
- A written examination (max. 20 points).
The final mark will be the sum of the report and written exam marks. To pass the exam, the report mark must be at least 5/10, the written exam mark must be at least 10/20, and the final mark must be at least 18/30.
The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suitable models based on the topics presented during lectures. Each candidate will have to provide a technical report detailing the employed methodology and a critical analysis of the obtained results. The report will assess:
- The degree of understanding of the theoretical principles of machine learning for pattern recognition
- The ability of the student to analyze a specific problem, assessing which approaches, among those that have been presented, are more suited to solve the task
- The ability of the student to apply and possibly extend the studied methods to devise suitable approaches for the specific case study
- The ability of the student to critically evaluate the effectiveness of the proposed approaches.
The written examination will consist of open questions covering the topics presented during the lectures. The written examination will assess:
- The theoretical understanding of the basic principles of machine learning for pattern recognition
- The knowledge and understanding of the different approaches that have been presented during the lectures
- The ability of the student to critically analyze and evaluate the different approaches.

Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.

The exam will assess the knowledge of the course topics, and the ability of the candidate to apply such knowledge and the developed skills to solve specific problems.
The exam will consist in two parts:
- A project to be developed during the course. The students will be able to choose individual or (small) group projects among a set of possible choices (max. 10 points).
- A written examination (max. 20 points).
The final mark will be the sum of the report and written exam marks. To pass the exam, the report mark must be at least 5/10, the written exam mark must be at least 10/20, and the final mark must be at least 18/30.
The projects will address specific classification tasks. For each project, a dataset will be provided, and the students will have to develop suitable models based on the topics presented during lectures. Each candidate will have to provide a technical report detailing the employed methodology and a critical analysis of the obtained results. The report will assess:
- The degree of understanding of the theoretical principles of machine learning for pattern recognition
- The ability of the student to analyze a specific problem, assessing which approaches, among those that have been presented, are more suited to solve the task
- The ability of the student to apply and possibly extend the studied methods to devise suitable approaches for the specific case study
- The ability of the student to critically evaluate the effectiveness of the proposed approaches.
The written examination will consist of open questions covering the topics presented during the lectures. The written examination will assess:
- The theoretical understanding of the basic principles of machine learning for pattern recognition
- The knowledge and understanding of the different approaches that have been presented during the lectures
- The ability of the student to critically analyze and evaluate the different approaches.

In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.

© Politecnico di Torino

Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY

Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY