PORTALE DELLA DIDATTICA

PORTALE DELLA DIDATTICA

PORTALE DELLA DIDATTICA

Elenco notifiche



Optimization for machine learning

01TVOQW, 01TVOPF

A.A. 2024/25

Course Language

Inglese

Degree programme(s)

Master of science-level of the Bologna process in Mechatronic Engineering (Ingegneria Meccatronica) - Torino
Master of science-level of the Bologna process in Physics Of Complex Systems (Fisica Dei Sistemi Complessi) - Torino/Trieste/Parigi

Course structure
Teaching Hours
Lezioni 20
Esercitazioni in aula 40
Tutoraggio 39
Lecturers
Teacher Status SSD h.Les h.Ex h.Lab h.Tut Years teaching
Calafiore Giuseppe Carlo Professore Ordinario IINF-04/A 20 40 0 0 5
Co-lectures
Espandi

Context
SSD CFU Activities Area context
ING-INF/04 6 B - Caratterizzanti Ingegneria dell'automazione
2024/25
Machine Learning (ML) encompasses a variety of methodologies and computational algorithms, mainly grounded in Bayesian statistics, for extracting information, clustering, detecting patterns, making decisions and predictions or, more generally, understanding phenomena from available data. Classical learning models created in the 1970s, such as Neural Networks, as well as later techniques such as Support Vector Machines (SVM), are witnessing a new wave of resurgence in both theory and applications in the present era of Big Data, where the deluge of unstructured information calls for automated and highly efficient methods of data analysis. Contemporary Machine Learning, in turn, constitutes an essential part of Data Science, an interdisciplinary field for which industry has a global excess demand of experts. In this course, we present the main tools for supervised learning (regression, regularization, classification) and unsupervised learning (clustering, dimensionality reduction), with a focus on the structure and features of the optimization algorithms that are needed to actually solve numerically the learning problems of interest. The course is structured into lectures in class, in which the context and methodologies are explained, and computer lab sessions, in which the students apply the methodologies to real-world data sets and problems coming from various fields, such as finance, business analytics, news, biology, medical diagnosis, etc.
Machine Learning (ML) encompasses a variety of methodologies and computational algorithms, mainly grounded in Bayesian statistics, for extracting information, clustering, detecting patterns, making decisions and predictions or, more generally, understanding phenomena from available data. Classical learning models created in the 1970s, such as Neural Networks, as well as later techniques such as Support Vector Machines (SVM), are witnessing a new wave of resurgence in both theory and applications in the present era of Big Data, where the deluge of unstructured information calls for automated and highly efficient methods of data analysis. Contemporary Machine Learning, in turn, constitutes an essential part of Data Science, an interdisciplinary field for which industry has a global excess demand of experts. In this course, we present the main tools for supervised learning (regression, regularization, classification) and unsupervised learning (clustering, dimensionality reduction), with a focus on the structure and features of the optimization algorithms that are needed to actually solve numerically the learning problems of interest. The course is structured into lectures in class, in which the context and methodologies are explained, and computer lab sessions, in which the students apply the methodologies to real-world data sets and problems coming from various fields, such as finance, business analytics, news, biology, medical diagnosis, etc.
The student will acquire knowledge of the basic tools used in machine learning, and an introductory insight on the functioning of the optimization algorithms that form the inner computational “engine” of these tools. The student will gain some experience in visualizing and analyzing labeled and unlabeled high-dimensional data sets and in extracting useful information from them. Complementing a student’s background in statistics, optimization or data mining, this course will help forming the skills of a Junior Data Scientist.
The student will acquire knowledge of the basic tools used in machine learning, and an introductory insight on the functioning of the optimization algorithms that form the inner computational “engine” of these tools. The student will gain some experience in visualizing and analyzing labeled and unlabeled high-dimensional data sets and in extracting useful information from them. Complementing a student’s background in statistics, optimization or data mining, this course will help forming the skills of a Junior Data Scientist.
Good knowledge of linear algebra, geometry, calculus and some exposure to probability and statistics is required. A previous course on numerical computing, optimization, or operations research is recommended but not strictly required.
Good knowledge of linear algebra, geometry, calculus and some exposure to probability and statistics is required. A previous course on numerical computing, optimization, or operations research is recommended but not strictly required.
Introduction to Machine Learning. Supervised and unsupervised learning. Parametric and nonparametric models. Classical examples in pattern analysis (e.g., hand-writing recognition). A brief historical perspective. Review of probability theory and statistics. Marginal and conditional distributions. Bayes theorem. Prior, likelihood, posterior. Bayesian inference. Regression problems. Over-fitting. Bias-variance tradeoff. Regularized regression. Linear regression with sparsity-inducing penalties. Ridge regression. The Lasso. The Elastic-Net. Applications (e.g., in image analysis and in computational finance). Logistic regression. Sparse logistic regression and applications (e.g., to text categorization). Algorithms for large-scale regularized regression: First-order methods. Proximal methods. The Fast Iterative Shrinkage-Thresholding Algorithm (FISTA). Coordinate descent and block-coordinate descent methods. Classifiers. Neural Networks. Training and the back-propagation algorithm. Maximum margin classifiers. Dual representation. Kernel methods and the Support Vector Machine (SVM). Clustering. K-means clustering. Gaussian mixtures and the Expectation-Minimization (EM) algorithm. Singular value decomposition and the Principal Component Analysis (PCA). Interpretability and the Sparse-PCA. Fast algorithms for Sparse-PCA.
Introduction to Machine Learning. Supervised and unsupervised learning. Parametric and nonparametric models. Classical examples in pattern analysis (e.g., hand-writing recognition). A brief historical perspective. Review of probability theory and statistics. Marginal and conditional distributions. Bayes theorem. Prior, likelihood, posterior. Bayesian inference. Regression problems. Over-fitting. Bias-variance tradeoff. Regularized regression. Linear regression with sparsity-inducing penalties. Ridge regression. The Lasso. The Elastic-Net. Applications (e.g., in image analysis and in computational finance). Algorithms for large-scale regularized regression: First-order methods. The Fast Iterative Shrinkage-Thresholding Algorithm (FISTA). Coordinate descent and block-coordinate descent methods. Classifiers. Logistic regression. Sparse logistic regression and applications (e.g., to text categorization). Decision trees Neural Networks. Training and the back-propagation algorithm. Maximum margin classifiers. Dual representation. Kernel methods and the Support Vector Machine (SVM). Clustering. K-means clustering. Gaussian mixtures and the Expectation-Minimization (EM) algorithm. Singular value decomposition and the Principal Component Analysis (PCA). Interpretability and the Sparse-PCA.
The course is organized in a series of lectures (about 1/3 of the course) and computer lab exercises and practice sessions (about 2/3 of the course).
The course is organized in a series of lectures (about 1/3 of the course) and computer lab exercises and practice sessions (about 2/3 of the course).
Course slides, handouts, and lab practice sheets will be made available to the students via the PoliTo Web portal. Useful reference textbooks are the following ones: C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006. J. Friedman, T. Hastie and R. Tibshirani, The Elements of Statistical Learning, Springer, 2009. G.C. Calafiore and L. El Ghaoui, Optimization Models, Cambridge Univ. Press, 2014.
Course slides, handouts, and lab practice sheets will be made available to the students via the PoliTo Web portal. Useful reference textbooks are the following ones: C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006. J. Friedman, T. Hastie and R. Tibshirani, The Elements of Statistical Learning, Springer, 2009. G.C. Calafiore, L. El Ghaoui, G. Fracastoro, A. Tsai, Financial Data Science, Cambridge University Press, 2025 (to appear) G.C. Calafiore and L. El Ghaoui, Optimization Models, Cambridge Univ. Press, 2014.
Slides; Dispense; Esercizi; Esercizi risolti; Esercitazioni di laboratorio risolte;
Lecture slides; Lecture notes; Exercises; Exercise with solutions ; Lab exercises with solutions;
Modalità di esame: Prova scritta (in aula);
Exam: Written test;
... The final exam consists in a written test, which will contain a mixture of methodological questions and numerical exercises (to be executed with pen and paper; use of a calculator is allowed). Depending on the numerosity of the course and exam participants, the written examination may take either the form of an open question test, or the form of a multiple-choice questionar.
Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.
Exam: Written test;
The final exam consists in a written test, which will contain a mixture of methodological questions and numerical exercises (to be executed with pen and paper; use of a scientific calculator is allowed) and which will take the form of multiple-choice questionar. Use of didactic material is NOT allowed. The exam will contain around 11-12 questions and it must be completed in approx. 1 hour. Each correct answer will give one point, each wrong answer will give -1/3 points and no answer gives zero point. The sum of points is then normalized and brought to a scale where the maximum score attainable is 30 cum laude.
In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.
Esporta Word