| KEYWORD |
Multi-agent algorithms for optimisation in Machine Learning
keywords MACHINE LEARNING, MATHEMATICAL MODELS, MULTI-AGENT SYSTEMS, OPTIMISATION, PARTICLE METHODS
Reference persons ANDREA TOSIN
Research Groups Metodi e modelli matematici per sistemi complessi
Thesis type MASTER THESIS, MODELLING, RESEARCH THESIS, THEORETICAL
Description **NOTICE** Those interested in this proposal of MSc thesis should contact the teachers by email. Please, do not send requests from Portale della Didattica.
A central problem in modern Machine Learning is the approximation of functions depending on a large number of variables. A strictly correlated problem is the search for maximum and minimum points of such functions, which, due to the high dimensionality, cannot be effectively computed, either analytically or numerically, using classical techniques, such as e.g., gradient descent. A further difficulty is that often those functions are only approximately known from empirical data and do not possess the regularity required by classical analytical techniques.
Multi-agent algorithms used for optimisation problems in Machine Learning are inspired by some interaction models in animal groups, which have been deeply studied, both empirically and theoretically, through several mathematical-physical approaches. The basic idea consists in imagining that a group of particles explores the space of the variables of the function to be optimised, looking for points where the function attains its maximum or minimum values. Each particle relies only on local information available in its neighbourhood but, thanks to the interactions with the other particles, it may move in more convenient positions or may attract other particles if its current position is sufficiently convenient. In the abstract, this process is similar to the search for food sources by insect colonies.
Some popular classes of multi-agent algorithms for optimisation problems in Machine Learning are the so-called "Particle Swarm Optimisation" and "Consensus-Based Optimisation". From the point of view of algorithms, the advantage of these methods is twofold: on one hand, they can be easily implemented and automated; on the other hand, their computational cost and the approximation error that they produce are virtually independent of the number of variables of the function to be optimised. Taking advantage of the mathematical-physical techniques developed to model interacting multi-agent systems (such as e.g., deterministic and stochastic dynamical systems, statistical mechanics and kinetic theory, hydrodynamic limits and macroscopic descriptions), it is possible to investigate rigorously the theoretical efficiency of particle optimisation algorithms and to design new algorithms inspired by innovative interaction models.
Required skills - Elementi di equazioni differenziali alle derivate ordinarie e parziali;
- Equazioni della fisica matematica;
- Elementi di teoria della probabilità;
- Modelli di trasporto e teorie cinetiche;
- Analisi numerica ed elementi di programmazione
Notes References:
- J. A. Carrillo, Y.-P. Choi, C. Totzeck, O. Tse. An analytical framework for consensus-based global optimization method, Math. Models Methods Appl. Sci., 28(6):1037-1066, 2018
- S. Grassi, L. Pareschi. From particle swarm optimization to consensus based optimization: stochastic modeling and mean-field limit, Math. Models Methods Appl. Sci., 31(8):1625-1657, 2021
- L. Pareschi, G. Toscani. Interacting Multiagent Systems: Kinetic equations and Monte Carlo methods. Oxford University Press, 2013
- R. Pinnau, C. Totzeck, O. Tse, S. Martin. A consensus-based model for global optimization and its mean-field limit, Math. Models Methods Appl. Sci., 27(1):183-204, 2017
Deadline 01/12/2021
PROPONI LA TUA CANDIDATURA