The course aims at providing a conceptual framework, based on a modern approach to probability theory, that provides a unifying language for different branches of theoretical sciences, including information theory, coding theory, statistical inference and statistical mechanics. The course also aims at providing a basic working knowledge of these subjects, by developing analytical and computational skills.

Students are expected to be able to formalize problems in modeling and inference of complex systems and apply the suitable tools for their analysis.

A solid background in mathematics (analysis, linear algebra). Basic knowledge in probability, thermodynamics, statistical mechanics and statistics is welcome but not necessary.

Definitions of probability, Kolmogorov axioms, de Finetti and Jaynes (4 hours) Stochastic independence, conditional probability, Bayes theorem and inference (4 hours) Random variables (4 hours) Classical probability: Urn models, balls and boxes, random walks (6 hours) Generating functions: Integer random variables, branching process (6 hours) Borel-Cantelli lemmas. Laws of large numbers. Limits in probability. (4 hours) Limit laws for sums of independent random variables. (6 hours) Limit theorems for extremes: The Random Energy Model. (6 hours) Information, Shannon theorem and the Asymptotic Equipartition Property. (4 hours) Mutual and relative information. (6 hours) Large deviations: thin tails and fat tails (8 hours) Distributions of maximal entropy, generalised thermodynamics (6 hours) Examples of correlated variables: Phase transitions (8 hours) Information theory, statistics and Bayesian inference (8 hours)

The course is based on pre-recorded lectures and lecture notes. Lectures in class are Q&A sessions on theoretical subjects and exercises.

W. Feller, An Introduction to Probability Theory and its Applications (J.Wiley & Sons 1968). Cover and Thomas, Elements of Information Theory (J. Wiley & Sons 2006). E. T. Jaynes, Probability Theory: the logic of science, (Cambridge U. Press 2003). M. Mezard, A. Montanari, Information, Physics and Computation (Oxford Univ. Press 2009). C.W. Gardiner, Handbook of stochastic methods (Springer-Verlag, 1985).

Dispense; Video lezioni tratte da anni precedenti;

Modalità di esame: Prova scritta (in aula); Prova orale obbligatoria;

Exam: Written test; Compulsory oral exam;

... Assessment is based on a mid-term written test, on exercises that cover the first part and a final oral exam.

Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.