PORTALE DELLA DIDATTICA

### Probability and Information Theory

01RTXPF

A.A. 2023/24

Course Language

Inglese

Course degree

Master of science-level of the Bologna process in Physics Of Complex Systems (Fisica Dei Sistemi Complessi) - Torino/Trieste/Parigi

Course structure
Teaching Hours
Teachers
Teacher Status SSD h.Les h.Ex h.Lab h.Tut Years teaching
Teaching assistant
Context
SSD CFU Activities Area context
ING-INF/05 8 B - Caratterizzanti Discipline ingegneristiche
The fundamental tools of probability theory are introduced, with particular reference to their applications to stochastic processes, information theory, inference problems, and the modelling of physical systems.
The fundamental tools of probability theory are introduced, with particular reference to their applications to stochastic processes, information theory, inference problems, and the modelling of physical systems.
The student must master the fundamentals of probability theory and its applications to stochastic processes, information theory, inference problems, and the modelling of physical systems. In particular, the student must be able to understand and apply concepts of statistical independence and correlation of random variables, to understand and estimate the amount of information associated to a (set of) random variable(s), to formulate and solve problems in statistical inference, and to model physical phenomena in terms of interacting random variables.
The student must master the fundamentals of probability theory and its applications to stochastic processes, information theory, inference problems, and the modelling of physical systems. In particular, the student must be able to understand and apply concepts of statistical independence and correlation of random variables, to understand and estimate the amount of information associated to a (set of) random variable(s), to formulate and solve problems in statistical inference, and to model physical phenomena in terms of interacting random variables.
Mathematical analysis.
Mathematical analysis.
Definition of probability, Kolmogorov axioms, random variables Stochastic independence, conditional probability, Bayes theorem and inference Classical probability: Urn models, balls and boxes, random walks Generating functions: Integer random variables, branching process Borel-Cantelli lemmas. Laws of large numbers. Limits in probability. Information, Shannon theorem and the Asymptotic Equipartition Property. Mutual and relative information. Distributions of maximal entropy. Limit laws for sums of independent random variables. Applications. Limits of validity of the CLT. Large deviations: thin tails and fat tails Limit theorems for extremes: The Random Energy Model. Examples of correlated variables: Phase transitions Information theory, statistics and Bayesian inference
Definition of probability, Kolmogorov axioms, random variables Stochastic independence, conditional probability, Bayes theorem and inference Classical probability: Urn models, balls and boxes, random walks Generating functions: Integer random variables, branching process Borel-Cantelli lemmas. Laws of large numbers. Limits in probability. Information, Shannon theorem and the Asymptotic Equipartition Property. Mutual and relative information. Distributions of maximal entropy. Limit laws for sums of independent random variables. Applications. Limits of validity of the CLT. Large deviations: thin tails and fat tails Limit theorems for extremes: The Random Energy Model. Examples of correlated variables: Phase transitions Information theory, statistics and Bayesian inference
Frontal lectures, including problem sessions.
Frontal lectures, including problem sessions.
W. Feller, An Introduction to Probability Theory and its Applications (J.Wiley & Sons 1968). Cover and Thomas, Elements of Information Theory (J. Wiley & Sons 2006). E. T. Jaynes, Probability Thoery: the logic of science, (Cambridge U. Press 2003). M. Mezard, A. Montanari, Information, Physics and Computation (Oxford Univ. Press 2009). C.W. Gardiner, Handbook of stochastic methods (Springer-Verlag, 1985).
W. Feller, An Introduction to Probability Theory and its Applications (J.Wiley & Sons 1968). Cover and Thomas, Elements of Information Theory (J. Wiley & Sons 2006). E. T. Jaynes, Probability Thoery: the logic of science, (Cambridge U. Press 2003). M. Mezard, A. Montanari, Information, Physics and Computation (Oxford Univ. Press 2009). C.W. Gardiner, Handbook of stochastic methods (Springer-Verlag, 1985).
... The examination will be based on a midterm written test and on an oral test on the course topics.
Gli studenti e le studentesse con disabilit� o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unit� Special Needs, al fine di permettere al/la docente la declinazione pi� idonea in riferimento alla specifica tipologia di esame.
The examination will be based on a midterm written test and on an oral test on the course topics.
In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.