Caricamento in corso...
Probability and Information Theory
01RTXPF
A.A. 2023/24
Course Language
Inglese
Degree programme(s)
Master of science-level of the Bologna process in Physics Of Complex Systems (Fisica Dei Sistemi Complessi) - Torino/Trieste/Parigi
Course structure
| Teaching | Hours |
|---|
Lecturers
| Teacher | Status | SSD | h.Les | h.Ex | h.Lab | h.Tut | Years teaching |
|---|
Co-lectures
Context
| SSD | CFU | Activities | Area context | ING-INF/05 | 8 | B - Caratterizzanti | Discipline ingegneristiche |
|---|
The fundamental tools of probability theory are introduced, with particular reference to their applications to stochastic processes, information theory, inference problems, and the modelling of physical systems.
The fundamental tools of probability theory are introduced, with particular reference to their applications to stochastic processes, information theory, inference problems, and the modelling of physical systems.
The student must master the fundamentals of probability theory and its applications to stochastic processes, information theory, inference problems, and the modelling of physical systems. In particular, the student must be able to understand and apply concepts of statistical independence and correlation of random variables, to understand and estimate the amount of information associated to a (set of) random variable(s), to formulate and solve problems in statistical inference, and to model physical phenomena in terms of interacting random variables.
The student must master the fundamentals of probability theory and its applications to stochastic processes, information theory, inference problems, and the modelling of physical systems. In particular, the student must be able to understand and apply concepts of statistical independence and correlation of random variables, to understand and estimate the amount of information associated to a (set of) random variable(s), to formulate and solve problems in statistical inference, and to model physical phenomena in terms of interacting random variables.
Mathematical analysis.
Mathematical analysis.
Definition of probability, Kolmogorov axioms, random variables
Stochastic independence, conditional probability, Bayes theorem and inference
Classical probability: Urn models, balls and boxes, random walks
Generating functions: Integer random variables, branching process
Borel-Cantelli lemmas. Laws of large numbers. Limits in probability.
Information, Shannon theorem and the Asymptotic Equipartition Property.
Mutual and relative information. Distributions of maximal entropy.
Limit laws for sums of independent random variables. Applications. Limits of validity of the CLT.
Large deviations: thin tails and fat tails
Limit theorems for extremes: The Random Energy Model.
Examples of correlated variables: Phase transitions
Information theory, statistics and Bayesian inference
Definition of probability, Kolmogorov axioms, random variables
Stochastic independence, conditional probability, Bayes theorem and inference
Classical probability: Urn models, balls and boxes, random walks
Generating functions: Integer random variables, branching process
Borel-Cantelli lemmas. Laws of large numbers. Limits in probability.
Information, Shannon theorem and the Asymptotic Equipartition Property.
Mutual and relative information. Distributions of maximal entropy.
Limit laws for sums of independent random variables. Applications. Limits of validity of the CLT.
Large deviations: thin tails and fat tails
Limit theorems for extremes: The Random Energy Model.
Examples of correlated variables: Phase transitions
Information theory, statistics and Bayesian inference
Frontal lectures, including problem sessions.
Frontal lectures, including problem sessions.
W. Feller, An Introduction to Probability Theory and its Applications (J.Wiley & Sons 1968).
Cover and Thomas, Elements of Information Theory (J. Wiley & Sons 2006).
E. T. Jaynes, Probability Thoery: the logic of science, (Cambridge U. Press 2003).
M. Mezard, A. Montanari, Information, Physics and Computation (Oxford Univ. Press 2009).
C.W. Gardiner, Handbook of stochastic methods (Springer-Verlag, 1985).
W. Feller, An Introduction to Probability Theory and its Applications (J.Wiley & Sons 1968).
Cover and Thomas, Elements of Information Theory (J. Wiley & Sons 2006).
E. T. Jaynes, Probability Thoery: the logic of science, (Cambridge U. Press 2003).
M. Mezard, A. Montanari, Information, Physics and Computation (Oxford Univ. Press 2009).
C.W. Gardiner, Handbook of stochastic methods (Springer-Verlag, 1985).
...
The examination will be based on a midterm written test and on an oral test on the course topics.
Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.
The examination will be based on a midterm written test and on an oral test on the course topics.
In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.