Politecnico di Torino
Politecnico di Torino
Politecnico di Torino
Academic Year 2009/10
Information theory and codes
Master of science-level of the Bologna process in Computer Engineering - Torino
Master of science-level of the Bologna process in Telecommunication Engineering - Torino
Master of science-level of the Bologna process in Computer Engineering - Torino
Teacher Status SSD Les Ex Lab Tut Years teaching
Elia Michele ORARIO RICEVIMENTO     50 0 0 0 9
SSD CFU Activities Area context
ING-INF/03 5 D - A scelta dello studente A scelta dello studente
01KRR; 01GQS
Objectives of the course
The course is a general introduction to the Shannon theory of digital communication systems. The key notion of measure of information for both discrete and continuous random variables is introduced and its far-reaching properties are derived in detail. In particular, Shannon's fundamental theorems are then formulated, illustrating how they have laid the foundations of a theory that has found applications in many fields of human activity.
The method followed is chiefly oriented to providing the axiomatic and algebraic foundations needed for a full understanding of the abstract principles of information theory and the sophisticated mathematics of error correcting codes. An equally important goal is to prepare the student to master both theoretical and technical concepts sufficiently well to handle immediate practical applications.
- Shannon's paradigm of human and artificial communication, and the origins and motivations of fundamental theorems of information theory.
- A measure of information: entropy of discrete random variables, and differential entropy.
- Mutual information of discrete and continuous random variables.
- Discrete channel capacity, and Additive White Gaussian Noise channel capacity.
- Data processing theorem.
- First Shannon theorem on source coding. Variable length codes and Huffman coding.
- Second Shannon theorem on discrete channel coding. Applications scenario.
- Linear block codes and introduction of finite fields, their main properties.
- Code parameters, existence conditions, and performance evaluations.
- Complete decoding and standard array.
- Cyclic codes: definition, existence conditions and BCH bound.
- Systematic and non-systematic encoding algorithms.
- GPZ decoding up to BCH bound.
- Hamming codes, BCH codes, Golay codes, Reed-Solomon codes, and Goppa codes.

Mac Williams, N. Sloane, The theory of error correcting codes, North-Holland, 1977
T.M. Cover, J.A. Thomas. Elements of Information theory, Wiley, 1991
J. H van Lint, Introduction to Coding Theory, Springer, 1982
R.J. McEliece, The Theory of Information and Coding, Cambridge, 2004
M. Elia, Note di Teoria dell'Informazione (in Italian)
Check availability at the library
Revisions / Exam
The examination consists of written exercises, some of which will be in the form of homework, others under supervision.

Programma definitivo per l'A.A.2009/10

© Politecnico di Torino
Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY
WCAG 2.0 (Level AA)