Politecnico di Torino
Politecnico di Torino
Politecnico di Torino
Academic Year 2007/08
Information theory and codes
Master of science-level of the Bologna process in Computer Engineering - Torino
Master of science-level of the Bologna process in Telecommunication Engineering - Torino
Master of science-level of the Bologna process in Computer Engineering - Torino
Teacher Status SSD Les Ex Lab Tut Years teaching
Elia Michele ORARIO RICEVIMENTO     5 0 0 0 9
SSD CFU Activities Area context
ING-INF/03 5 C - Affini o integrative Discipline ingegneristiche
Objectives of the course
The course presents the fundamental notions of information theory and coding. The method followed is chiefly oriented to providing the axiomatic and algebraic foundations needed for a full understanding of the abstract principles of information theory and the sophisticated mathematics of coding. An equally important goal is to prepare the student to master both theoretical and technical concepts sufficiently well to handle immediate practical applications.
A general introduction to the Shannon model of digital communication systems is provided. The key notion of measure of information for both discrete and continuous random variables is introduced and its far-reaching properties are derived in detail. Shannon's fundamental theorems are then formulated, illustrating how they have laid the foundations of a theory that has found applications in many fields of human activity. Motivated by the two Shannon theorems, namely the source coding theorem, and the channel coding theorem, both source encoding and channel code theory are then developed in full. In particular, the class of linear block codes is described, with emphasis on cyclic codes (BCH, and Reed-Solomon codes), and Goppa codes.

Syllabus: more informations
Shannon's paradigm of human and artificial communication, and the origins and motivations of fundamental theorems of information theory.
A measure of information: entropy of discrete random variables, and differential entropy. Mutual information of discrete and continuous random variables.
Discrete channel capacity, and Additive White Gaussian Noise channel capacity.
Data processing theorem. First Shannon theorem on source coding.
Second Shannon theorem on discrete channel coding. Applications scenario.
Linear block codes and introduction of finite fields, their main properties and relation with linear feedback shift registers.
Code parameters, existence conditions, and performance evaluations.
Complete decoding and standard array.
Cyclic codes: definition, existence conditions and BCH bound.
Systematic and non-systematic encoding algorithms.
GPZ decoding up to BCH bound.
Hamming codes, BCH codes, Golay codes, Reed-Solomon codes, and Goppa codes.
. MacWilliams, N. Sloane, The theory of error correcting codes,
North-Holland, 1977
T.M. Cover, J.A. Thomas. Elements of Information theory, Wiley, 1991
J. H vanLint, Introduction to Coding Theory, Springer, 1982
R.J. McEliece, The Theory of Information and Coding, Cambridge, 2004
M. Elia, Note di Teoria dell'Informazione (in Italian)
Revisions / Exam
Written examination at the end of the course

Programma definitivo per l'A.A.2007/08

© Politecnico di Torino
Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY
WCAG 2.0 (Level AA)