


Politecnico di Torino  
Academic Year 2007/08  
01JEVHT, 01JEVBP, 01JEVCY, 01JEVHZ, 01JEVKY, 01JEVKZ Information theory and codes 

Master of sciencelevel of the Bologna process in Computer Engineering  Torino Master of sciencelevel of the Bologna process in Telecommunication Engineering  Torino Master of sciencelevel of the Bologna process in Computer Engineering  Torino Espandi... 





Objectives of the course
The course presents the fundamental notions of information theory and coding. The method followed is chiefly oriented to providing the axiomatic and algebraic foundations needed for a full understanding of the abstract principles of information theory and the sophisticated mathematics of coding. An equally important goal is to prepare the student to master both theoretical and technical concepts sufficiently well to handle immediate practical applications.

Syllabus
A general introduction to the Shannon model of digital communication systems is provided. The key notion of measure of information for both discrete and continuous random variables is introduced and its farreaching properties are derived in detail. Shannon's fundamental theorems are then formulated, illustrating how they have laid the foundations of a theory that has found applications in many fields of human activity. Motivated by the two Shannon theorems, namely the source coding theorem, and the channel coding theorem, both source encoding and channel code theory are then developed in full. In particular, the class of linear block codes is described, with emphasis on cyclic codes (BCH, and ReedSolomon codes), and Goppa codes.

Syllabus: more informations
Shannon's paradigm of human and artificial communication, and the origins and motivations of fundamental theorems of information theory.
A measure of information: entropy of discrete random variables, and differential entropy. Mutual information of discrete and continuous random variables. Discrete channel capacity, and Additive White Gaussian Noise channel capacity. Data processing theorem. First Shannon theorem on source coding. Second Shannon theorem on discrete channel coding. Applications scenario. Linear block codes and introduction of finite fields, their main properties and relation with linear feedback shift registers. Code parameters, existence conditions, and performance evaluations. Complete decoding and standard array. Cyclic codes: definition, existence conditions and BCH bound. Systematic and nonsystematic encoding algorithms. GPZ decoding up to BCH bound. Hamming codes, BCH codes, Golay codes, ReedSolomon codes, and Goppa codes. 
Bibliography
. MacWilliams, N. Sloane, The theory of error correcting codes,
NorthHolland, 1977 T.M. Cover, J.A. Thomas. Elements of Information theory, Wiley, 1991 J. H vanLint, Introduction to Coding Theory, Springer, 1982 R.J. McEliece, The Theory of Information and Coding, Cambridge, 2004 M. Elia, Note di Teoria dell'Informazione (in Italian) 
Revisions / Exam
Written examination at the end of the course

