Information Theory studies how to quantify, represent, store, compress, transmit information. It is a fundamental tool for a data scientist and has many important applications in Classifiers, Machine Learning, Artificial Intelligence, Data compression, Data transmission and Cryptography. The course aims to provide the basis of Information Theory and show some of the most important applications for Data Science, including Classifiers, Data Compression and Cryptography. The course has a learning-by-doing approach where for each topic an assignment is proposed that must be solved by writing a Matlab or Python program and writing a report. About half of the hours are devoted to tutoring, where the teachers are available to help students with the assignments.

At the end of the course the student will know: - The key concepts of Information Theory - The meaning of Entropy for Data Science - The application of Information Theory to Classifiers - The application of Information Theory to Data Compression and Representation - The role of Information Theory for Cryptography and some of the most important Crypto Algorithms - How to realize a Matlab of Python project to analyze an Information Theory problem

Very basic programming skills in Matlab or Python. Basic notions of probability theory

The course topics are: 1. Introduction to Information Theory (10 h) 2. Application of Information Theory to Classifiers (14 h) 3. Application of Information Theory to Data Compression 1 (14 h) 4. Application of Information Theory to Data Compression 2 (14 h) 5. Information Theory and Cryptography (14 h) 6. Algorithms fo Cryptography (14 h)

The course is organized in six sections. For each section we propose an assignment that must be solved by writing a Matlab or Python program and a report. About half of the hours are dedicated to lectures (that will be recorded) and half to tutoring, where the teachers are available to help students with the assignment solution.

Thomas M. Cover and Joy A. Thomas “Elements of Information Theory” Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola “Dive into Deep Learning” Chapter 18 “Information Theory” (available online) Lecture recordings. Notes from the teachers.

Modalità di esame: Elaborato progettuale individuale;

Exam: Individual project;

... The scope of the exam is to verify that the student: - has acquired the basis of information theory - can develop Matlab/Python-based projects on the application of information theory to some data science problems - is able to analyze, present and discuss the results in a professional technical report. The course is divided in 6 sections. For each section an assignment (to be solved individually) is proposed. The student must deliver a pdf report containing all the results, the figures and the required answers and all the written Matlab or Python programs. Each assignment is evaluated and receives a mark (max = 30). To encourage the students to stay aligned to the course, the students who deliver their assignment withing 2 weeks from when it is assigned received 2 more bonus points (max mark = 32). At the end of the course the mean value of the assignment marks is proposed as the final mark. An extra assignment (or an oral exam with three questions from a list of questions prepared at the end of the course) is proposed to students who want to further increase their mark (max +3/30 to the final mark). Students who get at least 32 receive 30 cum laude.

Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.