01DTGSM

A.A. 2023/24

Course Language

Inglese

Degree programme(s)

Master of science-level of the Bologna process in Data Science And Engineering - Torino

Course structure

Teaching | Hours |
---|---|

Lezioni | 74 |

Esercitazioni in aula | 6 |

Lecturers

Teacher | Status | SSD | h.Les | h.Ex | h.Lab | h.Tut | Years teaching |
---|---|---|---|---|---|---|---|

Garello Roberto | Professore Associato | ING-INF/03 | 24 | 6 | 0 | 0 | 2 |

Co-lectuers

Context

SSD | CFU | Activities | Area context |
---|---|---|---|

ING-INF/03 | 8 | C - Affini o integrative | Attività formative affini o integrative |

2022/23

Information Theory studies how to quantify, represent, store, compress, transmit information. It is a fundamental tool for a data scientist and has many important applications in Classifiers, Machine Learning, Artificial Intelligence, Data compression, Data transmission and Cryptography. The course aims to provide the basis of Information Theory and show some of the most important applications for Data Science, including Classifiers, Data Compression and Cryptography. The course has a learning-by-doing approach where for each topic an assignment is proposed that must be solved by writing a Matlab or Python program and writing a report. About half of the hours are devoted to tutoring, where the teachers are available to help students with the assignments.

Information Theory studies how to quantify, represent, store, compress, transmit information. It is a fundamental tool for a data scientist and has many important applications in Classifiers, Machine Learning, Artificial Intelligence, Data compression, Data transmission and Cryptography. The course aims to provide the basis of Information Theory and show some of the most important applications for Data Science, including Classifiers, Data Compression and Cryptography. The course has a learning-by-doing approach where for each topic an assignment is proposed that must be solved by writing a Matlab or Python program and writing a report. About half of the hours are devoted to tutoring, where the teachers are available to help students with the assignments.

At the end of the course the student will know:
- The key concepts of Information Theory
- The meaning of Entropy for Data Science
- The application of Information Theory to Classifiers
- The application of Information Theory to Data Compression and Representation
- The role of Information Theory for Cryptography and some of the most important Crypto Algorithms
- How to realize a Matlab of Python project to analyze an Information Theory problem

At the end of the course the student will:
- Know the key concepts of Information Theory
- Understand the meaning of Entropy for Data Science
- Be aware of the application of Information Theory to Classifiers
- Be aware of the application of Information Theory to Data Compression and Representation
- Recognize the role of Information Theory for Cryptography and some of the most important Crypto Algorithms
- Know how to realize a Matlab of Python project to analyze an Information Theory problem

Very basic programming skills in Matlab or Python.
Basic notions of probability theory

Very basic programming skills in Matlab or Python.
Basic notions of probability theory

The course topics are:
1. Introduction to Information Theory (10 h)
2. Application of Information Theory to Classifiers (14 h)
3. Application of Information Theory to Data Compression 1 (14 h)
4. Application of Information Theory to Data Compression 2 (14 h)
5. Information Theory and Cryptography (14 h)
6. Algorithms fo Cryptography (14 h)

The course topics are:
1. Introduction to Information Theory (10 h)
2. Application of Information Theory to Classifiers (14 h)
3. Application of Information Theory to Data Compression 1 (14 h)
4. Application of Information Theory to Data Compression 2 (14 h)
5. Information Theory and Cryptography (14 h)
6. Algorithms fo Cryptography (14 h)

The course is organized in six sections. For each section we propose an assignment that must be solved by writing a Matlab or Python program and a report. About half of the hours are dedicated to lectures (that will be recorded) and half to tutoring, where the teachers are available to help students with the assignment solution.

The course is organized in six sections. For each section we propose an assignment that must be solved by writing a Matlab or Python program and a report. About half of the hours are dedicated to lectures (that will be recorded) and half to tutoring, where the teachers are available to help students with the assignment solution.

Thomas M. Cover and Joy A. Thomas “Elements of Information Theory”
Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola “Dive into Deep Learning” Chapter 18 “Information Theory” (available online)
Lecture recordings.
Notes from the teachers.

Thomas M. Cover and Joy A. Thomas “Elements of Information Theory”
Aston Zhang, Zack C. Lipton, Mu Li, Alex J. Smola “Dive into Deep Learning” Chapter 18 “Information Theory” (available online)
Lecture recordings.
Notes from the teachers.

...
The scope of the exam is to verify that the student:
- has acquired the basis of information theory
- can develop Matlab/Python-based projects on the application of information theory to some data science problems
- is able to analyze, present and discuss the results in a professional technical report.
The course is divided in 6 sections. For each section an assignment (to be solved individually) is proposed. The student must deliver a pdf report containing all the results, the figures and the required answers and all the written Matlab or Python programs. Each assignment is evaluated and receives a mark (max = 30). To encourage the students to stay aligned to the course, the students who deliver their assignment withing 2 weeks from when it is assigned received 2 more bonus points (max mark = 32).
At the end of the course the mean value of the assignment marks is proposed as the final mark.
An extra assignment (or an oral exam with three questions from a list of questions prepared at the end of the course) is proposed to students who want to further increase their mark (max +3/30 to the final mark).
Students who get at least 32 receive 30 cum laude.

Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.

The scope of the exam is to verify that the student:
- has acquired the basis of information theory
- can develop Matlab/Python-based projects on the application of information theory to some data science problems
- is able to analyze, present and discuss the results in a professional technical report.
The course is divided in 6 sections. For each section an assignment (to be solved individually) is proposed. The student must deliver a pdf report containing all the results, the figures and the required answers and all the written Matlab or Python programs. Each assignment is evaluated and receives a mark (max = 30). To encourage the students to stay aligned to the course, the students who deliver their assignment withing 2 weeks from when it is assigned received 2 more bonus points (max mark = 32).
At the end of the course the mean value of the assignment marks is proposed as the final mark.
An extra assignment (or an oral exam with three questions from a list of questions prepared at the end of the course) is proposed to students who want to further increase their mark (max +3/30 to the final mark).
Students who get at least 32 receive 30 cum laude.

In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.

© Politecnico di Torino

Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY

Corso Duca degli Abruzzi, 24 - 10129 Torino, ITALY