Caricamento in corso...
Large Language Models
01HZNOV, 01HZNSM
A.A. 2024/25
Course Language
Inglese
Degree programme(s)
Master of science-level of the Bologna process in Ingegneria Informatica (Computer Engineering) - Torino
Master of science-level of the Bologna process in Data Science And Engineering - Torino
Course structure
| Teaching | Hours |
|---|---|
| Lezioni | 42 |
| Esercitazioni in laboratorio | 18 |
Lecturers
| Teacher | Status | SSD | h.Les | h.Ex | h.Lab | h.Tut | Years teaching |
|---|---|---|---|---|---|---|---|
| Giobergia Flavio | Ricercatore L240/10 | IINF-05/A | 21 | 0 | 0 | 0 | 1 |
Co-lectures
Context
| SSD | CFU | Activities | Area context | ING-INF/05 | 6 | C - Affini o integrative | Attività formative affini o integrative |
|---|
2024/25
Large Language Models (LLMs) are state-of-the-art Artificial Intelligence systems that are achieving impressive human-level performance in generating fluent text and solving few-shot problems such as question answering and summarization. Thanks to their disruptive impact in several engineering fields, spanning from software development to biomedical engineering, LLMs will likely become one of the most valuable assets for the years to come.
The aim of this course is to provide a complete overview of LLMs, ranging from theoretical to practical perspectives. It first reviews the fundamentals behind LLM models including language models, encoder-decoder architectures, and transformers. Then, it introduces the main procedures for LLM training, instruction prompting, and learning from human feedback. Finally, it reviews and compares existing models and cloud LLM-based services.
The course will explore the application of LLMs to Software Engineering activities, with practical activities involving the application of prompting strategies (few-shot learning, Chain-of-Thought) to perform requirements engineering, design, development and testing activities.
Large Language Models (LLMs) are state-of-the-art Artificial Intelligence systems that are achieving impressive human-level performance in generating fluent text and solving few-shot problems such as question answering and summarization. Thanks to their disruptive impact in several engineering fields, spanning from software development to biomedical engineering, LLMs will likely become one of the most valuable assets for the years to come.
The aim of this course is to provide a complete overview of LLMs, ranging from theoretical to practical perspectives. It first reviews the fundamentals behind LLM models including language models, encoder-decoder architectures, and transformers. Then, it introduces the main procedures for LLM training, instruction prompting, and learning from human feedback. Finally, it reviews and compares existing models and cloud LLM-based services.
The course will explore the application of LLMs to Software Engineering activities, with practical activities involving the application of prompting strategies (few-shot learning, Chain-of-Thought) to perform requirements engineering, design, development and testing activities.
* Knowledge of the Transformer architecture (understand the architecture of transformers, including tokenization methods and attention mechanisms).
* Knowledge of LLM taxonomy and evolution (understand the progression and classification of language models, from foundational structures to advanced LLMs).
* Knowledge of model evaluation and improvement techniques: (understand and use metrics and methods for assessing the performance of LLMs, including techniques for model compression and efficiency improvements).
* Knowledge of LLM fine-tuning: (be able to customize and fine-tune LLMs for specific tasks).
* Knowledge of the main Python libraries for LLMs (e.g., HuggingFace suite, PyTorch).
* Knowledge of Practical Applications in Software Engineering (understand the phases of the software process and the way LLMs can be applied to each of them. Gain knowledge about how to engineer prompt for requirements specification, code generation, automated documentation, and software testing).
* Knowledge of Prompt Engineering (understand effective techniques for prompt engineering, including zero-shot, few-shot, and many-shot learning, and Chain-of-Thought reasoning).
* Ability to Address Ethical and Societal Impacts (understand the societal and ethical issues related to the LLMs development, training and utilization. Develop a critical viewpoint on AI development and recognize emerging trends, advancements and challenges in the field of LLMs).
* Knowledge of the Transformer architecture (understand the architecture of transformers, including tokenization methods and attention mechanisms).
* Knowledge of LLM taxonomy and evolution (understand the progression and classification of language models, from foundational structures to advanced LLMs).
* Knowledge of model evaluation and improvement techniques (understand and use metrics and methods for assessing the performance of LLMs, including techniques for model compression and efficiency improvements).
* Knowledge of LLM fine-tuning (be able to customize and fine-tune LLMs for specific tasks).
* Knowledge of the main Python libraries for LLMs (e.g., HuggingFace suite, PyTorch).
* Knowledge of Practical Applications in Software Engineering (understand the phases of the software process and the way LLMs can be applied to each of them. Gain knowledge about how to engineer prompts for requirements specification, code generation, automated documentation, and software testing).
* Knowledge of Prompt Engineering (understand effective techniques for prompt engineering, including zero-shot, few-shot, and many-shot learning, and Chain-of-Thought reasoning).
* Ability to Address Ethical and Societal Impacts (understand the societal and ethical issues related to the LLMs development, training and utilization. Develop a critical viewpoint on AI development and recognize emerging trends, advancements and challenges in the field of LLMs).
* Knowledge of machine and deep learning core concepts.
* Solid knowledge of the Python programming language and main libraries used in machine learning (NumPy and PyTorch)
* Knowledge of machine learning core concepts.
* Solid knowledge of the Python programming language and main libraries used in machine learning (NumPy/scikit-learn) or deep learning (PyTorch/TensorFlow)
* Introduction to Deep Learning (0.5 cfu)
* Transformers (1 cfu)
* Large Language Models - main concepts (1 cfu)
* Technical aspects of LLMs (0.5 cfu)
* Practical Applications and Usage of LLMs (2 cfu)
* Ethical Considerations and Future Directions (1 cfu)
* Introduction to Deep Learning (0.5 cfu)
* Transformers (1 cfu)
* Large Language Models - main concepts (1 cfu)
* Technical aspects of LLMs (0.5 cfu)
* Practical Applications and Usage of LLMs (2 cfu)
* Ethical Considerations and Future Directions (1 cfu)
* Theory lectures (in classroom): coverage of the theoretical topics about AI basics, LLMs, Software Engineering, and application of LLMs to Software Engineering;
* Labs (in classroom): exercises of LLM fine-tuning, prompt engineering, application of LLM to Software Engineering activities;
* Project work (in classroom/at home): the course will include a mandatory group project work. Work on the project will be performed in dedicated lectures and remotely at the group’s convenience.
* Theory lectures (in classroom): coverage of the theoretical topics about AI basics, LLMs, Software Engineering, and application of LLMs to Software Engineering;
* Labs (in classroom): exercises of LLM fine-tuning, prompt engineering, application of LLM to Software Engineering activities;
* Project work (in classroom/at home): the course will include a mandatory group project work. Work on the project will be performed in dedicated lectures and remotely at the group’s convenience.
Material (lecture notes and slides) provided by the lecturers;
Suggested state of the art literature (conference and journal papers):
* Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
* Brown, Tom, et al. "Language models are few-shot learners." Advances in neural information processing systems 33 (2020): 1877-1901.
All the material will be made available through a dedicated course website. All the material will be in English.
Material (lecture notes and slides) provided by the lecturers;
Suggested state of the art literature (conference and journal papers):
* Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
* Brown, Tom, et al. "Language models are few-shot learners." Advances in neural information processing systems 33 (2020): 1877-1901.
All the material will be made available through a dedicated course website. All the material will be in English.
Slides; Esercizi; Esercizi risolti; Esercitazioni di laboratorio; Esercitazioni di laboratorio risolte; Video lezioni dell’anno corrente;
Lecture slides; Exercises; Exercise with solutions ; Lab exercises; Lab exercises with solutions; Video lectures (current year);
Modalità di esame: Elaborato progettuale in gruppo; Prova scritta in aula tramite PC con l'utilizzo della piattaforma di ateneo;
Exam: Group project; Computer-based written test in class using POLITO platform;
...
The exam includes a group project and a written part. The final score is defined by considering both the evaluation of the group project and the written part. The teacher may request an integrative test to confirm the obtained evaluation. Both parts are mandatory.
The group project consists in leveraging state-of-the-art large language models to solve Software Engineering activities.
The project is assigned during the final part of the course. The project can be delivered during any of the academic year's exam sessions and its score is valid for the entire academic year.
The evaluation of the group project is based on the quality of the results provided and on the motivation provided by the students for the choices performed. The project will be considered as completed when a presentation is provided by the group. Such presentation can be provided in specific dates after the written exam sessions the academic year.
The written exam will cover the theoretical concepts of the course and will include multiple choice and open questions. For multiple choice questions, wrong answers are penalized. The score of each question will be specified in the exam text. The written exam lasts 90 minutes. Textbooks, notes, electronic devices of any kind are not allowed.
The maximum grade for the group project is 12. The maximum grade for the written part is 22. The final grade is given by the sum of the two parts. The exam is passed if the grade of the group project is greater than or equal to 8, the grade of the written part is greater than or equal to 12, and the overall grade is greater than or equal to 18. If the final score is strictly greater than 31 the registered score will be 30 with honor.
Gli studenti e le studentesse con disabilità o con Disturbi Specifici di Apprendimento (DSA), oltre alla segnalazione tramite procedura informatizzata, sono invitati a comunicare anche direttamente al/la docente titolare dell'insegnamento, con un preavviso non inferiore ad una settimana dall'avvio della sessione d'esame, gli strumenti compensativi concordati con l'Unità Special Needs, al fine di permettere al/la docente la declinazione più idonea in riferimento alla specifica tipologia di esame.
Exam: Group project; Computer-based written test in class using POLITO platform;
The exam includes a group project and a written part. The final score is defined by considering both the evaluation of the group project and the written part. The teacher may request an integrative test to confirm the obtained evaluation. Both parts are mandatory.
The group project consists in leveraging state-of-the-art large language models to solve Software Engineering activities.
The project is assigned during the final part of the course. The project can be delivered during any of the academic year's exam sessions and its score is valid for the entire academic year.
The evaluation of the group project is based on the quality of the results provided and on the motivation provided by the students for the choices performed. The project will be considered as completed when a presentation is provided by the group. Such presentation can be provided in specific dates after the written exam sessions, upon passing the written exam.
The written exam will cover the theoretical concepts of the course and will include multiple choice and open questions. For multiple choice questions, wrong answers are penalized. The score of each question will be specified in the exam text. The written exam lasts 75 minutes. Textbooks, notes, electronic devices of any kind are not allowed.
The maximum grade for the group project is 15. The maximum grade for the written part is 18. The final grade is given by the sum of the two parts. The exam is passed if the grade of the group project is greater than or equal to 7, the grade of the written part is greater than or equal to 9, and the overall grade is greater than or equal to 18. If the final score is strictly greater than 31 the registered score will be 30 with honor.
In addition to the message sent by the online system, students with disabilities or Specific Learning Disorders (SLD) are invited to directly inform the professor in charge of the course about the special arrangements for the exam that have been agreed with the Special Needs Unit. The professor has to be informed at least one week before the beginning of the examination session in order to provide students with the most suitable arrangements for each specific type of exam.