| KEYWORD |
Injecting prior knowledge in image interpretation tasks
keywords ARTIFICIAL INTELLIGENCE, DEEP LEARNING, KNOWLEDGE GRAPHS
Reference persons LIA MORRA
Research Groups DAUIN - GR-09 - GRAphics and INtelligent Systems - GRAINS
Description Image interpretation tasks, such as object detection and visual relationship detection, may benefit from neural-symbolic integration in order to inject prior knowledge, e.g., from knowledge graphs, about objects and their relationship into the training objectives of deep neural networks. The goal of this project is to design deep architectures combining convolutional neural networks and trasformers with neural-symbolic components such as Logic Tensor Networks to solve various image interpretation tasks, such as object detection, image classification or visual relationship detection. Multiple thesis are available to tackle the following open issues: i) how to integrate the knowledge into the training process of high-level semantic interpretation (scene graph generation); ii) how to automatically extract and compile prior knowledge from existing sources and format it for appropriate ingestion; iii) how to use Large Language Models (ChatGPT) to extract prior knowledge and logical constraints; iv) how to characterize neural-symbolic architectures, e.g. in terms of types of features learnt, efficiency and interpretability; v) how to combine concept extraction with knowledge induction to close the neuro-symbolic cycle, and how to simplify knowledge acquisition through human-in-the-loop appraoches
Prerequisites: programming skills (Python, Pytorch or other deep learning framework); good analytical and mathematical skills. Prior knowledge of neuro-symbolic techniques is not required – essential material to study on the topic will be provided. The candidate we are looking for is highly motivated and interested towards research-oriented activities.
Deadline 26/02/2025
PROPONI LA TUA CANDIDATURA