| KEYWORD |
Adaptive deep learning workload for nanorobotic System-on-Chip
Thesis abroad
keywords ARTIFICIAL INTELLIGENCE, CONVOLUTIONAL NEURAL NETWORKS, DEEP LEARNING, DEEP NEURAL NETWORKS, DRONES, EMBEDDED SYSTEMS, ENERGY EFFICIENCY, FIRMWARE DEVELOPMENT, LOW POWER, MICROCONTROLLERS, ROBOTICS, SOFTWARE, UAV
Reference persons DANIELE JAHIER PAGLIARI
External reference persons Dr. Daniele Palossi (SUPSI-USI, Università della Svizzera Italiana)
Research Groups DAUIN - GR-06 - ELECTRONIC DESIGN AUTOMATION - EDA, ELECTRONIC DESIGN AUTOMATION - EDA, GR-06 - ELECTRONIC DESIGN AUTOMATION - EDA
Thesis type EMBEDDED SOFTWARE DEVELOPMENT, EXPERIMENTAL, SOFTWARE DEVELOPMENT
Description Description and goal
Nano-drones are extremely flexible and agile robotics platforms but with very limited onboard energy/computational/memory resources. In this project, we aim at developing and deploying, aboard a nano-drone, a deep learning-based intelligence able to understand when the system should switch from a lightweight Convolutional Neural Network (CNN) to a more complex one, capable of higher regression performance but at a higher computational/memory price.
Given the robotic task of human pose estimation [1], the candidate will employ several variants of the same CNN which will trade regression performance in favor of less energy/operations/memory. Then, by extending the CNN’s output for the human pose estimation task by an additional “confidence” scalar output, the system will gain the required awareness to switch to heavier models when the task becomes too challenging for a lighter CNN -- which means when the lighter CNN performs poorly. The system will be deployed and in-field tested by employing the Crazyflie 2.1 nano-drone [2].
Tasks
- Familiarization with the PULP-Frontnet human pose estimation CNN [1,3];
- design, training, and testing of the 3 CNNs variants, with the additional confidence output;
- familiarization with the PULP GAP8 C embedded programming;
- deployment of the 3 CNNs on the PULP GAP8 SoC available on the Ai-deck pluggable PCB [4];
- in-field testing of the final prototype running the adaptive workload mechanism aboard the nano-drone.
References
[1] https://arxiv.org/abs/2103.10873
[2] https://www.bitcraze.io/
[3] https://github.com/idsia-robotics/pulp-frontnet
[4] https://www.bitcraze.io/products/ai-deck/
See also screenshot from 2022-04-21 20-30-37.png
Required skills Intermediate (the higher the better) Python and C programming skills. Familiarity with PyTorch framework. General knowledge of embedded programming for microcontrollers is favorable.
Notes Thesis in collaboration with Dalle Molle Institute for Artificial Intelligence (IDSIA) SUPSI-USI, Lugano, Switzerland. The thesis can be carried out partly in Lugano (allowing the candidate to run in-field tests and directly touch the results of the work) or fully from Torino (leaving in-field tests to our external collaborators).
Deadline 13/12/2022
PROPONI LA TUA CANDIDATURA