| KEYWORD |
Area Engineering
Using Large Language Models to support hands-free locomotion in Virtual Reality
keywords HUMAN-MACHINE INTERFACES, MACHINE LEARNING, VIRTUAL AND AUGMENTED REALITY, VOICE COMMANDS
Reference persons FABRIZIO LAMBERTI
External reference persons DAVIDE CALANDRA
Research Groups GR-09 - GRAphics and INtelligent Systems - GRAINS
Description The fusion of Large Language Models (LLMs) with Virtual Reality (VR) holds promise for a significant evolution in user experience. Within this context, exploring the utilization of LLMs for a hands-free locomotion system becomes particularly intriguing. This thesis aims to leverage the advanced natural language processing capabilities of these models to interpret user commands for teleportation, thus enhancing user interaction and immersion by allowing seamless and intuitive navigation without the need for physical controllers.
The primary objective will be to design and implement a framework that integrates LLMs for hands-free teleportation in VR environments. The activities will start with the design of the framework, defining a comprehensive set of teleportation commands and establishing the architecture for integrating LLMs with Unity. The implementation phase will involve developing a prototype, utilizing LLMs to interpret spoken commands and control VR teleportation. The system's effectiveness will be evaluated through user studies, collecting both quantitative and qualitative data to measure accuracy, response times, and overall user experience.
References:
- D. Calandra, F. G. Pratticò, and F. Lamberti, "Comparison of Hands-Free Speech-Based Navigation Techniques for Virtual Reality Training," in 2022 IEEE 21st Mediterranean Electrotechnical Conference (MELECON), Palermo, Italy, 2022, pp. 85-90. doi: 10.1109/MELECON53508.2022.9842994.
- J. A. V. Fernandez, J. J. Lee, S. A. S. Vacca, A. Magana, B. Benes, and V. Popescu, "Hands-Free VR," arXiv preprint arXiv:2402.15083, 2024
See also http://grains.polito.it/work.php
Deadline 15/01/2025
PROPONI LA TUA CANDIDATURA