| KEYWORD |
Design and implementation of a music visualization interface with haptic feedback for Deaf and Hard of Hearing (DHH) music students
keywords DEAFBLINDNESS, HAPTIC DEVICES, MUSIC, VISUALIZATION
Reference persons ANDREA BIANCO, CRISTINA EMMA MARGHERITA ROTTONDI
Research Groups Telecommunication Networks Group
Description Musical education of Deaf and Hard of Hearing (DHH) students is particularly challenging due to the essential involvement of auditory perception. For such categories of subjects, the adoption of extra-auditory cues is pivotal to support conveyance of music perception. Indeed, the process of hearing involves a multiplicity of perceptual modes, even beyond merely aural functions.
The goal of this project is the design and implementation of a music visualization interface enhanced with haptic feedback, leveraging a TanvasTouch device, which enables a customized design of tactile feedback localized to the user's fingertip without the need for surface vibration, by exploiting electroadhesion-based haptic technology. This way, music listening can be enhanced with both visual and tactile stimuli, which should be representative of some characteristics of the audio content being reproduced (e.g., timbre, pitch, tone, rhythm…). To this aim, signal processing techniques will be exploited for real-time spectral analysis of the audio signal (e.g., the extraction of mel-frequency cepstral coefficients), to extract meaningful features to control the visual/haptic representation rendering. The definition of the specific use-cases will benefit of the collaboration with experts in inclusive musical education: prof. Piera Bagnus (music conservatory of Como) and prof. Chiara Nicora (music conservatory of Novara).
Required skills good programming skills (C/C++), knowledge of signal processing techniques
Deadline 25/08/2024
PROPONI LA TUA CANDIDATURA