Multisensory Music Listening in Affective Virtual Environments

Previous research has shown that the vibrotactile enhancement of music signals can be effective in creating intense musical experiences. Virtual Reality is emerging as a new medium with which experiencing music and inducing emotions, but the study of the effects of tactile enrichments of music in virtual environments (VEs) has been largely overlooked thus far. It is unknown whether the same results found for real environments hold for VEs too, and what is the role played by vibrations in altering the emotion, perception and behavior of listeners in such spaces. To bridge these gaps, we created 12 affective musical VEs accompanying as many musical pieces and assessed the impact of adding synchronous vibrations. Results showed that the VEs involving the vibrations received significantly higher ratings of valence, arousal and presence compared to their audio-visual counterparts. The majority of participants preferred the condition with the vibrations. Cybersickness was not reduced in presence of vibrations, but it was found to be significantly higher for females than males. Although most participants reported that they were induced to move to a greater extent in presence of vibrations, the recorded motion of their head and hands did not increase when vibrations were provided.