Real-time Musical Haptics with Ultra-wideband: A Study on Latency, Reliability, and Perception

Ultra-wideband (UWB) radios are popular for accurate distance estimation between devices. However, UWB also offers low-power, fast, reliable wireless communication. We exploit it here in a real-time musical haptics system for live performances: a wearable, wirelessly activated via UWB by the performer's instrument, augments the audience musical experience with a tactile sensory layer. Two challenges are crucial to the experience quality: i) communication must be reliable, to prevent corruption of tactile signals, and ii) these must reach the audience synchronously with the instrument sounds. We perform micro-benchmarks of UWB links alone in a controlled setup, showing that the haptic signal can be delivered reliably over UWB before the instrument sound, thus enabling proper compensation delays to perfectly realign sound and tactile vibration. We confirm this holds on the end-to-end system including haptic components by characterizing four proof-of-concept prototypes combining different...

Ultra-wideband (UWB) radios are popular for accurate distance estimation between devices. However, UWB also offers low-power, fast, reliable wireless communication. We exploit it here in a real-time musical haptics system for live performances: a wearable, wirelessly activated via UWB by the performer's instrument, augments the audience musical experience with a tactile sensory layer. Two challenges are crucial to the experience quality: i) communication must be reliable, to prevent corruption of tactile signals, and ii) these must reach the audience synchronously with the instrument sounds. We perform micro-benchmarks of UWB links alone in a controlled setup, showing that the haptic signal can be delivered reliably over UWB before the instrument sound, thus enabling proper compensation delays to perfectly realign sound and tactile vibration. We confirm this holds on the end-to-end system including haptic components by characterizing four proof-of-concept prototypes combining different UWB-enabled instruments and wearables. Finally, we reconcile these objective measures with subjective ones via a user study focusing on perception, yielding very positive outcomes. Together, these results confirm the potential of UWB-based musical haptics for enhancing the audience experience at live performances in ways hitherto unexplored.