Immersive networked music performance systems: identifying latency factors

Music playing with binaural systems has thus far received remarkably little attention by the research community. Such an activity is particularly relevant for the case of networked music performances (NMPs), where geographically displaced musicians play together over a telecommunication network. Recent findings obtained in simulated settings have shown the preference of musicians for spatialized listening during collaborative playing using headphones, as opposed to listening with conventional stereophonic systems. This result has highlighted the need to enhance current NMP systems with the spatial rendering of the acoustic scene, leading to the development of immersive networked music performance (INMP) systems. A crucial aspect that needs to be addressed is the contribution of the spatial audio system to the overall latency of the audio processing and transmission chain between the network nodes. In this paper, we provide an overview of the INMP topic and identify the latency contributions of the components of an INMP system. We then relate such components to state-of-the-art hardware and software systems. Based on our analysis, we draw concluding remarks and discuss the open challenges for progressing the INMP field.