Real-Time and High-Quality Musical Audio Streaming Over IP

The Internet of Musical Things (IoMusT) is an emerging paradigm that envisions ecosystems of interconnected smart musical instruments and devices dedicated to the production and reception of musical content. Realizing the IoMusT vision requires addressing key challenges in real-time audio streaming over networks, including ultra-low latency, high reliability, and perceptual audio quality. One of the central components of the IoMusT paradigm is represented by Networked Music Performance (NMP) systems, which aim at enabling geographically displaced musicians to play together in a realistic way over the network. Prior to this thesis, there was a lack of comprehensive studies evaluating the ability of fourth-generation (4G), fifth-generation (5G), and transitional 4G/5G networks to support the strict Quality of Service (QoS) requirements of NMP and IoMusT applications in realistic multi-user scenarios. On the device side, a knowledge gap existed in quantitatively comparing the real-time audio performance of different embedded Linux architectures for IoMusT devices. Furthermore, tools and frameworks were needed to systematically evaluate and compare Packet Loss Concealment (PLC) algorithms, which are crucial for maintaining audio quality under inevitable network losses, especially considering the unique challenges posed by musical signals. The fundamental objective of this thesis is to address these research gaps in order to advance the state-of-the-art in real-time audio streaming over Internet Protocol (IP) networks for musical applications. The key research questions investigated include: Can 4G, 5G, and mixed 4G/5G networks meet the demanding latency, reliability, and quality requirements of IoMusT applications in multi-user scenarios?How do different embedded Linux architectures compare in terms of real-time audio performance metrics relevant for IoMusT devices?How can PLC algorithms be systematically evaluated and compared, taking into account the perceptual aspects of musical signals?Can perceptual aspects be taken into account to design better metrics and loss functions for PLC in NMP?To answer these questions, this thesis conducts realistic network simulations, develops opensource tools, and explores novel techniques. The outcomes demonstrate that 5G significantly improves IoMusT support compared to 4G, provide quantitative insights to guide platform selection for IoMusT devices, deliver a modular framework for evaluating PLC methods for NMP systems, and propose perceptually-motivated approaches to enhance concealment algorithm design. Overall, the research presented in this thesis advances the understanding of real-time audio streaming over modern wireless networks and provides concrete tools and techniques to support the realization of the IoMusT vision, enabling new forms of distributed musical experiences and collaborations.