Harmonizing the Musical Metaverse: unveiling needs, tools, and challenges from experts’ point of view

The Musical Metaverse (MM) represents an innovative frontier for the field of New Interfaces for Musical Expression (NIME). The MM holds the potential to redefine areas such as musical composition and performance via immersive environments based on technologically mediated social interactions. Despite substantial research on single-user immersive systems, the intersection of NIME and the MM remains largely unexplored. In this paper, we systematically explore this domain by examining previous and current approaches, alongside conducting interviews with eleven experts who have created multi-user immersive musical environments and authored publications on this topic. The goal is to map such an uncharted territory by collecting valuable insights and leveraging the perspective of experts to provide an understanding of the potentials and challenges inherent in creating immersive social environments for musical activities. Our results reveal that existing multi-user immersive environments make use of diverse implementation approaches but face challenges due to the absence of standardized technology stacks, particularly in networking and data synchronization.