Real-Time Audio Pattern Detection for Smart Musical Instruments

This paper presents an algorithm for real-time polyphonic audio pattern detection designed to be integrated into smart musical instruments. Such instruments will enable musicians to use predefined musical patterns as triggers for external peripherals, such as stage lights, haptic wearables, and mixed reality headsets, during live performances. The paper first introduces a data set of polyphonic patterns containing musical patterns with expressive variations recorded by 20 musicians. Secondly, it presents an approach to classify audio segments in real time, consisting of a recurrent neural network architecture. Thirdly, it compares the proposed method against a traditional dynamic time warping approach as a benchmark. Results show that, although the benchmark achieves higher precision (0.8 versus 0.74), the proposed method yields a better overall performance, with an F1 score of 0.76 compared with 0.65. Most importantly, the proposed method exhibits a lower computational inference time (14 ms versus 1038 ms) and reduced CPU usage (31.36% versus 48.9%) when deployed on a Raspberry Pi 4. The low inference time meets the latency requirements necessary for real-time musical applications, and the system demonstrates the feasibility of embedded intelligence in musical instruments that can recognize polyphonic patterns with expressive variations during live performances.