The dialogue experience with conversational agents can be greatly enhanced with multimodal and immersive interactions in virtual reality. In this work, we present an open-source architecture with the goal of simplifying the development of conversational agents operating in virtual environments. The architecture offers the possibility of plugging in conversational agents of different domains and adding custom or cloud-based Speech-To-Text and Text-To-Speech models to make the interaction voice-based. Using this architecture, we present two conversational prototypes operating in the digital health domain developed in Unity for both non-immersive displays and VR headsets. The architecture is publicly available on GitHub
Let's Give a Voice to Conversational Agents in Virtual Reality / Yin, Michele; Roccabruna, Gabriel; Azad, Abhinav; Riccardi, Giuseppe. - (2023), pp. 5247-5248. (Intervento presentato al convegno Interspeech 2023 tenutosi a Dublin, Ireland nel 20th August-24th August 2022).
Let's Give a Voice to Conversational Agents in Virtual Reality
Gabriel RoccabrunaSecondo
;Abhinav AzadPenultimo
;Giuseppe RiccardiUltimo
2023-01-01
Abstract
The dialogue experience with conversational agents can be greatly enhanced with multimodal and immersive interactions in virtual reality. In this work, we present an open-source architecture with the goal of simplifying the development of conversational agents operating in virtual environments. The architecture offers the possibility of plugging in conversational agents of different domains and adding custom or cloud-based Speech-To-Text and Text-To-Speech models to make the interaction voice-based. Using this architecture, we present two conversational prototypes operating in the digital health domain developed in Unity for both non-immersive displays and VR headsets. The architecture is publicly available on GitHubFile | Dimensione | Formato | |
---|---|---|---|
yin23b_interspeech.pdf
accesso aperto
Tipologia:
Versione editoriale (Publisher’s layout)
Licenza:
Tutti i diritti riservati (All rights reserved)
Dimensione
541.3 kB
Formato
Adobe PDF
|
541.3 kB | Adobe PDF | Visualizza/Apri |
I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione