In this paper, we study perception of intensity incongruence between auditory and visual modalities of synthesized expressions of laughter. In particular, we investigate whether incongruent expressions are perceived as 1) regulated, and 2) unsuccessful in terms of animation synthesis. For this purpose, we conducted a perceptive study with the use of a virtual agent. Congruent and incongruent multimodal expressions of laughter were synthesized from natural audiovisual laughter episodes, using machine learning algorithms. Next, the intensity of facial expressions and body movements were systematically manipulated to check whether the resulting incongruent expressions are perceived differently compared to the corresponding congruent expressions. Results show that 1) intensity incongruence lowers the perception of believability and plausibility, and 2) the incongruent laughter expressions displaying high intensity in the audio modality and low intensity in the body movement and facial expression are perceived as more fake than the corresponding congruent expressions. Such results have implications for both animation synthesis as well as expression regulation research.

Perception of intensity incongruence in synthesized multimodal expressions of laughter / Niewiadomski, R; Ding, Y; Mancini, M; Pelachaud, C; Volpe, G; Camurri, A. - (2015), pp. 684-690. (Intervento presentato al convegno ACII2015 tenutosi a China nel 2015) [10.1109/ACII.2015.7344643].

Perception of intensity incongruence in synthesized multimodal expressions of laughter

Niewiadomski, R;
2015-01-01

Abstract

In this paper, we study perception of intensity incongruence between auditory and visual modalities of synthesized expressions of laughter. In particular, we investigate whether incongruent expressions are perceived as 1) regulated, and 2) unsuccessful in terms of animation synthesis. For this purpose, we conducted a perceptive study with the use of a virtual agent. Congruent and incongruent multimodal expressions of laughter were synthesized from natural audiovisual laughter episodes, using machine learning algorithms. Next, the intensity of facial expressions and body movements were systematically manipulated to check whether the resulting incongruent expressions are perceived differently compared to the corresponding congruent expressions. Results show that 1) intensity incongruence lowers the perception of believability and plausibility, and 2) the incongruent laughter expressions displaying high intensity in the audio modality and low intensity in the body movement and facial expression are perceived as more fake than the corresponding congruent expressions. Such results have implications for both animation synthesis as well as expression regulation research.
2015
2015 International Conference on Affective Computing and Intelligent Interaction (Acii)
NEW YORK, NY
IEEE
Niewiadomski, R; Ding, Y; Mancini, M; Pelachaud, C; Volpe, G; Camurri, A
Perception of intensity incongruence in synthesized multimodal expressions of laughter / Niewiadomski, R; Ding, Y; Mancini, M; Pelachaud, C; Volpe, G; Camurri, A. - (2015), pp. 684-690. (Intervento presentato al convegno ACII2015 tenutosi a China nel 2015) [10.1109/ACII.2015.7344643].
File in questo prodotto:
File Dimensione Formato  
ACII15_niewiadomskieral.pdf

Solo gestori archivio

Tipologia: Post-print referato (Refereed author’s manuscript)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 3.12 MB
Formato Adobe PDF
3.12 MB Adobe PDF   Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/278964
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? 5
social impact