This paper examines gestural deixis production in a multimodal (natural language and pointing) interaction. Users (N=30) were required to fill in 4 forms by interacting with a simulated system. Labelling of Entry Fields (LEF, Complete-Partial; between-subject) and Feedback (Present-Absent within-subject) were manipulated in a 2*2 design. Results indicate that deixis occurred more frequently in the Partial-LEF condition. Specific input device constraints influenced intermodal integration: pointing was performed after the end of writing, although feedback presence increased modality shifts. Simulation offered a valuable tool for the development of multimodal interaction models. Referring to objects in space is a typical communication action. Referent identification can be achieved by several means: directly naming or describing the target; linguistic devices like anaphora and metonymia; specifying target location. Target location can be specified in a speaker-independent mode, by reference to external entities, or in a deictic mode, by reference to speaker position. Space deixis is conveyed by deictic terms, mainly.

Interface features affecting deixis production: A simulation study

De Angeli, Antonella;
1996-01-01

Abstract

This paper examines gestural deixis production in a multimodal (natural language and pointing) interaction. Users (N=30) were required to fill in 4 forms by interacting with a simulated system. Labelling of Entry Fields (LEF, Complete-Partial; between-subject) and Feedback (Present-Absent within-subject) were manipulated in a 2*2 design. Results indicate that deixis occurred more frequently in the Partial-LEF condition. Specific input device constraints influenced intermodal integration: pointing was performed after the end of writing, although feedback presence increased modality shifts. Simulation offered a valuable tool for the development of multimodal interaction models. Referring to objects in space is a typical communication action. Referent identification can be achieved by several means: directly naming or describing the target; linguistic devices like anaphora and metonymia; specifying target location. Target location can be specified in a speaker-independent mode, by reference to external entities, or in a deictic mode, by reference to speaker position. Space deixis is conveyed by deictic terms, mainly.
1996
Proceedings of the ICSLP Workshop on the Integration of Gesture in Language and Speech
Piscataway, N.J.
IEEE
De Angeli, Antonella; D., Petrelli; W., Gerbino
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/90291
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact