Interface features affecting deixis production: A simulation study

This paper examines gestural deixis production in a multimodal (natural language and pointing) interaction. Users (N=30) were required to fill in 4 forms by interacting with a simulated system. Labelling of Entry Fields (LEF, Complete-Partial; between-subject) and Feedback (Present-Absent within-subject) were manipulated in a 2*2 design. Results indicate that deixis occurred more frequently in the Partial-LEF condition. Specific input device constraints influenced intermodal integration: pointing was performed after the end of writing, although feedback presence increased modality shifts. Simulation offered a valuable tool for the development of multimodal interaction models. Referring to objects in space is a typical communication action. Referent identification can be achieved by several means: directly naming or describing the target; linguistic devices like anaphora and metonymia; specifying target location. Target location can be specified in a speaker-independent mode, by reference to external entities, or in a deictic mode, by reference to speaker position. Space deixis is conveyed by deictic terms, mainly.