Our ability to develop robust multimodal systems will depend on knowledge of the natural integration patterns that typify people’s combined use of different input modes. To provide a foundadon for theory and design, the present research analyzed multimodal interaction while people spoke and wrote to a simulated dynamic map system. Task analysis revealed that multimodal interaction occurred most frequently during spatial location commands, and with intermediate tiequency during selection commands. In addition, microanalysis of input signals identified sequential, simultaneous, point-and-speak, and compound integration patterns, as well as data on the temporal precedence of modes and on inter-modal lags. In synchronizing input streams, the temporal pecedence of writing over speech was a major theme, with pen input conveying location information first in a sentence. Linguistic analysis also revealed that the spoken and written modes consistently supplied complementary semantic information, rather than redundant. One long-term goal of this research is the development of predictive models of natural modality integration to guide the design of emerging multimodal architectures.
Scheda prodotto non validato
I dati visualizzati non sono stati ancora sottoposti a validazione formale da parte dello Staff di IRIS, ma sono stati ugualmente trasmessi al Sito Docente Cineca (Loginmiur).
|Titolo:||Integration and synchronization of input modes during multimodal human-computer interaction|
|Autori:||S., Oviatt; De Angeli, Antonella; K., Kuhn|
|Titolo del volume contenente il saggio:||Proceedings of the SIGCHI Conference on Human Factors in Computing Systems|
|Luogo di edizione:||New York|
|Anno di pubblicazione:||1997|
|Appare nelle tipologie:||04.1 Saggio in atti di convegno (Paper in proceedings)|