This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by Ferris, T. K. Articles by Sarter, N. B. Search for Related Content PubMed Articles by Ferris, T. K. Articles by Sarter, N. B. Social Bookmarking                         What's this?

Cross-Modal Links Among Vision, Audition, and Touch in Complex Environments

University of Michigan, Ann Arbor, Michigan

Nadine B. Sarter

University of Michigan, Ann Arbor, Michigan, sarter{at}umich.edu

Objectives: This study sought to determine whether performance effects of crossmodal spatial links that were observed in earlier laboratory studies scale to more complex environments and need to be considered in multimodal interface design. It also revisits the unresolved issue of cross-modal cuing asymmetries. Background: Previous laboratory studies employing simple cues, tasks, and/or targets have demonstrated that the efficiency of processing visual, auditory, and tactile stimuli is affected by the modality, lateralization, and timing of surrounding cues. Very few studies have investigated these cross-modal constraints in the context of more complex environments to determine whether they scale and how complexity affects the nature of crossmodal cuing asymmetries. Method: A microworld simulation of battlefield operations with a complex task set and meaningful visual, auditory, and tactile stimuli was used to investigate cuing effects for all cross-modal pairings. Results: Significant asymmetric performance effects of cross-modal spatial links were observed. Auditory cues shortened response latencies for collocated visual targets but visual cues did not do the same for collocated auditory targets. Responses to contralateral (rather than ipsilateral) targets were faster for tactually cued auditory targets and each visual-tactile cue-target combination, suggesting an inhibition-of-return effect. Conclusions: The spatial relationships between multimodal cues and targets significantly affect target response times in complex environments. The performance effects of cross-modal links and the observed cross-modal cuing asymmetries need to be examined in more detail and considered in future interface design. Application: The findings from this study have implications for the design of multimodal and adaptive interfaces and for supporting attention management in complex, data-rich domains.

Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 50, No. 1, 17-26 (2008)
DOI: 10.1518/001872008X250566


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				C. Ho and C. Spence Using Peripersonal Warning Signals to Orient a Driver's Gaze Human Factors: The Journal of the Human Factors and Ergonomics Society, August 1, 2009; 51(4): 539 - 556. [Abstract] [PDF]

				L. A. Jones and N. B. Sarter Tactile Displays: Guidance for Their Design and Application Human Factors: The Journal of the Human Factors and Ergonomics Society, February 1, 2008; 50(1): 90 - 111. [Abstract] [PDF]