Mode independent interaction pattern design

This paper proposes an approach for cross-modal user interface design using interaction patterns. A meta-domain is introduced in which the user interface can be designed without the determination of its realisation in a particular interaction domain. When fully described in the meta-domain, the user interface can be realised in different domains (visual, auditory, tactile etc) exploiting the strengths of each different mode to its maximum. The paper proposes a set of interaction patterns along with visual and auditory realisations. To prove the concept, a prototype was designed for a real-world application, Microsoft Explorer. An auditory display prototype was developed making extensive use of virtual audio environments for non-visual visualisation of objects in the Explorer interface, taking advantage of a spatial layout. The prototype was evaluated with visually impaired, blind and sighted computer users.