Audiovisual Colocation

Audiovisual Colocation is a new kind of immersive media technique in which sounds are mapped to the same spatial locations as their visual sources in the screen. This course covers the general principles of audiovisual colocation, its many use cases, and also shows how to build your own colocative audiovisual media system.

Audiovisual colocation places sound and image cues in close spatial proximity to each other, so that the displayed media functions more analogously to natural perception. Most audiovisual systems spatially dislocate sound and image information sources, by placing visual information within the screen, and audio information external to the screen through headphones or speakers. Audiovisual colocation enhances the object-event correlations in mediated content, by vibrating the screen so that sounds emanate directly from the associated imagery via software mapping to an array of audio transducers on the screen’s backside. With virtual or augmented reality-based media, the system is reconfigured as a modular wall panelling system. This presentation will cover the main aspects of audiovisual colocation, including: 1) an overview of the relevant historical and aesthetic context for spatial sound; 2) a quick review of the patent filing; 3) empirical perceptual experiments demonstrating the phenomenological and affordance capacities of the prototype display; 4) the connection between new media poetics and cognition for colocative media; and 5) a discussion of commercial applications and use cases.