Cross-Sensory Perception in Music and Visual Media: A Neurocognitive Approach to Multimedia Counterpoint
MetadataShow full item record
Despite differences in language and methodology, the fields of music theory, multimedia studies, and cognitive neuroscience are exploring many of the same questions. This dissertation seeks to define and describe multimedia counterpoint using parallels between musical counterpoint and multisensory scene analysis. Research in crossmodal and intramodal association, crossmodal modulation, and multisensory integration is used to define the concept of a multimedia voice, to describe contrapuntal relationships between voices, and to explore the perceptual effects of those relationships in a work. The dissertation presents the Crossmodal and Intramodal Association (CIA) model to map relationships between music and visual parameters in three dimensions: type, intensity, and quality. The model is applied in two analyses: the “color crescendo” from Schoenberg’s opera Die glückliche Hand and a comparison of two scenes from Pink Floyd’s film The Wall.