Skip to main content

Sounds from seeing silent motion: Are they real, who hears them, and what looks loudest?, Dr Elliot Freeman, City University

When: —
Venue: Birkbeck Main Building, Malet Street

No booking required

Sounds from seeing silent motion: Are they real, who hears them, and what looks loudest?

I have been investigating a little-known but highly prevalent form of synaesthesia, which I myself experience all the time: I can hear things I see moving. I get an auditory 'shhh' sensation when I see people walking, shop displays flashing, in fact anything that moves or suddenly changes in my visual field. I have used psychophysics, brain stimulation, and a large-scale survey to explore the following questions, and here are some short answers.

(1) Do others have this ability too? Yes, up to 1 in 5 people admit to having visual-evoked auditory sensations. They perform better when using these sounds to remember sequences of flashes, but perform worse when sounds evoked by flashes interfere with detection of real sounds.

(2) What looks loudest? In an internet survey, >4000 individuals rated the sounds evoked by silent movies. Ratings were highest for movies depicting predictable sounds, e.g. collisions, but independently for movies containing abstract motion energy, regardless of meaning. This suggests both high-level and low-level routes for mapping visual motion to sound.

(3) Does it correlate with other traits or abilities? Among other things, ratings correlated with frequency of musical imagery, and the opposite phenomenon of auditory-evoked flashes (while dozing). This suggests increased sensory excitability (or reduced inhibition) as a possible aetiology.

(4) Does my brain work differently? A brain stimulation study suggests that people who hear flashes use their auditory and visual cortices in cooperation to perform a sequence discrimination task, while for others vision and audition compete.

Altogether my results that there is a spectrum of sensitivity to visual-evoked auditory sensations, with high prevalence, and it may depend on systemic physiological variables such as inhibition, rather than anatomical differences often proposed for other rarer kinds of synaesthesia.