CROSS-MODALITY
Humans readily make connections between sensory experiences, and these are often shared. Get started to learn more!
* * *
What is cross-modality?
Cross-modality refers to the general phenomenon that we tend map experiences in one modality onto another - often so automatically that we don't notice! For example, people reliably map bright light onto high pitch.
What makes you associate one sensory experience with another?
One mechanism of crossmodal association or many?
All these answers are right! Research suggests that our senses connect in many ways, and these aren't mutually exclusive. However, each mechanism may have different consequences for behaviour.
We make crossmodal mappings between pretty much every pair of sensory modalities
-
Sweet tastes are mapped to sounds with more energy in the lower frequencies than sour tastes.
-
Sweet tastes are also matched with red-pink colours, while sour tastes are more yellow and green.
-
Rough surfaces are “seen” as brown, and soft surfaces “look” pink.
-
Still water maps to rounded shapes and sparkling water to angular shapes.
Is it synaesthesia?
Cross-modality is generally different from synaesthesia, where a stimulus in one modality involuntarily evokes a sensory experience in another modality. While the two phenomena are distinct, many researchers believe they are related.
Let's try some associations! Think about what primary colours you associate with basic shapes. How would you do it?
-
For Bauhaus artist Wassily Kandinsky and some of his contemporaries, a triangle was associated with yellow, a square with red, and a circle with blue.
-
Other members of the Bauhaus movement thought a square is blue and a circle is red. This is what Japanese participants think too, which may relate to their flag.
-
German participants seem to make colour-shape associations based on real-world objects: sun → yellow circle, warning signal → red triangle.
Cross-modality & Food
Watch this short video where University of Oxford Professor Charles Spence talks about his pioneering research in crossmodal and multisensory perception.
Eating and drinking are among life's most multisensory experiences.
Coffee tastes better when the machine is quiet, and less sweet if the mug is white. Food tastes less sweet or salty when flying, and ginger biscuits taste more gingery when served from a plate having a rough texture.
There's more ...
Cross-modality is just one way of how our senses interact with each other.
Synaesthesia is another, as is sound symbolism and onomatopoeia, and sensory and inter-sensory metaphors – we talk of sharp minds and sharp cheeses. There’s also memory: for example, many people associate soft, repetitive sounds with the sea. It seems automatic, but it is optional: a person that has never seen the sea will probably not have that association.
CROSS-MODALITY
We hope you enjoyed learning more about crossmodal associations!
* * *