Crossmodal sensory perception

In a recent post I wrote about a beer tasting session with Pete Brown at the Green Man Festival, in which he talked about how the music we hear may influence our perception of flavour. In the UK, much of the influential research in this area comes from the Crossmodal Research Laboratory in the Department of Experimental Psychology at the University of Oxford.

The Laboratory was founded in 1997 and the team there study the integration of information across the different sensory modalities (hearing, vision, touch, taste, and smell). This is an area of research that is changing the way we view our senses. Traditionally, vision, hearing, touch, smell, and taste have been studied in isolation. However, recent research has shown sensory processing within a single sense is modulated by information from the other senses.

One area of interest to the Oxford laboratory is how our understanding of multisensory perception can be used by the food industry to improve the perception of foods and drinks. Professor Charles Spence, who heads the Crossmodal Research Laboratory, was interviewed for a recent Guardian article by Amy Fleming (link below). She notes that much of the lab’s work is funded by Unilever, while Prof Spence sits on the scientific advisory board of PepsiCo.

Therefore, this is an area of research that people should be aware of, in order to make informed purchasing decisions. For example, it has been found that ‘crunchy’ correlates with fresh, so food manufactures are making crisps and so forth that sound crunchy even though they are not so fresh. The information also informs product design and marketing. More beneficially (for us), food manufacturers are using crossmodal perception research findings to gradually reduce the salt and sugar content of foods (to meet Government guidelines). One of the early results of crossmodal perception, for instance, was that product colour affects perceptions of flavour and sweetness.

Charles Spence has written a book with his colleague Betina Piqueras-Fiszman called ‘The Perfect Meal’ (published next week in the UK), which presents the laboratory's recent findings on crossmodal perception for general readers. It is structured around the dining experience in a restaurant. It looks at the factors that influence flavour perception, including visual, tactile, cognitive and aural stimuli. For example, the subtle effects of the colour of the plates, the shape of the glass, the names of dishes, and the background music. So, for instance, whisky tastes better in a “woody” room, while food plated to resemble a work of art tastes better than when it is indifferently put on a plate.

A signature dish for crossmodal perception is the ‘Sound of the Sea’ served at Heston Blumenthal’s Fat Duck restaurant. This seafood and fish dish plays on taste, aroma, sound and the overall nostalgic experience of the seaside. It is served with an iPod (inside a conch shell from which headphones emerge) playing seaside sounds, specifically waves crashing on a beach. The flat glass plate on which the food is served is placed on top of a rectangular box containing a bed of sand, while edible sand (made from tapioca), pieces of edible seaweed, and a wave of salty sea foam (vegetable and seaweed broth) surround the fish and seafood. Charles Spence collaborated with Heston Blumenthal on the creation of this dish, which is based on work concerning sound and flavour done in the Oxford laboratory.

So, if your waiter comes across all Derren Brown, there may be crossmodal perception at play. When it comes to food advertising, packaging and the perception of processed food products, however, you (the consumer) are not supposed to be aware of the psychology being applied. So, now is a good time to read up on what’s being cooked up in the lab (links below).

For instance, a recent paper from the laboratory found that the perception of green, yellow, and orange drinks was influenced by the shape of the glass in which the drink was presented, and the authors advised that for advertising and product packaging the appropriateness of the glassware be carefully considered. Another paper confirmed that fruit juices were considered 'sweet and low in sourness' were consistently matched with rounder shapes and speech sounds, and lower-pitched sounds, and were generally liked more; meanwhile, those juices that were rated as tasting 'sour' were consistently matched with angular shapes, sharper speech sounds, and sounds with a higher pitch, and were liked less.

The Oxford team have also found that the sounds of a food product’s name are generally associated with both sensory and conceptual attributes. This forms part of a wider area of study, looking at how retail spaces can provide non-verbal cues to improve sales.

Further reading:

Pete Brown on beer and music
http://sfnottingham.blogspot.co.uk/2014/08/green-man-2014-welsh-beer.html

Crossmodal Research Laboratory, Department of Experimental Psychology, Oxford
http://www.psy.ox.ac.uk/research/crossmodal-research-laboratory

‘Charles Spence: The food scientist changing the way we eat’, by Amy Fleming
http://www.theguardian.com/lifeandstyle/2014/sep/24/charles-spence-food-scientist-changing-eat-flavour

Make your own 'Sound of the Sea'
http://www.umamiinfo.com/2011/03/expert-recipe---the-sound-of-the-sea.php

‘Beverage perception and consumption: The influence of the container on the perception of the contents.’ Wan and Spence, 2015 (in press). Food Quality and Preference 39: 131-140.

‘Do you say it like you eat it? The sound symbolism of food names and its role in the multisensory product experience.’ Favalli et al., 2013. Food Research International 54: 760-791.

‘Retail atmospherics and in-store non-verbal cues: an introduction.’ Grewel et al., 2014. Psychology and Marketing 31: 469-471.