Posts

Hallucinations and Why They're Key to Effective Immersion

Hallucinations and Why They’re Key to Effective Immersion

/in , , /by

Hallucinations are both much more common and benign than you may expect. And the art of engendering them in your audience is the key to effective immersion.

Hallucinations and Why They're Key to Effective Immersion

Hallucinations are both more common and benign than you might expect,

Let me tell you about some weird things that have happened when I’ve been listening to binaural (3D) audio.

Some years back, I was listening to the first cut of our ‘Walk with Lady Mary’ audio trail for Conwy Council and the Forestry Commission. One sequence involved us journeying back to the 16th century to meet Lady Mary’s Tudor ancestor, who comes barrelling out of the woods on his trusty steed. On the soundtrack, a horse neighed and, from the corner of my eye out of my office window, right on the edge of my peripheral vision, I saw a horse trotting along the road outside. When I turned, it wasn’t a horse at all but a car. But I was sure I’d seen a horse…

Another time, I was listening to an underwater track for a piece of scuba diving VR. Now, I haven’t spent much under the sea but I did do some scuba diving when I was a backpacking around Australia in the early 90s. That track alone created a strange sensory reaction – my skin went cold and goosebumps started to rise on my arms. The track had clearly surfaced that 30-year-old memory of diving off the Great Barrier Reef – but not the visual part of it, just the feeling of being under the sea.

I had a similar sensory reaction when I was sitting in my office listening to a soundtrack recorded on a tropical lake in South America. It was a cold, winter’s day and I didn’t have the heating on (it was the early days of PastPorte and I had to mind the pennies…) Even though I was wearing two pairs of socks, my toes were getting a little stiff with cold. But I was listening to this sound track and the strangest thing happened – my toes began to warm up, as if my brain believed I was really in the tropics.

How do you explain all of this? Well a recent article in The Atlantic on the nature of hallucinations may go some way to doing so.

As soon as I say ‘hallucinations’ that will make you think of people seeing things that aren’t there. That’s a partially correct definition of the word but a hallucination is actually any sensory perception of something that isn’t there. Ever felt your mobile vibrate in your pocket, taken it out and found there’s no reason why it would have done so? Or felt a phantom rain drop on a cloudy day when it wasn’t raining? Or thought you heard someone in your house calling your name, checked and found that nobody had? Well, all of those occurrences are forms of hallucinations. And before you question your sanity (or I question mine) you should know that hallucinations are remarkably common things and for the most part, entirely benign and not the result of any deep seated psychosis.

So what causes us to perceive things that aren’t actually there? Well, the experts think it may be down to how our brain perceives the world around us.

Corlett and Powers, both from Yale School of Medicine, have advanced the theory of ‘predictive coding.’ “When we go about the world, we’re not just passively perceiving sensory inputs through our eyes and ears,” Corlett says. “We actually build a model in our minds of what we expect to be present.”
It’s Corlett and Powers theory that the brain doesn’t always get this model right – it has the capacity of overpredict – to be fooled, in essence. I can expect something to be there which isn’t, and this expectation can be so strong that the brain creates something that isn’t there. A hallucination.

I should say at this stage that the predictive modelling theory is speculative and isn’t yet accepted psychological fact but it goes some way to explain the false sensory reactions that I, and others that have listened to binaural audio have experienced. Those hyper realistic three dimensional sounds have made our bodies build a false model of our surroundings and elicited some quite remarkable sensory hallucinations.

Why particularly does sound have the power to do this? We think it’s something to do with our sensory hierarchy. We’re so visually dominant – so programmed, from our earliest days of roaming the Africa plains, to look out for danger – that it’s easier to fool those other senses that are well down the pecking order. And clearly, this ability to fool the brain aurally can create these sorts of false sensory reactions – benign hallucinations which have the ability transport people, immerse them further in our imagined-worlds and create truly deep-seated and memorable multi-sensory experiences.

We think we’ve barely scratched the surface of the potential of audio – and other, second tier senses such as smell – in cultural attractions, particularly inside heritage attractions where there’s already an authentic visual backdrop to work with. The secret to effectively to transporting people to new times and places isn’t necessarily through their eyes – where they’ll likely see the flaws in your illusion – but through those other, secondary senses that are easier to fool. And the theory of predictive modelling would suggest that you only have to create part of the illusion – then you can rely on their imagination to do the rest…

Our Guide to Immersive Storytelling in Museums

/in , , , /by

Immersive storytelling is an exciting and memorable way to bring history to life but it can be a daunting challenge. We share some tips for getting it right.

Bringing the past to life is one of the most memorable and exciting ways of interpreting history. And with all the immersive tech now available, helping visitors to feel, smell and breathe the air of historical events is more and more possible.

Binaural audio – with the right concept, script and actors – can trick the brain into believing you have time travelled and are experiencing a new reality – even though the sights around you haven’t changed. Virtual reality can take you anywhere, to be anything, moving through time and space. Haptics bring an extra sensory dimension, smells can be hugely evocative and all of it can be combined to create spine-chillingly powerful effects.

It’s a very exciting prospect. But perhaps a little overwhelming.

There are so many possibilities and so much choice of kit it’s easy to be blinded by the tech and lose sight of what it is you want to achieve and what suits your space.

There are four key mantras that we at PastPorte tend to work by – and live by. They work for us. And we think they’ll work for you too.

1. Keep it simple.

Rarely is the success of your immersive experience going to depend on how much you’ve spent on the kit, how many microphones you’ve used, how many speakers you’ve plugged up or how many projectors you’ve got going.

It can be hard to stick to keeping it simple. Especially when seduced by constant tech innovations.

One of Pastporte’s most successful installations used just two simple resonant speakers, purchased in a domestic high street store, loaded with a pared-back atmospheric soundscape.

Also we try and use silence. Lots of it. The brain gets overstimulated with a constant assault from sophisticated, exciting sounds and starts to regard them as normal. Binaural audio works best if it’s a constant surprise. But to achieve that, the brain almost needs to have forgotten what it sounds like. So, in any story, we build in sections that are silent or low key to create contrast and keep up the surprises. We think that makes an experience more immersive.

2. If there’s something to see, keep your visitors eyes up.

Encouraging visitors to look at a screen and use a multi-media guide is great if there’s nothing to see at your attraction.

But most museums and attractions are all about feasting your eyes on what’s around you – aren’t they? So why would you want to distract from that by asking your visitors to look away from your visual treats and instead, get their eye candy from a screen (especially for kids when their parents are likely to feel they’re too screen-dependent already) ?

If there’s something to look at, let them look at it but augment their experience in other ways. Here too binaural audio can build your experience into a full immersive, time-travelling wonder. Especially if it’s the right audio.

3. Build on your strengths.

Make your artefacts the centre of your immersive story. Turning an artefact into an interactive, immersive exhibit is a neat trick – one we’ve regularly used at PastPorte. Using a gorgeous, if knackered, bellows camera, installed headphones and an AV screen, we transported viewers to Antarctica, 1915. The leather bellows (with associated musty smell), mahogany and brass housing all started to do our work for us – before the viewer had even begun our immersive time travel experience.

A visual, tangible object that can tee up the brain for what is to come adds extra impact. And the added bonus of using an artefact that already feels part of the period and is appropriate in the setting means the whole experience is more integrated yet also more surprising.

4. Content is king.

Whatever approach you choose, and however you choose to deliver it, it’s the content that will make your experience stand out. However great the tech set-up, if the content doesn’t tell the story well, no-one will remember your experience.

Spend time devising the right story and the right approach to telling that story, and constantly review both as you develop the experience. It can be easy to lose sight of the most important thing in amongst the glitz of all that fabulous, shiny kit.

But it’s the content that will live on when the latest tech fashions have come and gone.

The Museum & Heritage Attraction Guide to Immersive Tech: Binaural Sound

/in /by

We use a number of technologies and techniques to immerse visitors in our clients’ stories. Binaural sound is one of the simplest, but most effective.

Binaural sound is so immersive, your visitors will be agape.

Binaural sound is so immersive, your visitors will be agape.

I’m walking through the most English of English gardens – all manicured lawns and artistically sculpted hedges. What I’m hearing, though, isn’t quite so quintessentially English – it’s the distinctly tropical sound of humming cicadas. But despite the fact I know that this is clearly the ‘wrong soundtrack’, my brain is telling me that this distinctive hum must be coming from the garden around me. Such is the power of binaural sound.

Binaural sound is an immersive sound technology delivered via headphones. If you know how to use it properly, you can trick the user’s brain into hearing sounds all around them – to the left, right, behind, in front, above, at different distances and any combination of those directions!

I had the experience described above when testing our audio guide for the National Trust’s Hidcote Manor Garden. The audio brings the garden’s creator, Lawrence Johnston, back to life – binaurally – to take you on a tour of his garden and his life. One sequence takes the listener back to one of Johnston’s 1920s plant hunting expeditions on the South African veldt. My ears were telling me I was in Johnston’s beautiful arts and craft garden. My ears were telling me that I could hear cicadas. And, because of the nature of binaural, my brains was telling me that those cicadas must be in Johnston’s garden.

Our 'Walk with Lawrence Johnston' experience, created for Hidcote Manor Garden.

Binaural sound is so immersive, your visitors will be agape.

Binaural is so effective because, unlike stereo, it’s recorded in exactly the same way the brain hears. Think about how you hear a sound – a sound to your right will be heard by your right ear fractionally before your left ear, and the sound you hear in your left ear will be distorted a little by the fact that the sound is passing through your head en route. You only have two ears (a safe assumption I hope) so your brain can’t properly triangulate. Therefore, it has to work out from these fractional differences, and other contextual clues, exactly where the sound is coming from. If you understand how the brain works, you can ‘trick’ it into hearing sounds in all sorts of places.

For example, one of the common misconceptions about binaural is you can’t make the listener hear sounds in front of them. This is because your sight is a large part of your hearing (bear with me). The brain can’t locate a sound properly with only two sound sources, so your first instinct is to look for the thing that could be making that sound. Once the likely culprit is spotted – for example, a mobile phone when you can hear a ringing – your brain will allocate the sound to this object.

You can use this knowledge to make binaural even more effective if you understand the location your listener is standing in when they listen to it. In our ‘Walk with Lawrence Johnston’, the distant bleating of sheep on the soundtrack coincides with the listener overlooking a distant field of sheep. When I was testing the experience, I couldn’t tell if the bleating was ‘real’ or was part of the soundtrack. Certainly my brain was telling me it must be coming from the sheep – and I was ‘hearing’ that sound coming from a field slap bang in front of me.

Another clue your brain looks for is context. If, by experience, the brain knows a sound comes from a certain place, then this element in a binaural soundtrack will be heard in this place. So the sound of a plane will be heard high above your head, the close yapping of little dogs will be heard down low by your feet, the tweeting of birds will be heard in any tree that is in vision.

The final thing to note about binaural is to do with the nature of sound vs. visuals in our sense hierarchy. Dinah and I have found many virtual reality experiences disappointing partly because we’re so visually dominant, our brains often ‘see through’ the trickery. Because sound is a ‘secondary sense’, it’s easier to fool the brain. We like to say that sound is ‘hardwired’ into the imagination – get it right and it really can be transportative. It’s one of the reasons that binaural sound is so effective. When I first heard those cicadas, my brain said ‘wrong’ but after a matter of seconds it had switched back to focusing on what I was seeing, rather than what I was hearing, and the cicadas became the ‘actual’ soundtrack to the garden.

How can you use binaural sound? Well, you can use it to transport your visitors to wherever and whenever you fancy. We’ve used it to transport listeners from a 21st century shopping centre to a 19th century factory floor, back 300 years in a Methodist chapel to eavesdrop in on the conversations of two of the movement’s founding fathers, and to pitch them into the midst of a medieval battle. But the possibilities are endless.

Imagine for a few moments how binaural sound could enhance a visit to a castle. Other senses are stimulated – visitors can see the walls, towers, moat and courtyard, smell the distinct damp limey odour of the interiors and touch the cold stone walls. Binaural sound – the babble of the tradesman in courtyard, the crackling fire and bawdy banter of the great hall, the low moans from the dungeons – would be all that you’d need to transport your visitors back 1000 years.

Or what about creating a binaural audio tour of one of your experts taking visitors around your gallery? Not only would the commentary be natural – a stark contrast to the stilted scripted audio guides that curse many attractions – but the binaural would make the listener feel they were actually part of that group and that the expert was with them, guiding them around and sharing his knowledge in ‘real time’. How much richer an experience for your visitors would that be?

Whetted your appetite to try some binaural for yourself? Well, fish out your headphones and try one of the sample below and let us know what you think via the comments below.

And if you’d like to know how binaural, or any of the other immersive techniques we’ve described in this blog, could enhance your museum, gallery or attraction, just drop us a line.

Why Multimedia Guides are Often Not the Answer

/in , /by

Multimedia guides are all the rage.  But that doesn’t mean they’re the right interpretation solution. In fact, in most cases we think they’re the wrong choice. Here’s why.

Don't we spend enough of our day staring at tiny screens?

Don’t we spend enough of our day staring at tiny screens?

You’re about to visit a castle, explore a gallery or investigate a museum. You’re going to scale mighty stone towers, delve into dark and dank dungeons, admire exquisite works of art, marvel at the ingenious artefacts of our ancestors. And, of course, you’ll want to spend a large amount of your time peering at a tiny video screen too.

Sounds odd when you put it like that, doesn’t it? Why would you want to spend time staring at a tiny video screen when you’re surrounded by all that wonderful heritage and culture? But you must want to do it – that can be the only explanation for the plethora of multimedia guides out there.

Look, I have no doubt the multimedia guides are popular – we’re primarily visual creatures and we’re mesmerised by moving images – but that doesn’t make them right. I’ve got 2 children and when I take them to the park, I see plenty of parents mesmerised by the screens on their smartphones when, frankly, they’d be much better off playing with their kids.

I’m not saying multimedia guides are wrong per se, I just think they’re often introduced when they’re not needed – a ‘so and so has got one, so we’ll have one’ phenomenon that doesn’t deliver the best experience for visitors. Besides, museums and heritage attractions have barely scraped the surface when it comes to the potential of audio to enhance their visitor experience.

When we approach each project, we have 2 guiding principles – ‘eyes up’ and ‘augmenting reality’. Let me explain.

‘Eyes up’ is a description of how we want visitors to explore a museum, gallery or heritage attraction – keeping their eyes up taking in the visual feast that surrounds them, rather than down, locked onto a 5 inch screen.

Augmenting reality’ is the job of interpretation – using media to enhance the sights, sounds and smells of the environment the visitor is exploring, not to replace them. Let me give you an example.

When we were approached by the National Trust’s Hidcote Manor Garden, our challenge was to help visitors to understand the history of the garden and its creator, Lawrence Johnston.

Of course, a multimedia guide would have been an option, but do we really want people to be squinting at a tiny screen when they’re exploring one of the world’s most beautiful arts and crafts gardens? That’s an emphatic ‘no’.

Lawrence Johnston, who we brought back to life via binaural audio.

Lawrence Johnston, who we brought back to life via binaural audio.

So we created a pure audio experience. In fact, better than that, we created a 3D audio experience using a little known audio technique called binaural sound. Because it’s recorded in the same way the brain hears, audio recorded in binaural isn’t just heard to left and right like stereo, but all around the listener. It’s an immersive, and often quite spooky, experience.

Using this technique, we brought Lawrence Johnston back to life to take visitors on a tour of his gardens and his life. Visitors felt like Johnston (played by an actor) was with them – sometimes encouraging them to follow him further into the garden, sometimes whispering conspiratorially into their ear. We also used this technique to transport visitors to different times and places – the garden in its 1930s heyday and packed with socialites, or the South African veldt on one of Johnston’s plant-hunting expeditions.

Were there visuals ? Yes – a map to guide visitors around the garden and one static archive photograph to accompany each of the six stories. But our main aim was to ensure that we were augmenting the reality – the beauty of the garden – creating an eyes up experience where Johnston’s enthusiastic descriptions enhanced what they were seeing.

That’s not to say there isn’t a place for ‘replacing reality’ sometimes too. We’ve created audio visual experiences that have been designed to stand alone – not working with their environment – because that environment was a gallery offering no other stimulus other than blank white walls and plain wooden floors. Then you have to conjure up something from nothing.

And we’ve created mobile games too – but games that encourage people to interact with the space around them, and to minimise time interacting with the screen.

But the temptation when you have a screen available is to fill it with something – and it’s then you could fall into the trap of creating content that distracts rather than enhances.

Our advice would be to think augment rather than replace. Like you and me, your visitors probably spend too much of their time looking at tiny screens. Free them up to look at something genuinely inspiring for a change.