What is Immersive Storytelling (and Why Should You Do It)?

Immersive storytelling will unlock the true potential of your museum or attraction. But what is it precisely? And how can it increase visitor satisfaction and attract new audiences?

what_is_immersive_storytelling

“Stepping forward, you feel a slight shiver of trepidation as your eyes struggle to penetrate the gloom. The first thing you sense is the noise – the distant howling of an Antarctic wind outside and, more loudly and worryingly, a series of ominous, sometimes downright alarming, creaks all round you.

You can feel the wooden planks of a ship’s deck under your feet and the smell of pitch fills your nose. As your eyes adjust, you realise the dim light is cast by oil lamps, gently flickering. Suddenly, the lamps sputter out and you’re pitched into utter darkness. Your breath quickens and your heart thuds loudly in your chest as, frantically, you wonder what will happen next.”

Those aren’t the first few lines of Patrick’s Cornwall’s latest naval blockbuster but the first few seconds of our Shackleton Immersive Cinema experience at the Fram Museum in Oslo. The experience of those first few moments demonstrates the power of immersive storytelling and highlights its two key aspects.

Shackleton Immersive Cinema at the Fram Museum, Oslo.

What is Immersive Storytelling (and Why Should You Do It)?

Firstly, this is all about placing your audience at the heart of the story. They’re there – in this case, stepping back 100 years into the gloomy hold of Shackleton’s Endurance as the life is slowly crushed out of it by the asphyxiating sea ice. They may be able to influence events, or they may be mere bystanders, but they’re going to experience the same events, the same emotions, as the people who were actually there.

Secondly, all of their senses are engaged. The distant howl of the Antarctic wind, the dim glow of the oil lamps, the feel of the wooden deck underfoot, the strong smell of pitch assailing their nostrils and snaking through their mouth on to their tongues.

So if that’s answered the question “what is immersion?”, what about the “why immersion?” query. Well, there are multiple reasons.

I’ve worked in marketing for over 20 years and the holy grail for marketers is word of mouth. That’s because people telling other people about your product or service is free. Of course, the theory’s the easy bit – in practice it’s much harder. But there are 2 key principals principles to generating word of mouth. The first is “unexpectedness“. The second is “emotion“.

It’s still the case that, the expectation of a museum experience involves lots of glass cases and lots of reading. Pitching people into the heart of an Antarctic ship as the sea ice squeezes it so hard it shoots out of the water, well, that’s unexpected – even in the most high profile of attractions.

Words can stir the emotions no doubt but nothing can top the emotional impact of being there – of seeing what those involved would have seen, hearing what they would have heard, feeling what they would have felt. Each emotion stimulated creates a little hook in the memory – making it more likely they’ll recall your story long after any written account would have been forgotten. The sheer impact of stirring all those emotions will mean they’ll be desperate to share their experience with their friends and family.

So immersion can help your stories live longer with your visitors, and compel them to tell their friends. Not bad. But I also think it can do more. It can not only can it bring you new visitors but, it can bring in a different type of visitor.

Two groups are key to audience development – they’re Out and Abouters and Explorer Families (as defined by our friends at the National Trust – you can read more about their audience segments here.)

For these two groups, museums and heritage attractions are just one of a number of leisure time options. They have the desire to learn but it’s balanced with the desire to have fun – either with their partner or with their kids (or both). The traditional museum experience holds limited appeal but the chance to have fun while adding to their knowledge – well, that’s very attractive. Immersive storytelling ticks all their boxes

But there’s one final reason for this approach, and it has nothing to do with your visitors. When I was young, I used to run around castles and yearn to be a knight – to experience the pomp and chivalry of the joust, or the adrenalin-pumping adventure of battle. I’m sure you had similar childhood fantasies, otherwise you wouldn’t be doing what you do.  With immersive storytelling, you can fulfill your own fantasies, and those of your visitors, even if they’ve been buried for decades. What could be more rewarding than that?

Immersive storytelling is groundbreaking, a little risky and may take you out of your comfort zone (although people like us are here to help.) But looking at your visitors ‘ wide-eyed stares of wonder and hearing their cries gasps of excitement makes all that effort worthwhile.

That’s why we do it, and why we think you should do it too.

Hallucinations and Why They're Key to Effective Immersion

Hallucinations and Why They’re Key to Effective Immersion

Hallucinations are both much more common and benign than you may expect. And the art of engendering them in your audience is the key to effective immersion.

Hallucinations and Why They're Key to Effective Immersion

Hallucinations are both more common and benign than you might expect,

Let me tell you about some weird things that have happened when I’ve been listening to binaural (3D) audio.

Some years back, I was listening to the first cut of our ‘Walk with Lady Mary’ audio trail for Conwy Council and the Forestry Commission. One sequence involved us journeying back to the 16th century to meet Lady Mary’s Tudor ancestor, who comes barrelling out of the woods on his trusty steed. On the soundtrack, a horse neighed and, from the corner of my eye out of my office window, right on the edge of my peripheral vision, I saw a horse trotting along the road outside. When I turned, it wasn’t a horse at all but a car. But I was sure I’d seen a horse…

Another time, I was listening to an underwater track for a piece of scuba diving VR. Now, I haven’t spent much under the sea but I did do some scuba diving when I was a backpacking around Australia in the early 90s. That track alone created a strange sensory reaction – my skin went cold and goosebumps started to rise on my arms. The track had clearly surfaced that 30-year-old memory of diving off the Great Barrier Reef – but not the visual part of it, just the feeling of being under the sea.

I had a similar sensory reaction when I was sitting in my office listening to a soundtrack recorded on a tropical lake in South America. It was a cold, winter’s day and I didn’t have the heating on (it was the early days of PastPorte and I had to mind the pennies…) Even though I was wearing two pairs of socks, my toes were getting a little stiff with cold. But I was listening to this sound track and the strangest thing happened – my toes began to warm up, as if my brain believed I was really in the tropics.

How do you explain all of this? Well a recent article in The Atlantic on the nature of hallucinations may go some way to doing so.

As soon as I say ‘hallucinations’ that will make you think of people seeing things that aren’t there. That’s a partially correct definition of the word but a hallucination is actually any sensory perception of something that isn’t there. Ever felt your mobile vibrate in your pocket, taken it out and found there’s no reason why it would have done so? Or felt a phantom rain drop on a cloudy day when it wasn’t raining? Or thought you heard someone in your house calling your name, checked and found that nobody had? Well, all of those occurrences are forms of hallucinations. And before you question your sanity (or I question mine) you should know that hallucinations are remarkably common things and for the most part, entirely benign and not the result of any deep seated psychosis.

So what causes us to perceive things that aren’t actually there? Well, the experts think it may be down to how our brain perceives the world around us.

Corlett and Powers, both from Yale School of Medicine, have advanced the theory of ‘predictive coding.’ “When we go about the world, we’re not just passively perceiving sensory inputs through our eyes and ears,” Corlett says. “We actually build a model in our minds of what we expect to be present.”
It’s Corlett and Powers theory that the brain doesn’t always get this model right – it has the capacity of overpredict – to be fooled, in essence. I can expect something to be there which isn’t, and this expectation can be so strong that the brain creates something that isn’t there. A hallucination.

I should say at this stage that the predictive modelling theory is speculative and isn’t yet accepted psychological fact but it goes some way to explain the false sensory reactions that I, and others that have listened to binaural audio have experienced. Those hyper realistic three dimensional sounds have made our bodies build a false model of our surroundings and elicited some quite remarkable sensory hallucinations.

Why particularly does sound have the power to do this? We think it’s something to do with our sensory hierarchy. We’re so visually dominant – so programmed, from our earliest days of roaming the Africa plains, to look out for danger – that it’s easier to fool those other senses that are well down the pecking order. And clearly, this ability to fool the brain aurally can create these sorts of false sensory reactions – benign hallucinations which have the ability transport people, immerse them further in our imagined-worlds and create truly deep-seated and memorable multi-sensory experiences.

We think we’ve barely scratched the surface of the potential of audio – and other, second tier senses such as smell – in cultural attractions, particularly inside heritage attractions where there’s already an authentic visual backdrop to work with. The secret to effectively to transporting people to new times and places isn’t necessarily through their eyes – where they’ll likely see the flaws in your illusion – but through those other, secondary senses that are easier to fool. And the theory of predictive modelling would suggest that you only have to create part of the illusion – then you can rely on their imagination to do the rest…

Our Guide to Immersive Storytelling in Museums

Immersive storytelling is an exciting and memorable way to bring history to life but it can be a daunting challenge. We share some tips for getting it right.

Bringing the past to life is one of the most memorable and exciting ways of interpreting history. And with all the immersive tech now available, helping visitors to feel, smell and breathe the air of historical events is more and more possible.

Binaural audio – with the right concept, script and actors – can trick the brain into believing you have time travelled and are experiencing a new reality – even though the sights around you haven’t changed. Virtual reality can take you anywhere, to be anything, moving through time and space. Haptics bring an extra sensory dimension, smells can be hugely evocative and all of it can be combined to create spine-chillingly powerful effects.

It’s a very exciting prospect. But perhaps a little overwhelming.

There are so many possibilities and so much choice of kit it’s easy to be blinded by the tech and lose sight of what it is you want to achieve and what suits your space.

There are four key mantras that we at PastPorte tend to work by – and live by. They work for us. And we think they’ll work for you too.

1. Keep it simple.

Rarely is the success of your immersive experience going to depend on how much you’ve spent on the kit, how many microphones you’ve used, how many speakers you’ve plugged up or how many projectors you’ve got going.

It can be hard to stick to keeping it simple. Especially when seduced by constant tech innovations.

One of Pastporte’s most successful installations used just two simple resonant speakers, purchased in a domestic high street store, loaded with a pared-back atmospheric soundscape.

Also we try and use silence. Lots of it. The brain gets overstimulated with a constant assault from sophisticated, exciting sounds and starts to regard them as normal. Binaural audio works best if it’s a constant surprise. But to achieve that, the brain almost needs to have forgotten what it sounds like. So, in any story, we build in sections that are silent or low key to create contrast and keep up the surprises. We think that makes an experience more immersive.

2. If there’s something to see, keep your visitors eyes up.

Encouraging visitors to look at a screen and use a multi-media guide is great if there’s nothing to see at your attraction.

But most museums and attractions are all about feasting your eyes on what’s around you – aren’t they? So why would you want to distract from that by asking your visitors to look away from your visual treats and instead, get their eye candy from a screen (especially for kids when their parents are likely to feel they’re too screen-dependent already) ?

If there’s something to look at, let them look at it but augment their experience in other ways. Here too binaural audio can build your experience into a full immersive, time-travelling wonder. Especially if it’s the right audio.

3. Build on your strengths.

Make your artefacts the centre of your immersive story. Turning an artefact into an interactive, immersive exhibit is a neat trick – one we’ve regularly used at PastPorte. Using a gorgeous, if knackered, bellows camera, installed headphones and an AV screen, we transported viewers to Antarctica, 1915. The leather bellows (with associated musty smell), mahogany and brass housing all started to do our work for us – before the viewer had even begun our immersive time travel experience.

A visual, tangible object that can tee up the brain for what is to come adds extra impact. And the added bonus of using an artefact that already feels part of the period and is appropriate in the setting means the whole experience is more integrated yet also more surprising.

4. Content is king.

Whatever approach you choose, and however you choose to deliver it, it’s the content that will make your experience stand out. However great the tech set-up, if the content doesn’t tell the story well, no-one will remember your experience.

Spend time devising the right story and the right approach to telling that story, and constantly review both as you develop the experience. It can be easy to lose sight of the most important thing in amongst the glitz of all that fabulous, shiny kit.

But it’s the content that will live on when the latest tech fashions have come and gone.

Transporting Visitors to Tribeca into the Heart of the Turning Forest

How do you stand out amongst 50 virtual reality experiences jostling for attention in a crowded, noisy, shiny modern gallery? This was our challenge when we showcased a BBC R & D/VRTOV Virtual Reality experience at Storyscapes, the Tribeca Film Festival’s VR fest. 

Immersed in the Turning Forest

Immersed in the Turning Forest.

No-one likes queuing but it’s an unavoidable reality when it comes to visiting theme parks. Even the most innovative, modern venues haven’t solved this problem.

But maybe there’s a good reason not to. When you’re standing in line, drumming your fingers, trying to stop the kids from sending you slightly mad, you are already part of the Ride experience. You may not have stepped on to the rollercoaster yet, but your mind and body are anticipating the thrill to come. As you get closer to the front, you’re primed, ready to throw yourself over the brink, to fully submit to adrenalin heaven.

So how do you create that same mindset when presenting a VR experience? This was our challenge when we showcased a BBC R and D/VRTOV Virtual Reality experience at Storyscapes, the Tribeca Film Festival’s VR fest in April. The installation was one of 50 Virtual Reality experiences jostling for attention in a crowded , noisy, shiny modern gallery. Our task was to – almost instanteously – transport visitors to a world of fantasy forests that would trigger memories of childhood adventure. And to create a near sound-proofed space in which they could let their imaginations fly and become fully immersed in our VR story.

For this was one thrilling VR story. Written by Shelley Silas with binaural sound design from Eloise Whitmore and Chris Pike and VR from internationally-renowned VR artist, Oscar Raby, the Turning Forest took the audience on a journey through a magical wood – home to a friendly monster, with musical teeth, that you play by reaching out and plucking them.

The story deserved the most effective, evocative sound-proofed scene-setter we could dream up. Within (a small, BBC-sized) budget. Built in New York but procured from a UK backwater with minimal broadband. And in less than a month. No pressure then.

After several iterations and phone and internet bashing to wring out the best deals, we ended up with a beautiful bivouac-style tent. On to a strong domestic pergola (from China via Kmart) we draped acoustic blankets (think very thick duvets) and topped them with yards of autumnal camouflage netting with red, ochre and brown leaves intertwined. Inside there was a carpet of fake grass and small grassy swivel stools to sit on. With smells of the forest – created by the BBC’s Zillah Watson’s children – it was an almost fully sensory experience. Vibrating backpacks tuned to the pitch of the monster’s movements took the audience one step closer to total sensory immersion (perhaps we should include a Heston Blumenthal contribution next time?)

Grander plans had come and gone: Spiraling cones, leading to dark sound-proofed centres, mirrors and odd-shaped rooms to create the sense of the space getting smaller while the visitor gets bigger and igloo-style structures made of rolled acoustic blankets.

But it worked. And the audience left with that wide-eyed stare of awe we’d hoped for.

And I’m looking forward to the next opportunity, when we can pull out those big plans and create the most immersive experience ever imagined.

An Introduction to Virtual Reality for Museums and Heritage Attractions.

Oculus Rift?  HTC Vive? Google Cardboard? VR? AR? Confused? Read on to have the world of VR unravelled…

Oculus Rift VR

Oculus Rift VR.

By midday on Christmas Day, I’d already ridden a rollercoaster, flown unaided over a bottomless chasm and crept up close to a T-Rex tucking into his lunch.

If you’re making the assumption that I’d hit the mulled wine a little early, you’d be wrong. I’d been sampling the world of virtual reality, using the Google Cardboard I’d bought for my 2 boys (OK, yes, Dad was rather hogging their present.)

2016 is set to be a BIG year for virtual reality with headsets becoming commercially available for Facebook-owned Oculus, HTC-owned Hive and Sony-owned Playstation. And it’s already creating a buzz in our sector with British Museum and the Natural History Museum VR offering virtual reality experiences last year, and a Buckingham Palace VR experience launching on Google Expeditions just last week.

So what precisely is virtual reality? How are the proposed offerings going to differ? And how can you give it a whirl for yourself without getting your hands on one of the prized (and expensive) developers’ kits?

For a definition of virtual reality, I’m going to turn to a rather handy website I found dedicated to the subject (vrs.org.uk.) They defined VR as ‘a three dimensional, computer generated environment which can be explored and interacted with by a person.‘ It’s important to note that virtual reality takes you out of your environment and into a completely new world, not to be confused with augmented reality which digitally enhances your existing environment (that’s the subject for another post.)

The term virtual reality dates back to Jaron Lanier and the Visual Programming Lab in the 1980s (the 1992 film Lawnmower Man was based on Lanier) but certain aspects of virtual reality can be traced back to the 19th century and the first stereoscopic viewer developed by Charles Wheatstone in 1838.

All of the virtual reality offerings that I’m going to talk about are headset based – that is, you strap on a pair of goggles to enter your virtual reality world (vs. say a flight simulator, which is also a type of virtual reality but clearly isn’t headset based). From this base offering, the solutions differ in an attempt to create more realism. Some are purely visual , others bring your hands into the virtual reality world with you so you can manipulate objects, and others allow you to move around (that is, using your own feet rather than a games console controller).

All are striving for the holy grail of ‘sense of presence’, where the experience is so close to how you interact with the real world that you brain tells you you’re actually there. The opposite of this is ‘sense of nausea’ – created by the VR graphics rendering too slowly, creating a mismatch with the motion sensors in your ears and producing the VR equivalent of travel sickness.

OK – so I’m hoping you’ve got a sense of what VR is, and what makes it less virtual and more reality. Let’s take a brief look at the different offerings out there.

Oculus

Oculus was the company that propelled virtual reality back onto the agenda. Founder by a teenage Palmer Luckey, the instigator of a wildly successful Kickstarter campaign in 2012, the company has been Facebook-owned since 2014.

oculus_rift
The Oculus Rift headset features 2 high resolution graphic displays (one for each eye, so you get the stereoscopic 3D effect), complete 360 degree head tracking, a 110 degree field of view and over ear headphones for a 3D audio effect. It also comes with a external sensor (to detect your movements), Oculus controller (for moving around and interacting with your VR environment) and an Xbox One controller (because many of the games developed for the Rift will require the controller for movement, shooting etc.)

The Rift is already available for pre-order ($599 if you’re thinking of buying one) and will be shipping at the end of March. The Touch sensors, which will enable users to point, wave, pick up virtual objects with their hands etc won’t be available until next quarter.

You should know that the Rift is a ‘wired’ experience (i.e. your headset will be wired to PC) and you’ll need a high specification PC (8GB of RAM, high specification graphics card) for it to work.

Oculus is the solution for those who want to create a top end VR experience for visitors to their museum or attraction and have a significant budget to invest in the hardware and the content. The Oculus Rift audience is going to be pretty niche initially – due to the investment required to get set up – so even if your experience is made available to download, there aren’t going to be that many people who’ll be doing it. That being said, because premium VR experiences will be rare, you’ll have visitors queueing up to try it out.

HTC Vive

Yes – you’re right, HTC are a primarily a mobile phone manufacturer. They rather took the tech industry by surprise when they announced the Vive at the Game Developers’ Conference last March. It may not have the first mover advantage of the Rift – pre-orders don’t start until 29th February with shipments starting in April – but in one way, the Vive promises to be an even more immersive experience than the Rift. Why? The ability to physically move around in the virtual world.

HTC-Vive-Pre-Headset-4

While the Rift is designed to be primarily a ‘sit down’ device, the Vive comes with laser trackers that allow you to move around in a small space – up to 15 foot by 15 foot, to be precise. As you’ve got a pair of goggles strapped to your face and can’t see the objects around you, it has a ‘chaparone’ mode which brings outlines of those obstacles into your virtual world so you don’t bump into anything.

Other than that, the Vive is very similar to the Rift – same resolution of display, same field of view and a pair of controllers that bring your hands into your virtual world. It will also be a ‘wired’ experience requiring a powerful (and expensive) PC to run effectively.

If you’re offering VR in your museum or attraction, Vive would be a good choice especially if your content works with the concept of visitors moving around a little. Of those that have reviewed Vive and Rift, most rate the former as more realistic experience. However, I suspect it’s audience may be even smaller than that of the Rift – it simply doesn’t have the buzz, awareness and marketing muscle of its Facebook-owned rival.

Sony Playstation VR

This headset will be designed to work with Sony’s Playstation 4. It’s very similar in specification to the Rift and the Vive – a slightly narrower field of view and some rather natty LED’s on the front that remind me of Tron (remember that movie?) The main difference is that many people will already have the hardware – a Playstation 4 – required to use the headset and even if they don’t, it will cost them a fraction of the cost to purchase one of the high spec PCs required to operate the Rift and Vive.

sony_playstation_vr_morpheus_2

However, because a console isn’t as powerful as a PC, the VR experience isn’t likely to be as ‘premium’ but because it’s more accessible, and selling into an existing community of gamers, I’d expect the Playstation VR to be an early leader in the ‘serious VR’ sector.

If you create a VR experience for Playstation, it may be slightly less premium but the hardware to run it at your museum or attraction is probably going to cost you an awful lot less!

Samsung Gear VR

The final 2 offerings we’re going to talk about – Samsung Gear VR and Google Cardboard – trade off quality of experience for accessibility as they both turn smartphone into a virtual reality viewer.

samsung_galaxy_gear_vr

The Samsung Gear VR is the more sophisticated of the two. For £90 you get the headset which you put your phone into (it’s important to note that it only works with an S5 Note or S6/S6 Edge/S6 Edge+ at present). The headset itself comes with a touchpad, so you can interact with your virtual world, and it’s own sensors (which you connect to your phone via micro USB) for more sophisticated head tracking than the phone’s own sensors can handle.

Because it uses your phone, it’s an experience completely free of wires, but of course it’s nowhere near as sophisticated as the Vive, Rift or Playstation – primarily because the phone screen can’t deliver the resolution of the purpose built rivals.

That being said, it’s much cheaper hardware wise for you to offer than the 3 premium offerings, and your experience could ‘stretch’ beyond your museum as Gear VR experiences are so easy for people to download (via the Play Store) and try for themselves at home.

Google Cardboard

You can’t get much more accessible than a lump of cardboard that you can buy on Amazon for £9.99 , construct yourself and then can slot your smartphone into to create a virtual reality viewer. This is what I bought for my kids for Christmas and if you’re curious about VR but don’t want to fork out nigh on £1,500 for a top end experience, this is a great way to try it out for yourself.

Google_Cardboard

Bear in mind it’s not going to work with any mobile – your phone needs to have a gyroscope so it can track your head movements. You could also do with c2GB of RAM to handle the software. Once you’ve established your phone is compatible, download Google Cardboard from the Play store to get yourself started and then pick one of any number of Google Cardboard VR experiences also available to download.

You’re not going to get any ‘sense of presence’ with Google but it’s alot of fun and a great introduction to VR.

If you create a cardboard VR experience, the hardware will cost you next to nothing and you can upload your experience to the Play/App Store and have a ready audience willing to download it and try it out.

That’s plenty enough to digest on VR for now. In the next couple of posts Dinah and I will share our experiences with VR – me with cardboard and Samsung Gear and Dinah with Oculus – our thoughts on what makes effective VR experiences and some applications for the museums and heritage sector.