You, as a virtual reality (VR) developer, can—no, you must—pay special attention to the creation of presence, whether you’re developing educational games, designing user interfaces, or simply making a virtual space in which people hang out and interact socially. (This is possibly true for 360-degree cinematographers, though that’s a whole other discussion.) Do it wrong and you break the illusion, bringing the user back to the real world.
I hovered over the sunken boat, air tank on my back, drifting gently toward the edge. Several brightly colored fish darted past me and over the crumbling railing. I heard a low rumbling moan coming from somewhere far to my left. Seconds later, a grayish-blue mass (a blue whale!) appeared seemingly out of nowhere, larger than any animal I’d ever seen, nearly at arm’s length. That thing was so large it nearly filled my entire field of vision. The dozens of small fish that had flashed about moments ago had all swum into nooks, hiding from this monster. Its humongous eye moved past, hesitating momentarily right in front of me, as if checking me out, the strange new creature staring right back at it. As it swam on, I could see the smoothness of its skin, the grace of its movement, the sheer mass of 80 feet of flesh gliding by. I’d have been rendered speechless had I not been underwater already. Then, the image froze for maybe half a second while the graphics card caught up on processing, and the moment was over. The whale kept swimming, but I was already back in the studio, draped in cords from the heavy virtual-reality headset and earphones strapped to my head, wondering where the nearest wall was.
Presence
Presence is the feeling that you’re really there, whether watching a movie, playing a video game, or experiencing VR. Presence happens when technology blurs the line between simulation and reality by adding depth, when people lose themselves in the experience and forget that they are in a created reality. As VR technology advances in the near future, we may no longer be able to tell the difference between reality and VR.
Presence in VR is especially difficult to create, easy to ruin, and nearly impossible to get back once it’s gone. Without presence, a VR experience can feel literally and figuratively flat. But with good presence, your mind forgets everything outside the virtual environment, including the headset and cord tethering you to a computer. It forgets the rush hour traffic and the bills to pay and everything else. You feel like you’re really soaring through alien skies, exploring an ancient pharaoh’s tomb with your history class instructor, or swimming with whales on a shipwreck.
In a paper titled “Research on Presence in Virtual Reality: A Survey” (well worth reading, as it dives much deeper into the topic of presence than I do in this article), the authors quote Matthew Lombard’s slightly more scientific definition of presence:
Presence (a shortened version of the term “telepresence”) is a psychological state of subjective perception in which even though part or all of an individual’s current experience is generated by and/or filtered through human-made technology, part or all of the individual’s perception fails to accurately acknowledge the role of the technology in the experience. Except in the most extreme cases, the individual can indicate correctly that s/he is using the technology, but at some level, and to some degree, her/his perceptions overlook that knowledge and objects, events, entities, and environments are perceived as if the technology was not involved in the experience.
In other words, feeling like you’re really there.
Whatever your purpose, presence is the factor that will ultimately make or break the success of your program. If you pull it off, people will stay, learn, and come back for more. If you don’t, the whole experience could be forgotten within minutes.
How to create presence
VR presence can be defined in multiple ways (as described in detail in the aforementioned paper; see References), but at its most basic, presence can be broken down into three broad categories, listed in order of most important to least important for their impact on creating presence:
- Visual presence
- Auditory presence
- Sensory or haptic presence
In this article, we’ll discuss each of these categories and describe their importance to elevating the quality of a VR experience. We’ll also explore some miscellaneous methods for creating presence that don’t fit nicely into any categorization.
Visual presence
Visual presence is not only the most important category, but also the most nuanced. Visual presence itself comprises multiple factors, each contributing to the overall sense of “really being there.” We’ll dive into the technical aspects of creating presence in future articles, but for now, the list below will kick-start your thinking on what it takes to create visual presence in VR:
- Locomotion—How do you move around in the virtual space?
- Tracking—Where are your head and controllers or hands as they move in and through the virtual space?
- Latency—Does the image move along with your head?
- Persistence—Do objects blur as they move?
- Resolution—How detailed are the images?
- Field of view (FOV)—How much of the virtual world can you see at a time? How wide and tall is your peripheral vision?
- Comfortable eyebox and headphones, with minimal or no cords. Is the headset light and comfortable enough to be forgotten?
Numbers one through five can all be created within the two primary VR development software programs, Unity and Unreal Engine. Together, these five factors make the visual environment look realistic and prevent nausea in the viewer. Numbers six (FOV, which is important because with a limited FOV, it feels like you’re looking down a tunnel instead of existing in a full virtual environment) and seven (the headset and cords, which, if heavy, restrictive, or uncomfortable, can constantly and painfully remind you of your real world body) are primarily dependent upon the type of VR headset used by the viewer, which I discussed in detail in this earlier article.
Besides the above technical aspects, a virtual experience must satisfactorily create the Four Illusions of VR in order to achieve visual presence:
- The illusion of being in a stable spatial space—Does the environment around you look and feel real?
- The illusion of self-embodiment—Do you feel like you really are the avatar you are controlling? Is movement intuitive and realistic?
- The illusion of physical interaction—Does your interaction with objects in the virtual environment look and feel real?
- The illusion of social communication—Are you able to effectively and seamlessly interact and communicate with other people in the virtual environment?
Admittedly, these Four Illusions often require sound and haptics to fully work their magic on the viewer, but video is the primary driver of these illusory effects.
Auditory presence
Audio presence is important, but not nearly as important as video. To prove the point, you can easily experience VR with the sound off, but try closing your eyes and see how much fun you have in VR. That said, the New York Times’ “Notes on Blindness” project shows just how vivid and fun 360-degree audio can be.
Despite audio’s secondary effect on virtual presence, good 360-degree audio can make or break a VR experience. Game companies and other VR experience developers devote a lot of attention to sound. Without good 360-degree sound, the virtual world seems flat. With good 360-degree sound, you can literally hear the shapes of rooms by hearing sounds “reflect” off virtual walls just like they do in the real world. Sighted people rarely think about this spatial auditory recognition consciously, but take it away, and a room seems uncannily quiet or simply “off.” Similarly, we can innately grasp the size, shape, and velocity of objects, people, and animals by how their sounds (e.g., speaking, breathing, footsteps, rustling of clothes or hair, vibration of air) move through space and bounce off walls back to our ears.
Binaural or stereoscopic sound is perhaps the most important factor in creating a 3-D soundscape. For a very simple example of binaural sound, if your friend is standing to your left and speaking to you, not only will your left ear hear the voice as louder, the sounds will reach your left ear slightly before your right. These subtle cues are how we can know a friend is standing to our left in the real world. The same is true in VR. If you want to take your VR experience to the next level, you must pay close attention to audio. If you’re shooting 360-degree video, consider investing in one or more high-end microphones in order to better capture the full sound environment of each scene. In fact, some sound engineers and VR game developers have devoted whole careers to perfecting 360-degree and VR audio.
Sensory or haptic presence
Haptic presence helps VR users to experience physical objects, movement, walls, borders, and, interestingly, data. Experiencing data means seeing text, chart, or graphs in a virtual environment and being able to read it, click on it, move it, hide it, change it, or otherwise interact with the data. On traditional 2-D screens, this interaction of data has become straightforward and intuitive for most of us; but in VR, developers are having to create whole new ways of interacting with data that simply don’t apply in 2-D.
In their simplest form, VR haptics include small vibrations of hand controllers, indicating when your virtual hand has touched, passed through, or in some other way interacted with a virtual object. For example, when you move your real hand and real game controller, say, forward and to the right, the virtual hand meets a virtual object, perhaps a wall or a ball. Since your real hand keeps moving and feels no wall or ball, the controller can vibrate to let you know you’ve hit something. Obviously, you can see this wall or ball, and you may even be able to hear it too, if the wall/ball is emitting a noise of some kind. However, the vibratory feedback makes the wall or ball seem even more real, or at least more noticeable, which is a step toward seeming real.
Admittedly, a small vibration in your hand isn’t much, but at least it’s something. It engages the brain on yet another level. Several VR hardware companies are currently developing haptic gloves that track individual finger movements, as well as provide varying resistance and pressure along the fingers and skin, and allow the users to feel actual heat, texture, moisture, and air movement that matches the programmed experiences within the virtual world. One company, Monobanda, has even created a haptic belt that allows you to control the locomotion of your virtual avatar with your breathing. The flagship product from Virtuix (also a maker of haptic gloves) is a VR treadmill that allows players to literally walk or run in any direction. Still other companies are working on olfactory and gastronomic haptic devices, allowing VR users to smell and taste virtual objects. As you can imagine, porn companies are pushing the boundaries of haptic technology with something called “teledildonics,” but I’ll let you Google that for yourselves.
As haptic technology improves, the range and sensitivity of virtual sensory experiences will increase, as will the percentage of the body that can interact with virtual objects, until we will eventually be able to don full haptic bodysuits. People are working on this, but it could be many years before this kind of hardware is commercially available. Until then, we’ll just have to enjoy fantastic virtual 360-degree visual and audio worlds with the occasional hand vibration.
Miscellaneous techniques to create presence
The Void, an interactive VR experience arcade or theme park with locations in Salt Lake City and Toronto, is pushing the boundaries of haptics and presence, going well beyond what we normally think of as haptics. Among the many innovative techniques they’ve developed, when you’re walking through the experience, they blow moist air in your face, make you traverse a narrow balance beam walkway that virtually appears to be suspended high in the air, and, most impressively, redirects you along a curved hallway seemingly indefinitely. This curved hallway is actually a big circle, but the virtual visual cues make you think you’re walking in an apparent straight line for a distance that far exceeds the dimensions of the actual building, without your brain noticing you’re really walking along a curved path.
Further, The Void also uses a good story with engaging narrative techniques (e.g., you get to be the star of a virtual Ghostbusters story) to create a fantastic sense of presence. VR storytelling is very different than 2-D or verbal storytelling because both pace and attention are self-directed. For example, as a VR designer, developer, or storyteller, you can’t reveal an important plot point (puzzle clue, educational instruction) while the player is facing in another direction and expect the story (puzzle, lesson) to make any sense.
Storytelling, plus these various new techniques employed by The Void, plus many additional old tricks that have been used by haunted houses and immersive theater productions for ages, perhaps each deserves their own categories of presence. At the very least, they’ll likely evolve into their own separate categories over time as the technology continues to improve at an exponential rate.
The whale, take II
You can create presence in VR in many ways. Ultimately, you’ll need to pay attention to each of the above factors in order to create a truly compelling experience. Pondering all this complexity, I decided to give the virtual whale experience (theBlu: Encounter, by WeVR) another try.
I tried on a different VR headset attached to a faster, more powerful computer. I was instantly transported back to the underwater world I was unceremoniously jarred out of minutes prior. This time the whale swam by without freezing up. I explored undersea caves, played in coral reefs, and steered clear of creepy bioluminescent creatures mesmerizing their prey. This is the kind of VR presence I love. I could play in this world all day long.
References
Bible, Thomas. “Binaural Audio for Narrative VR.” Oculus Story Studio Blog. 31 May 2016.
Ching, Teo Choong. “The Concept of Presence in Virtual Reality.” Medium. 27 August 2016.
Hoagland, Edward. “Feeling My Way Into Blindness.” New York Times. 17 November 2016.
Schuemie, Martijn J., Peter Van Der Straaten, Merel Krijn, and Charles A.P.G. Van Der Mast. “Research on Presence in Virtual Reality: A Survey.” CyberPsychology & Behavior, Vol. 4, No. 2. April 2001.
Sparks, Matt. “Metafocus: Overview of Virtual Reality and Mixed Reality in eLearning.” Learning Solutions Magazine. 23 November 2016.
Additional resources on creating presence in VR
Bye, Kent. “VR Presence Researcher Finds Full Embodiment to be Key Component in Plausibility” (podcast). Road to VR: Voices of VR, No. 555. 10 July 2017.
Callewaert, Carl. Build Architectural and Gaming Environments That Create Presence in VR (video). Vision VR/AR Summit. 23 February 2016.
Celia, Matthew. How To Deliver on the Promise of Presence When Creating 360 VR (video). Mettle. 18 December 2016.
Hoffman, Hunter, Todd Richards, Barbara Coda, Anne Richards, and Sam R. Sharar. “The Illusion of Presence in Immersive Virtual Reality during an fMRI Brain Scan.” CyberPsychology & Behavior, Vol. 6, No. 2. April 2003.
Jerald, Jason. The VR Book: Human-Centered Design for Virtual Reality. San Rafael, CA: Morgan & Claypool/ACM Books, 2015.
Lang, Ben. “Oculus Shares 5 Key Ingredients for Presence in Virtual Reality.” Road to VR. 24 September 2014.
Larsson, Pontus, Daniel Västfjäll, Pierre Olsson, and Mendel Kleiner. “When What You Hear is What You See: Presence and Auditory-Visual Integration in Virtual Environments.” Proceedings of PRESENCE 2007: The 10th Annual International Workshop on Presence. October 2007.
Marinkovic, Sasa. “First Rule Of VR: Don’t Break The Presence.” TechCrunch. 7 February 2015.
Torisu, Takashi. “Sense of Presence in Social VR Experience.” Interactive Architecture Lab Blog, University College London Bartlett School of Architecture. 17 August 2016.
Voica, Alex. “Presence in VR: How to provide a superior experience.” VR 360. 24 June 2014.