What would be the best way to involve a learner to ensure maximum retention? Make the learning experience engaging, make it engrossing—in short, make it immersive.
Immersive learning involves placing the learner in an interactive and engaging environment, replicating a real-life scenario to teach particular skills or concepts. As the name suggests, it is immersive, which means it helps erase all kinds of distractions and enables the learner to concentrate on the lesson being imparted.
Solutions such as virtual reality (VR), augmented reality (AR), mixed reality (MR), 360-degree images/videos, and simulations are effective ways to make learning immersive. They not only help develop interest but also ensure engagement and retention. Immersive learning is proving to be the latest and often the best means of education for both educators and learners. While learners can take many courses independently, educators can make use of immersive media to explain concepts in a better and more effective manner.
Stages of immersive learning development
A lot has been written and discussed on immersive learning; its features and advantages. In this article, I will look at the development aspect. My aim is to provide a workflow of development that can be implemented for any kind of immersive learning solution. (Figure 1) These include 3-D immersive training, AR, VR, MR and Virtual Worlds.
Figure 1: Stages of immersive learning development
Selecting target platform
The first step in any kind of immersive learning development is to assess the learning needs and select a suitable delivery method. This may vary from VR-based solutions to WebGL platform where users can simply immerse themselves in a 3-D virtual environment using a compatible web browser. This also decides the hardware requirements at the user’s end since a VR experience requires a very high-end system configuration whereas an optimized WebGL-based solution may be able to run on an entry level computer—laptop or desktop—depending on the operating system.
If portability is the factor, then a similar solution can be designed for an Android or iOS application, which can be compatible for both mobile and tablet devices. Appropriate target platform selection is very important since all the following development stages would affect the output quality of the immersive learning environment.
Wireframing
Before diving into the application development, it is important to sketch out the flow for end-user experience. This includes defining the major steps of user engagement in terms of experience, feedback, and loops in the process, if any. Sometimes segmentation helps the development team to work on multiple stages of the process simultaneously since the output of the stages are not interdependent. For example, the 3-D developer can work on detailing of the models and the developer can work on coding the interactions. Since this needs some pre-work, the 3-D developer can prepare dummy block models (discussed later in this article) for reference and the programmer can use those for logical interactions. In a later stage, the block models will be replaced by the highly detailed 3-D models.
Asset development
A very crucial stage in immersive learning development is 3-D modeling. It is the modeling of assets that makes the virtual environment more realistic, giving the user a feel of the real world. Using advanced tools and techniques, it is possible to make the environment look alive in a 3-D scene. (Figure 2) Detailing is the most important aspect of 3-D modeling. Texturing tools can help in the 3-D painting process.
Figure 2: 3-D virtual environment of an operation theatre
Shaders play a vital role in rendering visuals. Shaders are programs that run in the graphics rendering pipeline and tell the computer how to render pixels. These programs are called shaders because they are often used to control lighting and shading effects. Custom shaders can be programmed to match the texture and color of original objects.
Another aspect of modeling is the polygon and triangle count, which controls the performance parameters of the end product such as frame rates and smoothness. High polygon count does add details in shapes, but at the same time it also calls for a high-performance system to run it. Optimization is the key here! Based on target platform selection the developer can set a limit of the maximum polygon count to balance the visual quality and performance.
Animations
In any virtual environment, animation plays a vital role to add life to it. Animation may include movement of physical objects or virtual characters. For virtual characters or living creatures, replicating the organic movement is the key challenge. Simulating hair, clothes, and other particle-based animations requires skill and expertise. (Figure 3)
Figure 3: Animating characters in a scene
Through animation, it is possible to provide visual feedback to the user. Each activity of the user can trigger a preset animation programmed in the interaction. Like a human body, animated objects are embedded with a skeleton. Their motion and movements are defined following the skeletal structures. Setting up this skeleton in a 3-D model is called rigging. The upper surface of the model is called mesh and the process of attaching this mesh over the skeleton is called skinning. With proper skinning, there will be no unwanted deformation on the surface of the 3-D model when it is subjected to movements.
Building interaction
In an immersive environment constant effort must be made to keep the learner engaged. Challenges, activities, and feedback are the key elements used to achieve this. Each of these elements should be contextual and represent the authentic task in the environment.
Block models (Figure 4) can be used in a 3-D scene to estimate the actual movement and position of the final asset. Block models are simple geometric structures with minimum detailing. They are used as references to define occupied spaces in 3-D environments. Using the block models, programmers can start with interaction building and later replace the block models with final 3-D assets. Doing so, two processes can be carried out simultaneously—setting up the scene and programming.
Figure 4: Block model example
While building the interactions, it must be kept in mind that user engagement is maintained throughout the course.
This can be achieved by implementing branching and level creation, which further helps create a sense of challenge and role playing. Branching and level creation are nothing but the menu and navigation within the learning journey. These help present a visual reminder to the users of their status in the journey—their current stage, completion status, and task goals.
When posing a challenge to users it is also important to measure user response. This can be achieved by calculating the response time within the challenge. This responsiveness is a key parameter to estimate the understanding level of the user. Different visual feedback and actions can be programmed analyzing this responsiveness.
Lighting and texturing
A well-lit scene will increase its aesthetic value as well as make it engaging to the learner. (Figure 5) Light is very important to reflect the real-world situation, as our brain responds very well to contrast and color variations. The scene should have a dynamic range in terms of color and brightness level. Texturing, which is the mapping of a 2-D image on a 3-D surface, needs to be tweaked based on the lighting output. Lighting and texturing are correlated and should be tuned accordingly.
Figure 5: 3-D scene with appropriate lighting
Testing and deployment
The most crucial step before delivery of a product is its in-house testing. This includes interaction testing, content proofreading, and looking for any scope of improvement in aesthetics.
Device testing is necessary for multiple platforms, as well as UI set up for various screen sizes. Performance on high-end to low-end systems is measured for an optimum delivery. Any final stage optimization is done based on these results.
The deployment depends upon the delivery method. It might be application packaging or through web-hosting, depending upon the type of delivery and user accessibility. Various operating systems (iOS, Android, macOS, Windows) need specific deployment platforms and authorization. These things can be taken care of at the time of development with any test application to avoid delay in the final stage.
The most important elements
The two most important elements to be considered while developing any immersive learning solution are observable attributes (virtual environments, 3-D models and events that occur) and non-observable attributes (cognition, engagement, and feedback). Through an agile development pipeline, both these elements can be strengthened. This can be achieved by simultaneously running multiple tasks of different stages. A thoughtful project plan and an efficient development team are a must to create such immersive solutions.
Experience is what immersive learning is all about. This engaging and dynamic method of learning not only enhances learning experience but also ensures better retention. Immersive learning solutions help open up a whole new world to acquiring skills and concepts. Though at a very nascent stage today, the future of learning lies in immersive learning solutions. New lands are being created to be explored by both learners and educators. Experiencing is going to be the new word for learning. So fasten your seat belts and get ready for the ride.
From the editor: Want more?
The 2020 Realities360 Conference & Expo, March 31-April 2, 2020 in Orlando, Florida, delivers over 50 sessions covering the critical topics that will help you develop new skills, strategies, and expertise within the AR & VR world.
Save $50 on your registration when you register and pay online between Saturday, February 8 and Friday, March 27, 2020 at 12 PM PT. Additional discounts are available—see the details here on the registration page.