A Publication of WTVP
Above: Simformotion, manufacturer of Cat Simulators, is integrating the Oculus headset into its line of heavy equipment simulators to completely immerse the user in the 3D simulated environment.

From healthcare to education and beyond, augmented reality and virtual reality are transforming a range of industries… and adoption continues to grow.

The adoption of Virtual Reality (VR) and Augmented Reality (AR) is becoming more prevalent as the benefits of safety, knowledge retention and cost savings emerge. The hardware (head-mounted displays, or HMDs) needed to view VR is coming down in price, making it accessible to more users, while the widespread availability of smartphones places the possibility of AR at anyone’s fingertips. Altogether, this technology is a viable solution for training in a multitude of industries.

How serious are businesses taking AR and VR? In its Worldwide Semiannual Augmented and Virtual Reality Spending Guide, International Data Corporation predicts that worldwide revenues for the AR/VR market will reach $13.9 billion in 2017, an increase of 130.5 percent over the $6.1 billion spent in 2016. AR/VR spending is expected to accelerate over the next several years, achieving a compound annual growth rate of 198 percent over the 2015-2020 forecast period and totaling $143.3 billion in 2020.

Understanding the Terms
VR refers to virtual reality, an artificial, computer-generated environment that recreates a real-world environment. The user is completely immersed in the simulated experience, which is achieved by wearing an HMD headset. The five major players in this category are the Oculus, HTC Vive, Google Daydream, Samsung Gear VR and Sony Playstation VR.

AR refers to augmented reality, where a computer-generated image is placed in the real-world environment and allows the user to interact with it. AR is developed into apps and used on mobile devices, or via a headset like the Hololens, to blend digital components into the real world in such a way that they enhance one another.

Gaming and entertainment are the first genres that come to mind for both AR and VR, but training is where AR and VR are most impactful. The ability for people to train and learn how to react in dangerous situations in a simulated environment—before having to react in a real-life event—can literally be a life-saver.

Serious Game Developer Andrew Pittman shows iBi Managing Editor Jonathan Wright how to pull up menus, instructions and other visual—but virtual—cues within the Hololens world. The Hololens blends digital components into the real world in such a way that they enhance one another.

Need Drives Adoption
The healthcare and military sectors paved the way in all types of simulation training. AR and VR is the current frontier. Other industries are now creating AR and VR experiences around safety and skill training, including education and the skilled trades.

Jump Trading Simulation & Education Center is probably the region’s most well-known campus for virtual training and education in healthcare. Jump recently partnered with CSE Software to develop and launch the AR Anatomy app. Used in conjunction with printed triggers, users can view the effects of different medical conditions upon human anatomy. Combining both 3D animation and augmented reality, AR Anatomy is the first of several experiences to be developed. It explores venous thromboembolism (VTE) by viewing the accumulation of blood clots inside the heart in augmented reality.

“Viewing human anatomy using current 2D imaging techniques makes it difficult to get an accurate picture of a medical problem, yet we don’t have any good three-dimensional methods to tap into the true state of what’s going on,” says Dr. Matthew Bramlet, Director of the Jump Simulation Advanced Imaging and Modeling Program. “The ability to convert these images into 3D digital pictures that can be plugged into augmented or virtual realities improves clinician understanding dramatically, which then reshapes our plans, diagnoses and the path forward.”

The construction industry has also taken note and is making advances using AR and VR for training and process review. From architects to engineers to tradesmen, the ability to see a design before it is built, learn the safety triggers and get buy-in on decisions from the customer in a timely manner is invaluable. VR makes sense.

Minneapolis-based Mortenson, for example, created an entire reality-capture division. The Integrated Delivery Advancement Team (IDAT) provides company-wide leadership and guidance in the applied use of virtual design and construction systems, tools and processes. As part of the construction of Penn State University’s Pegula Ice Arena, the Mortenson IDAT team brought arena maintenance staff into a 10-by-10-foot virtual environment to simulate the new arena and get their feedback. In the medical field, Mortenson's use of VR allows nurse practitioners to address where the best locations are for wall-installed equipment, saving the company thousands of dollars in change orders for relocating the equipment and the infrastructure behind the wall to support it.

Locally, Simformotion, manufacturer of Cat Simulators, is integrating the Oculus in its line of heavy equipment simulators to completely immerse the user in the 3D simulated environment. CSE’s 3D developers have added the Leap Motion controller so users can see their hands when they reach for the controls during training sessions. “Adding the ability to track a user’s hands and display them in the virtual world not only bridges the gap between our physical and virtual hardware, but it also helps them connect to their VR environment and become fully immersed,” says Andrew Pittman, a serious game developer for CSE. “The combination of our motion system, and the VR experience of seeing your virtual hands, stops feeling virtual and starts feeling very real.”

In addition to integrating VR technology into simulators, Caterpillar commissioned CSE to create a VR Mars experience for the NASA 3D-Printed Habitat Challenge it hosted at the Edwards Demonstration and Learning Center in August. Users donned the HTC Vive to transport themselves to Mars, pick up and wash rocks, probe the environment, and more.

Using the HTC VIVE headset and handheld controllers, Wright attempts to pick up a “rock” in a 3D simulated Martian environment, as displayed on the screen.

The Hololens (and other wearables) is changing the way techniques and skills are transferred. For example, users in the field can call up instructions for disassembly of a broken machine part, and get virtual instructions or images as they carry out the task. “Most of these devices can not only show virtual images in real-world space, they have the ability to send exactly what the user sees to an expert miles away. This could provide instant help and instruction to users in remote locations where it may take several days to get an expert on-site,” says Pittman. The user not only has the ability to send live video of the situation, the expert can then send back virtual instructions and step-by-step images that would overlay on top of the actual real-world components. This allows any user to become an expert almost instantly.

Back to School
Colleges and universities are a natural fit for exploring AR and VR for education. Medical schools have been using AR/VR technologies for the last couple of years, and their affiliated universities are branching out into other industries.

Because Jump is a joint collaboration between OSF HealthCare and the University of Illinois College of Medicine at Peoria, its space is continually filled with medical students. “Compared to traditional and passively experienced visualization methods, the immersive qualities of AR/VR learning will absolutely lead to improved cognition in medical education,” explains Kyle Formella, Jump Simulation Manager of Medical Visualization. “We’re taking it beyond just seeing or hearing, to interaction and exploration. The ability for a medical student to identify complex and rare pathological conditions in AR/VR, conditions that may be rarely encountered in direct examinations, prepares them in a way that traditional learning practices cannot always provide.”

Other colleges in Illinois are using VR as part of curriculums. Knox College in Galesburg, for example, is setting up an AR/VR lab. The VIVE headset will be used to expose students to introductory computing courses. “We believe VR and AR are here to stay, and as a new media/computer science educator, it is important to immerse students in these new media forms in a responsible, ethical way,” says Dr. Monica McGill, Associate Professor, Department of Computer Science.

The University of Illinois, Chicago is home to the Electronic Visualization Laboratory, whose mission is to enable scientific and engineering discoveries by designing and developing high-performance visualization, virtual reality and collaboration systems using advanced networking infrastructure. And it’s not just Illinois universities setting up AR/VR labs. Look throughout the country and you can find schools educating students with AR and VR technology:

While AR and VR technology is in its infancy, it is already opening doors to better learning and training. It’s only going to get better as R&D continues in the field. If we can explore body parts or visit Mars today through a VR experience, where could we go tomorrow? iBi

Annette Kiesewetter is the Marketing Strategist for CSE Software Inc. and Simformotion LLC.

Go behind the scenes with managing editor Jonathan Wright and photographer David Vernon for more on "AR / VR Technologies at CSE Software.

Serious Game Developer Daniel Miller discusses AR/VR with iBi Managing Editor Jonathan Wright
Wright tests out the latest Cat Simulator from Simformotion, CSE's sister company
Simformotion, manufacturer of Cat Simulators, is integrating the Oculus headset into its line of heavy equipment simulators to completely immerse the user in the 3D simulated environment.
Serious Game Developer Andrew Pittman explains some of the nuances of the augmented simulator.
Serious Game Developer Andrew Pittman explains some of the nuances of the augmented simulator.