xR, AR, VR, MR: What’s the Difference in Reality?

What does eXtended Reality (XR) mean, how does it relate to VR, AR and MR and which do you need for the metaverse?

eXtended Reality (XR) is a ‘catch-all’ term for technologies that enhance or replace our view of the world. This is often through overlaying or immersing computer text and graphics into real-world and virtual environments, or even a combination of both.

XR encompasses augmented reality (AR), virtual reality (VR), and mixed reality (MR). While all three ‘realities’ share common overlapping features and requirements, each has different purposes and underlying technologies.

XR is set to play a fundamental role in the metaverse. The ‘next evolution of the Internet’ will converge real, digital, and virtual worlds into new realities, accessed via an Arm-powered ‘gateway’ device such as a VR headset or pair of AR smart glasses.

XR technologies share some fundamental similarities: A core part of all XR wearable devices is the ability to use visual input methods such as object, gesture, and gaze tracking to navigate the world and display context-sensitive information. Depth perception and mapping are also enabled through the depth and location features.

However, XR devices vary based on the type of AR, MR, and VR experience and the complexity of use case that they are designed to enable.

What is Augmented reality (AR)?

Augmented reality (AR) enhances our view of the real world by overlaying what we see with computer-generated information. Today, this technology is prevalent in smartphone AR applications that require the user to hold their phone in front of them. By taking the image from the camera and processing it in real-time, the app is able to display contextual information or deliver gaming and social experiences that appear to be rooted in the real world.

While smartphone AR has improved significantly in the past decade, its applications remain limited. Increasingly, the focus is on delivering a more holistic AR experience through wearable smart glasses. These devices must combine an ultra-low-power processor with multiple sensors including depth perception and tracking, all within a form factor that is light and comfortable enough to wear for long periods.

AR smart glasses need always-on, intuitive, and secure navigation while users are on the move. This requires key advancements in features such as depth, occlusion (when one object in a 3D space is blocking another object from view), semantics, location, orientation, position, pose, and gesture and eye tracking.

In 2021, a number of new smart glasses models arrived on the market, including the Spectacle smartglasses from Snap, Lenovo ThinkReality A3, and Vuzix’s Next Gen Smart Glass. AR smart glasses are set to enhance all of our lives in the future, as described in the video below. They’ll also likely play a role as gateways to the metaverse where virtual elements and reality intersect.

What is Virtual reality (VR)?

Virtual reality (VR) completely replaces a user’s view, immersing them within a computer-generated virtual environment. This type of XR technology has existed for a while, with gradual improvements. It is used primarily for entertainment experiences, such as gaming, concerts, films, or sports but it’s also accelerating into the social domain. For VR, the immersive entertainment experiences will require capabilities like HD rendering pipeline, volumetric capture, 6DoF motion tracking, and facial expression capture.

VR is also used as a tool for training and in education and healthcare, such as rehabilitation. To make these experiences possible (and also seamless) for the end-user, the focus of VR technology is often on high-quality video and rendering and ultra-low latency.

Finally, VR devices are now enhancing video conferencing experiences through platforms like RecRoom that enable virtual meet-ups in different virtual worlds. RecRoom, which now supports the Oculus Quest, was featured in episode three of Arm’s New Reality series in 2020 that discussed immersive experiences with VR.

While some VR devices must be tethered to a PC, standalone VR devices such as the Oculus Quest 2 pictured are able to deliver AAA gaming and metaverse experiences. Powered by high-end Arm processors, these standalone VR devices can be taken anywhere. In the future, standalone VR devices will become gateway devices to the metaverse.

What is Mixed Reality (MR)?

MR sits somewhere between AR and VR, as it merges the real and virtual worlds. There are three key scenarios for this type of XR technology. The first is through a smartphone or AR wearable device with virtual objects and characters superimposed into real-world environments, or potentially vice versa.

The Pokémon Go mobile game, which took the world by storm back in 2016, overlays virtual Pokémon in real-world environments via a smartphone camera (this was also demonstrated on a HoloLens 2 as shown at Microsoft Ignite 2021). This is often touted as a revolutionary AR game, but it’s actually a great example of MR – blending real-world environments with computer-generated objects.

Mixed reality is also beginning to be used to enable VR real-world players to be superimposed into video games to bring real-world personalities to game streaming platforms such as Twitch or YouTube.

Advancing AR and VR Experiences

Arm focuses on developing technology innovations that power the next generation of xR devices. Arm CPU and GPU technology delivers a number of benefits, including improved performance and increased power efficiency.

Learn More