In the past few years, both Virtual Reality and Augmented Reality have been dominating the tech industry like never before. Be it the media coverage, video games or other commercial applications, the reality technologies can be found everywhere. The market is already witnessing an enormous growth and the future is going to be even more favourable in terms of sales and revenues.
We have already discussed Virtual Reality vs. Augmented Reality and the related terms, but now it’s time to get acquainted with the concept of Mixed Reality aka Hybrid Reality. This term is a combination of both Virtual and Augmented Reality and is gradually taking over the market with its highly impressive capabilities.
Mixed Reality is what lies between the real and virtual worlds and despite being a mix; it functions as an independent technology. It’s a much stronger and higher level of reality technology that encompasses all kinds of variations made in a virtual or real environment.
To get a clearer idea, let’s move on to the details:
Mixed Reality Continuum
In 1994, Paul Milgram and Fumio Kishino defined mixed reality as something that lies “anywhere between the extrema of the virtuality continuum.” This Reality-Virtuality Continuum extends from the real environment, followed by augmented reality and augmented virtuality, to the virtual environment.
Mixed Reality Continuum (Image Courtesy: realitytechnologies.com)
As you can see in the above diagram, the two extreme ends of the continuum are denoted as a real environment and virtual environment. The former is completely real with no computer-generated data while the latter contains an environment where everything is computer generated. In between, there are two technologies namely
Augmented Reality: In this, virtual information is layered on to the real world scene where users can experience virtually created visuals, aurals, and feelings. For example, using Smart glasses or other headsets, while playing a video game, would cut you off from the real world.
Augmented Virtuality: This is the inverse condition where real world objects are inserted into a computer-generated virtual world. Example, using an app for the interior decoration or remodeling of any part of the house, presents a digital view of the real objects.
What is Mixed Reality?
Mixed Reality can be defined as virtual spaces where physical and digital objects co-exist and interact with each other in real-time. In a broader sense, it includes augmented reality, augmented virtuality, and other kinds of mixed configurations. In such an environment the user is free to navigate through both virtual and real surroundings, whereby virtual objects are anchored into the real world space.
This technology makes use of sensors, advanced optics, and next-gen computing powers that are stacked into a device (usually headset). By using this device, the user can overlay virtual objects in real space and time, interact with them, and manipulate them; where everything appears to be “real”.
Mixed Reality Technology
Mixed Reality maps a physical environment into a 3D image and hence evaluates where exactly to place the digital content in the real space. It creates surroundings where the use of the transparent lens, spatial sound along with the understanding of the physical environment allows the display of holograms with realistic looks, sound, and behaviour, which can sometimes be felt.
Mixed Reality Project X Ray Game Demo (Image Courtesy: YouTube)
The picture mentioned above is a demonstration of Microsoft HoloLens Gaming System where the user is playing Project X Ray. The device that can be seen in his hands appears in the form of a holographic gauntlet which is the weapon to fight enemies. While playing the game, the user is free to move around the room and would hear the sound as if it’s all happening in the same room.
Another amazing fact about this technology is that the robots in the game know exactly where the walls and furniture are located and they can use this information to plan attacks on the player. There are different modes and there is even a holographic shield that can protect the player from larger robots.
Mixed Reality Applications
The use of MR technology is not just restricted to the video games and entertainment. There are numerous fields and ways in which we can use MR to interact, play, learn and communicate with the virtual world in a real setup.
Given below are some of the applications of Mixed Reality:
Microsoft HoloLens Holoportation (Image Courtesy: YouTube)
In near future, MR is going to re-define how we interact with our friends and family. With the help of Microsoft HoloLens, one could even see and talk to people located thousands of miles away through Holoportation. In a demonstration video, it is shown how two people are talking face to face without even being present in the same room.
Not only this, but the entire session can also be recorded and played. Besides, being a 3D representation, it can be miniaturized and seen with all the different perspectives. It’s almost like traveling in time and being able to watch the memories with all possible angles.
Project Esper Demo (Image Courtesy: Vimeo)
As technology has already brought significant changes in education, Mixed Reality is going to enhance it even more with all the detailing and descriptive visuals. Project Esper is one such venture by 3D4Medical Lab which aims to aid medical students in learning about anatomy by using Augmented and Mixed Reality.
Using this technology, students can have a detailed view of the human body with all the parts and organs they need to zoom in or zoom out through hand gestures. It would also display details of the organs selected by the user. Moreover, students can even record and play the lessons and learn things through digital 3D images.
Mixed Reality Advertising (Image Courtesy: realvision.ae)
The access to devices and gadgets gives companies new opportunities for market promotion. It’s a unique way to display interactive details about products to the end consumers in a way that has never happened before. Many brands will be displaying 3D content on the packaging itself including digital discounts and product benefits. Example, Coke can deliver a realistic 3D gaming, music or sporting event experience.
4. Machine Learning
Mixed Reality Machine Learning (Image Courtesy: evolveent.com)
Since Mixed Reality combines physical objects in real spaces with the virtual elements, there needs to be some relevant content with every object that has to be utilised as a part of the technology. So, an AR startup named Blippar is working on a real-time deep learning engine using advanced computer vision.
Through this technology, almost all the everyday objects will be recognized and some amount of digital content will be displayed on the top of these objects through web. Instead of using voice or written commands, it would simply need your eyes to unlock search queries and would display information like the nutrition value of a fruit, recipe of a food item, or store locations of a commodity.
Microsoft HoloLens Use Case in Entertainment (Image Courtesy: thedrum.com)
Microsoft is making efforts to change the way people watch sports on their television. We already have features like pausing live telecasts and recording them to be viewed later or casting images and videos from mobile phones to television and even browse internet on the TV sets.
But the next big thing is going to be an exciting feature of HoloLens that would allow users to have a 360 degree view of the game on TV, where the off-screen area is automatically projected on the walls. They would also be able to view a holographic 3D view of the game on their coffee table or zoom in to focus on a particular player through hand gestures. It’s still early to reach the masses, but things are changing gradually for sure.
Mixed Reality Business Application (Image Courtesy: YouTube)
While dealing with business clients, business enterprises can make a great use of mixed reality to encash the benefits of the technology. For example, a sales rep can go to the doctor and instead of a usual meeting they can use MR headsets for the demo of the drug, its chemical composition, and how it works inside the body. This would give a clear and better idea about the product and would help in promotion.
Another example is explained in a video released by Windows where a girl is required to set up a shoe store for a group of clients. With the help of Microsoft HoloLens she is able to move holograms around the space in real-time and even take design inspirations to prepare the whole interior. If this could happen in the future, it would completely change the way business deals are performed in business.
Market Players and Investments in MR
Microsoft HoloLens Headset (Image Courtesy: microsoft.com)
The pre-requisite for Mixed Reality to reach the masses, is to make it as portable and affordable as possible. Presently, Microsoft HoloLens is the world’s first and only untethered holographic computer that can be purchased at $3,000.It works without any wires, cables, external camera or PC connection and has more computing power than an average laptop. Currently, it’s designed to be used in business for tasks like construction and architecture.
Apple’s Future Plan for Developing an MR mobile device (Image Courtesy: Wearable)
As per the market reports, it is speculated that Apple’s upcoming iPhone 8 is going to be a mixed reality device. It would feature next-gen OLED display, wireless charging and 3D Sensor by Primesense. Moreover, it would be fitted with eye sensors offering a higher frame rate and polygon count than a PC tethered MR device and Nvidia graphics card.
Apart from this, there is a Florida-based Startup named Magic Leap that intends to introduce groundbreaking developments but as of now, everything is restricted to video demos. A lot of big brands (including Google) are investing and the value has already reached to $ 4.5 Billion. Another name in the list is Meta 2 which displays virtual 3D holograms across the visor with a wider field of view, where the user can grab, pull, or stretch the objects. It’s likely to be available for purchase by the end of 2017 or later.
Magic Leap Promises Mixed Reality to be Magical (Image Courtesy: YouTube)
Moving on, there are certain companies who wish to make this technology affordable for the masses. An AR company named Zappar has announced to develop world’s first cardboard mixed reality headset named ZapBox. It comes with cardboard controllers and a special wide-angle camera lens for just $30 on Kickstarter. On the other hand, a startup named Occipital has designed a hybrid MR/VR HMD solution called the Bridge at $399, specially developed for iOS users.
ZapBox Cardboard Headset Components (Image Courtesy: Digital Trends)
Needless to say, MR has great potentials to take over the future and all the tech giants are investing huge amounts of money for its advancement. Apart from Microsoft and Apple, companies like Lenovo, Acer, and Asus are also developing their respective prototypes. Besides, leading market players like Google, Facebook, and even Snap Inc., are performing acquisitions and putting loads of money into such ventures.