Microsoft has invited the media to a mixed reality event this October in a move that highlights the growing mainstream importance of augmented and virtual reality. The event is expected to highlight applications of Microsoft’s new Windows Mixed Reality (WMR) headsets, reflecting an effort to stay competitive with rivals in the VR headset market space such as Oculus and HTC.
There were 2.1 million VR and AR headsets shipped in the second quarter of 2017, an increase of 25.5 percent year-over-year. Driven by demand for advertising, gaming and automotive industry applications, the global VR headset market is expanding at a compound annual growth rate of 35 percent and is on track to reach a value of $5 billion by 2022.
In order to carve out a share of this growing market, companies such as Microsoft are seeking to gain a competitive edge by introducing innovations that bring the mixed reality experience closer to real life. Here’s a look at some ways VR innovations are bridging the gap between virtual and real experience.
Making VR Smart with AI
The next generation of mixed reality headset will incorporate an artificial intelligence chip, representing a merging of VR and AI that is one of the most significant trends in current research and development. AI is needed to enable VR headsets to perform applications that require big data analytics, such as computer vision and object recognition.
For instance, let’s say you’re wearing a virtual headset when your dog walks into your room, and your VR set then creates a virtual image of your dog that you can pet with your virtual hands. In order for VR headsets to do this, they must able to recognize visual patterns through machine learning, a process that scans data for statistical trends in order to build a mathematical model that can make accurate predictions.
Cloud-based apps such as Facebook can perform computer vision operations such as recognizing persons and objects in photos because they have a large amount of server space and computing power to devote to big data analysis. But running this amount of data through the cloud takes time, which can create a lag between VR time and real time. In order for a VR headset to handle computer vision locally in real-time, it requires very powerful local data processing hardware.
Toward this end, hardware manufacturers have developed technology such as the Snapdragon machine learning mobile platform, which is fast and powerful enough to process camera and sensor data locally in real-time in order to generate accurate VR simulations that stay in sync with your physical movement. This makes what you see on your VR headset as close as possible to what you would see in the real world.
Integrating Speech Recognition
Being able to perform rapid AI calculations also can empower VR headsets to integrate virtual reality with real-time speech recognition. Speech recognition can be used in VR for purposes such as talking to virtual characters in games, interacting with virtual customer service agents or selecting from a long menu of options where it’s faster to input your selection verbally.
Facebook has already developed speech recognition technology that can be used with the Oculus Rift or Samsung Gear headsets. Current applications include verbally searching for information and automatically adding captions to videos. As VR speech recognition becomes more sophisticated, voice-controlled VR personal assistants are expected to become another popular application.
Adding a Sense of Touch
Another way virtual reality is getting more realistic is integrating the sense of touch into the VR experience. Haptic technology, which simulates the sense of touch in a computer environment, has already entered VR through devices such as styluses. Some researchers are looking to take this a step further. For instance, Cornell University’s Organic Robotics Lab has partnered with Nvidia to develop the Omnipulse, a gaming interface that combines soft rubber with air pockets that can be adjusted pneumatically to simulate the feeling of different textures.
For example, the Omnipulse can be adjusted so that gripping it feels like you’re holding a gun. Other companies are developing full-body VR haptic suits so that effects such as the impact of punches and bullets can be virtually felt.
By making virtual sights, sounds and sensations a closer approximation to real-time interactions, VR is increasingly bridging the gap between virtual and real experiences. As mobile devices continue to develop faster processors capable of handling local AI machine learning, and as 5G networks accelerate the speed that virtual information can be downloaded, expect to see increasingly realistic, real-time VR applications spread out from the gaming community to become more mainstream over the next few years.