Opinions on the ideal VR, AR and MR headsets for the near future

chip sineni
Haptical
Published in
6 min readDec 20, 2016

--

Image from “Lonely Ghosts” Tango project

I am a co-founder of two companies (Phosphor Games and TRIXI studios) that in 2016 shipped games on HTC Vive, Oculus, PSVR, Lenovo Tango, and are wrapping up applications on Google Daydream, GearVR, and HoloLens. Making applications for these systems means spending thousands of hours with the devices; experimenting and prototyping uses that will never go public.

Based on the experience of developing on all these platforms, one acquires a strong opinion of what core features a mass market Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) device should have, for them to take-off with consumers. There tends to be a lot of consensus with the developers on what these devices need, but not a lot of information presented on it.

Listed below are the technologies that a mass market device needs for users to experience baseline VR, AR and MR applications . These features are not intended for the highest-end platforms. High-end systems will specialize more and have different requirements per their use case.

NEAR TERM Virtual Reality /Mixed Reality Mass Consumer Device:

The primary use of this device is for a user to be placed inside a different world and be fully immersed, for productivity or entertainment. It needs to be extremely easy to use, and for the user to not get sick. This device does not need to output photo-real graphics, it just requires proper immersion of the graphics that are there.

  1. Device should not be tethered to a non-movable station. (should not be directly hooked up to a PC or console)
  2. Device should not require external sensors to set up and calibrate. (anyone can quickly use it)
  3. Device needs motion tracking (“inside-out”) to move user through physical space. (this will make mobile platforms feel immersive, and removes common motion sickness)
  4. IDEAL: Device should have 180 stereo video cameras. This feature is needed so users can:
  • Experience Mixed Reality content, seeing virtual things in their actual worlds
  • Still see world around them while in VR, for both safely and privacy

5. IDEAL: Device should have a 3D Depth Sensor. This is so user’s actual world can be recognized and they can:

  • Use their actual hands to touch interact with the world
  • Track physical controllers
  • Have mixed reality objects properly interact and occlude in the world
  • See world objects they might bump into for VR (enhanced chaperone / guide system)

This future spec device is not waiting on any technology and could be built today. Once these features are standard, they can incrementally get better (better recognition, better graphics, etc).

Ideally this device should really be a “self-contained” VR / MR headset , meaning it does not use multiple separate components, like a phone attaching to a Head Mounted Display. Why ? Issues with mobile devices coupled with HMDs are:

  1. There is a lot of friction of getting a mobile device in the headset.
  2. There are lots of OS software issues with mobile devices that expect a single screen and touch input, but are used as stereo output with controllers.
  3. Hardware is less optimized for proper VR use if it needs to act as both your phone and an HMD display and brains.
  4. There are ergonomic issues that would allow devices to look nicer if there was no variable sized mobile device going in and out. HMDs also need consistent locations to place sensors.
  5. It is confusing for a user to purchase a phone they do not want, to use an headset they do want .

Microsoft’s HoloLens is almost this device, except it is not for consumers and has a poor field of view for fully immersive VR experiences. Many people are unaware how well the HoloLens does Virtual Reality (not just Mixed Reality) The “HoloTours” application is a good example of this.

HoloLens has the best OS user experiences of any Head Mounted Display. Unlike other headsets, there is no awkward hand-off in the OS between a system’s primary 2D use and shoehorning it to work as a stereoscopic viewer. You put a HoloLens on, and it ‘just works’, anywhere.

Occipital’s future HMD is also very close to this device. The only difference is the it is not a self-contained device to correct those five issues mentioned above.

A device like this is needed to truly have “Social VR”, where people can easily get into a virtual world and share experiences and move around naturally in non-technical ways.

NEAR TERM Augmented Reality /Mixed Reality Mass Consumer Device:

The differences between what AR glasses would PRIMARILY be used for and what they IDEALLY could be used for are considerable. This changes the device specifications significantly, so the details are listed separately.

The PRIMARY consumer use of AUGMENTED REALITY glasses are for things you currently use your mobile device for, and for information you want at a (literal) glance. This means two dimensional HUD style functionality for: messaging, social, phone calls, photos, GPS, and information query. New uses might include facial and spatial recognition.

Users will glance at information when they need it, and look back into the real world when they don’t.

  1. This device should be indistinguishable from everyday sunglasses so it is not noticeable. This means that the “brains” of this device will most likely be a separate unit, like a mobile sized device that is either tethered like headphone cables, or non-tethered via a wireless connect.
  2. The “brains” could also control the display, in addition to voice or hand or eye gesture.
  3. Both lens should be full display screens. It should have a wide field of view that completely covers what the user sees, like how real glasses/ sunglasses work.
  4. Device should last entire day without charging.

The IDEAL use is a MIXED REALITY solution for entertainment like game playing, and productivity like training or visualization. These features are very useful and fun, but might not be used every day by every user (for example, not everybody uses their mobile device for 3D applications like games).

Here the device needs more complicated hardware to perform these tasks and needs perfect synchronization so the virtual objects fit in the real world.

  1. IDEAL: Lens that can make it feel like objects are correctly placed in the world so that the user doesn’t experience eye strain and can have a more comfortable and immersive experience. Eyestrain can occur in MR more than VR because you are looking at both real and virtual objects together, so user is focusing at things both near and far away.
  2. IDEAL: Device needs motion (inside-out) tracking to move the user through physical space. This allows the user to get closer and further away from virtual objects in the world.
  3. IDEAL: 3D Depth Sensor. This allows the actual world to be recognized. This is so the user can:
  • Use their actual hands to touch interact with world and use them as controllers
  • Track physical controllers
  • Have mixed reality objects properly interact and occlude in the world

The PRIMARY AR device is similar to what next versions of what ODG, Vuzix, and Epson are promising to bring to market.

The current Hololens is an early IDEAL MR device, but their device is notably different than ‘everyday sunglasses’ and would be awkward to wear in public. Magic Leap’s device as mentioned in the press would match the IDEAL MR.

Next up: the LONG TERM VR/AR/MR devices!

chip sineni is the co-founder of Trixi Studios and Phosphor Games. The opinions and information provided here is the writer’s personal view and may not necessarily represent the perspective of our editorial. Send us a message, if you’d like to contribute an article to Haptical.

--

--

Method Designer, father of 3, 20+ year game developer, focus on #AR #MR #VR @TRIXIstudios + @Phosphorgames @BrookhavenVr #PHANTOGEIST