How Facebook should use Oculus Rift

Oculus has been purchased by Facebook. A $2 billion dollar bet on what may be the  future digital platform.  Virtual Reality is not new it has a long history.  The common impression of VR has been shaped by the film industry but real systems have been around for decades.

First described in the 1935 science fiction short story, Pygmalion's Spectacles by Stanley G Weinbaum as a pair of goggles which provide:

"a movie that gives one sight and sound [...] taste, smell, and touch. [...] You are in the story, you speak to the shadows (characters) and they reply, and instead of being on a screen, the story is all about you, and you are in it."

In the fifties Morton Eilig made the Sensorama, an immersive theatre system using a stereoscopic film display device with fans, and a moving seat. In 1968 Ivan Sutherland, the father of computer graphics, built the first computer based virtual reality head mounted display system. The 70’s bought us flight simulators, the 90’s gave us the Nintendo Virtual Boy, and as an owner of one I can tell you it is a real, not virtual, pain to use. Along the way the military, architects, and surgeons have found uses for the tech.

Today, with is the Oculus Rift, the technology can finally deliver on the promise of truly immersive, highly responsive environments. In a form factor, and a price point where it has the chance to go mainstream. The price is high but is bound to come down. Oculus are not the only game in town, technological advances, mass production and competition from Sony   and another notable competitor, the Avegant Glyph, will help lower prices. Facebook might even sell the headset at a loss. Because the immersive nature of the experience is so different, so compelling, they could set up a pay per experience market. 

The question is what will Facebook or the other players do with the technology? Mark Zuckerberg’s statement seems to me to have hit the nail on the head:

"After games, we're going to make Oculus a platform for many other experiences. Imagine enjoying a court side seat at a game, studying in a classroom of students and teachers all over the world or consulting with a doctor face-to-face -- just by putting on goggles in your home.

This is really a new communication platform. By feeling truly present, you can share unbounded spaces and experiences with the people in your life."

The key word is “experiences”, its all about the experiences. Before working in the web, I came from a games background, one which excels at experiences. I have always though that what is lacking from accessing the web via the traditional pages model is the ability to deliver emotions. Sure you can watch an emotional video or read a poem that moves you to tears or emotionally respond to a song delivered through the web, but the experience of using the web does not inherently have the capacity to deliver emotions.

Games deliver excitement, loyalty, fear, tension, humor, you feel your whole body respond to games. Heart rates increase, you shout at your real or virtual opponents and punch the air when you win. When was the last time you got any of this from a web page?

The web we have designed is a content delivering system. We do it very well, and I am sure will get much better at it. We have interfaces which provide quick access to content structures. We try to minimize the number of interactions to access content. We have established conventions for how we organize and manoeuvre through content. We keep is simple and intuitive.

I will put on  my VR goggles to attend a conference or take a guided tour of the Sistine Chapel, learn about a new subject through rich immersive media, or meet my friends in a virtual world.  A vastly improved second life.

However, using three dimensions to navigate through a collection of news articles and video reports is laborious and inefficient. Currently we navigate in a 2d digital world. Adding that extra dimension brings big cognitive, UI and learning overheads. It is worth investing in these to receive an emotional hit, but not to read the news, or manipulate a sales tool. A 3D world is not a good way to deliver the written word. 

Firebox a VR browser by James McCrae, Toronto University

I remember conversations in 80’s where people spoke of  cathedral builders as a model for content delivery. Use the 3d architectural structure and details as a way to access content. People are still playing with this notion. In my mind failing. The cathedral model was more a story telling mechanism than content delivery. It worked in the middle ages partly because it was a way of expressing the power of God and of the ruling class but also because of high illiteracy rates. Give me a decent 2d information architecture any day.

Today content and context are the kings. In a VR world it is the experience. I do not see VR replacing the way we consume the bulk of our content, not in the next ten, twenty years at least. The current 2d world will live along side the 3d virtual ones. Increasingly augmented reality driven by context will deliver a third way, but that is another story.

Interacting with wearables - the ring is the thing

Lets face it, the method of interacting with Google Glass via swiping the leg of the glasses is far from ideal.  Voice is also available but very public. What is needed is a small wearable input device that allows gestures, menu selection, cursor movement, mouse clicks etc.

In the wearable PC labs and geek tech corners of the web are gloves, bracelets and ring mice. These all tend to look bulky and not something you would want to be seen wearing away from your desk, or even in public. However a recent Kickstarter project funded a device which shows promise.

Mycestro is a touch sensitive 3d mouse that fits on the finger. It uses low power Bluetooth 4 to pair with your PC, but could be iPad, Glass or Smartwatch. It uses the 3D gestural motion of your finger to control a cursor, and a touch sensitive pad for selection and initiation of the device.

I would not be surprised if Apple did not create a ring input device as an addition to other existing and future devices. It could work with Apple TV, your Mac, iPad or a future glasses project. With their design and engineering flair they could make it usable, discrete and fashionable.

I see a future where we will wear multiple connected wearable devices. Selecting the best of breed for each situation. Just as we can currently pick the mouse and display that fits our needs and budget.

Update:

Another entrant into this area. These are just concept designs form a competition on crowdSpring but even if this device does not eventually appear I have no doubt that rings will be important input and control devices. They are small, personal, unobtrusive and can be used without making large gestures.

Leap Motion a flawed and isolated advance

I was one of the people who funded Leap otion on Kickstarter. I had been wowed by the potential of the product. I put my money down and waited in eager anticipation. I loved the idea of an almost magical way of interacting. I dreamt of the way it might, might, help my daughter with disabilities interact with computers. She has mastered the iPad - the biggest advances in computing for those with special needs-  and I hoped that this might provide a new way for her to use technology.

I have been using Leap Motion for over two weeks now. The best two words to sum up my experiences are, disappointed and tiring. High expectations are always hard to deliver on, but in the case of Leap Motion its biggest problem is not to do with the technology or the hardware. Granted there are only a limited number of useful applications in the Airspace store. The ones that work best, to my mind are the simple games where the accuracy of the device is not as important as the gestural interaction. When you want to really interact with your computer, when you try and use Leap Motion as a replacement for the mouse it falls down. The problem is the lack of touch. Using the device to gesture to move things around or change views is fine but if you want to select, or attempt to move items with fine control, the lack of haptic feedback is a real problem.

Selecting requires a steady hand and the need to keep it in place to indicate that the item you are pointing at is the one you want to select. This requires the unnatural behaviour of holding your hand, pointed finger very still in 3 dimensions for a few seconds. Touch based interfaces faciliate near instant selection. But more importantly they provide a surface which your finger(s) can push against and rest upon. Keeping your hand of fingers resting in thin air is very unsatisfactory. If you try and do this for anything but a short time you find gestures and selections make you arm tired. The old human computer interaction term, coined in the 80's, "Gorilla Arm" applies. This was originally used to describe the fatigue induced from extended use of vertical touch screen interfaces, but it sets in even earlier where there is no surface to resist or use to gain some support from.

So, rather than be a device I use habitually, it has already become another novelty article I pull out to briefly wow others, and then put away do real work. Another technology which fails to work for my daughter. A bold and interesting technology, but fundamentally flawed. So what is the real future of interaction? How will we interact with devices in ten, twenty years time? I suspect there will be two. One is already here and well established, the other is in its early stages.

Touch based devices will become increasingly more common. We forget that decent responsive, natural touch screens have only been in the consumer market for a few years. Prior to the first iPhone most touch screens used resistive panels as still used in some cash machines. They required you to actually register a push and only allowed single finger interaction. The iPhone brought in high resolution capacitive screen which supported multi touch.  While the technologies to deliver these intuitive experiences are several decades old, the commercial versions in smartphones are already so common they seem old hat.  Touchscreens in devices and objects will continue its rise in use against the old guard of the mouse. As tablets and internet of things devices proliferate, these will exceed those devices which still use a mouse or track pad. However despite the success of IOS and Android, touch user interfaces are still subordinate to the mouse in highly functional tools such as professional image and video editing. The mouse has been around since 1965 and still has life in it for at least another five or ten years.

Speech interaction paradigms will grow in complexity and ability to work as effective natural language interfaces. Speech input and feedback is an odd method for use in public spaces and while it possesses the potential to allow complex instructions, is useless at "house keeping" interaction. Asking a computer to move a window from one position to another or to swipe a virtual paintbrush across a screen along a given path, is not where speech input works. But instructions which contain meaning, "where is the nearest restaurant which my friends recommend", "find when my doctor can next fit me in",  "when is my current account likely to go into overdraft", are more useful scenarios. But these are not excel or word type applications. And here is the problem.

When predicting the future is that we tend to see it in terms of today's capabilities. In these predictions I am grounded in the historic models of the desktop and newer tablet application model, and the latest siri and Google now and Google glass models. As we have seen in the past, new interaction approaches often come as a part of new interaction paradigms. Command line gave way to direct interaction because of higher resolution displays, graphical interfaces and the computer mouse. Without the combination of these elements we could not have had spreadsheets or WYSIWYG word processing and later desktop publishing. At the same time it did not make command line programming impossible, just less usable and effective when compared to the new alternatives.

Until recently we interacted with computers at the desktop, they were go to devices for specific uses. The world has radically changed, we now use smart phones, tablets, connected devices, as well as the PC. New interaction models are appearing, most focusing on context.  Many contexts are focused around direction and making a computerised device provide a service in the form of information, or rudimentary functionality. Google Now is based on a lack of explicit interaction, on an anticipation model based on individual and aggregated big data. While this may work in environmental and lifestyle applications, they fail to hit the mark in productive situations where we generate new content or documents or rich creations. The traditional territory of the PC. Tablets try and provide some tools for content production, but they have yet to reach the productivity levels one can achieve with a mouse at a PC. Kinect shows how one may interact with a living room computing system. Again, applicable in one context but not another.

Leap Motion has it tough. Put aside the flaws I have already identified. Leap Motion is trying to slot into an existing system, rather than the much, much harder task of creating a complete new paradigm. It may be that my disappointment with it may stem from the fact that it does not fit, and it is unclear what the supporting technologies are required to create a new effective paradigm. One where, like the mouse,  Leap Motion is just one element. A new model where new forms of applications will be discovered which may transform our ways or working as much as spreadsheets, word processing and the like have done in the past.