Smartphones - what ever happened to ergonomics?

Smartphones have become the swiss army knife of the digital age. The expression, "There's an app for that", describes the seemly endless diversity of functions you can use your smartphone. While many of us have dozens of apps most of use very few. Mostly we habitually use a core set which allow us to communicate, discover and record. These three categories also describe three modes of holding our phones.

To communicate we can phone, email, text, or post to social media. To phone we tend to hold the phone up to our ear in the traditional, making a call mode. Despite siri and the improvements in voice to text dictation, the other communication modes are usually done with the phone in one hand at approx 18 inches from our face. Frequently we do this one handed.

We surf the web and discover information using the same single handed mode. When possible we will use two hands for this task, but frequently we are in situations where one hand is more convenient.

Finally we use our smartphones to record our lives in the form of photographs and video. This tends to be a two handed operation, but again it can sometimes be done single handed.

While this is what we want to do with our phones their design is driven by the need to improve the technology that allow these functions to happen. How we actually physically hold the device is not a focus of improvement. What ever happened to ergonomics?

Pick up your iPhone, Samsung, HTC, Sony or any other new smartphone. Their design is primarily focused round the conflicting goals, of incorporating as large a high resolution screen as is practical combined with as much battery life as possible within the slimmest size. The camera, additional hard switches are added in there, but their location and utility is of lower consideration than the screen/battery compromise. The whole package is encased in materials which say as much about the brand values and social statement as the functional needs of the device.

Coupled with this is an ever shortening product cycles. New models come out every year. The core technology increases in capability, the camera gets more better, the pixel density gets higher and the screen seems to inevitably get larger, and the products get more anorexic. The product cycle is more driven by the same forces that gave us ever increasing fins on the American automobiles of the 1950's. It makes last years, or hopefully the competitor's current phone seem, so yesterday.  At no stage in this headlong rush does the ergonomic failings of the slim cuboid seem to be addressed.

When looking at our high definition, retina displays we do not actually hold our phones, we cradle them in our hand balanced on our pinky. One of our most personal and valuable habitually used possessions is balanced in a precarious position. All the people who we see with cracked screens stand testament to this. The larger the screen the slimmer the form factor the harder this single handed use becomes. If you try and grip your device tightly and then use your thumb to select things on screen it quickly becomes tiring. You need a relaxed grip to comfortably use your thumb.

When we take pictures we are holding a thin slab, with none of the ergonomic enhancements actual cameras have, no hand grips, or ridges to make griping easier. Not even a tactile surface which offers some friction to aid grip. And when we hold these slabs to our ears we increasingly need to stretch our fingers to grasp the device.

The software attempts to make concessions to the ergonomic problems which Steve Jobs so rightly pointed out at the iPhones birth. The arc of reach of the thumb is a limiting factor in the size of the display. So now the iPhone 6 has an software trick which brings the top of the screen within reach.

Jonathan Ive is a very clever man, he is a leader of the cult of the design object. Beauty in the materials and details of the finish. Shaving a tenth of a millimeter here, a new radius there. But for all his passion about making the perfect product he rarely makes phones which are holdable in use. The original iPhone and up to the 3s were much more ergonomic devices. But as the phone has evolved and grown to today's iPhone 6 Plus they have increasingly failed in terms of ergonomics. It is not an easy task. Can a multi capable device have one form which satisfies all uses? Maybe not. But which single use do our current phones actually do well.

The internet of me

There is an old gag that on leaving the house a man makes the sign of the cross. He is not religious, it stands for: spectacles, testicles, wallet, watch. A quick check to see they have all the essentials items a man needs to be a man.

Today’s version would have to include the phone. It is far more vital to our daily existence than any of the others. In fact increasingly we are now leaving home with a digital ecosystem which defines what it is to be a digital citizen. Each item no longer exists in isolation. Our smartphones are the key, the hub for all our new devices and has the potential to replace at least one of them, the wallet.  This trend will only continue.

As we adopt wearables we move from having a single digital device to a collection of independent but symbiotic internet devices. Our wristbands and smart watches can act as information conduits but also collect data, and feed it back to our smartphone. They will assess our health and wellbeing. Some of the work in deciding what to do with the data will take place on the device but most will be done on our smartphone apps talking to external health databases and where appropriate to our health providers and insurers.

Smart glasses will augment our experiences, again using the smartphone as the conduit to the external data sources. Image recognition, location services, pricing comparison, social communication will all travel through the smartphone hub to our personal internet of devices.

A lot of the time this will result in automatous actions that do not require input from the owner. Where you are, your schedule, your habits, your diary, your needs will be assessed to make decisions on your behalf. Within your own ecosystem of physical devices and the wider personal information stored in countless cloud services rules will run and instructions will be sent out.

While we are still awaiting the impact of the Internet of Things we are starting to see the Internet of Me. A world where devices are networking and making decisions for my benefit. Based on my very personal needs and state. The internet of me means the power of my network of devices is greater than of them individually.  If a new device comes along or I upgrade my health monitoring device then the other devices in the network could have access to this new data and functionality. The combination of the data from more personal sources makes the decisions which can be made more accurate, wider reaching etc.

Google always have, and recently Apple have been opening out their platforms so that third parties can  build solutions that will easily communicate and co-operate. For the time being we have to be in one or the other camp, but device makers will profit from supporting both. The added capabilities and ease of collaboration will determine what is capable on each platform.

Commercially it will open a whole new market, where tomorrow’s Apples and Facebooks will be born. Certain device combinations will hit the sweet spot and deliver killer apps. A man will not be judged by the cut of his suit but the configuration of his internet.

Socially, The Internet of Me will further widen the digital divide. Already just having access to the internet gives an advantage over those without access. If we extend this to the Internet of Me then the individual with their own ecosystem has a definite advantage over one who does not. They may be able to get access to beter healthcare or lower insurance premiums. They can have immediate access to greater amounts of contextual information. They are more likely to be targeted with promotions and offers and make savings.  How will society and governments deal with these digital inequalities, or will they even try?

So what is the new sign of the cross? Smartphone, smartwatch, ARglasses, biometric earphones, and not forgetting testicles. Somethings never change.

Leap Motion a flawed and isolated advance

I was one of the people who funded Leap otion on Kickstarter. I had been wowed by the potential of the product. I put my money down and waited in eager anticipation. I loved the idea of an almost magical way of interacting. I dreamt of the way it might, might, help my daughter with disabilities interact with computers. She has mastered the iPad - the biggest advances in computing for those with special needs-  and I hoped that this might provide a new way for her to use technology.

I have been using Leap Motion for over two weeks now. The best two words to sum up my experiences are, disappointed and tiring. High expectations are always hard to deliver on, but in the case of Leap Motion its biggest problem is not to do with the technology or the hardware. Granted there are only a limited number of useful applications in the Airspace store. The ones that work best, to my mind are the simple games where the accuracy of the device is not as important as the gestural interaction. When you want to really interact with your computer, when you try and use Leap Motion as a replacement for the mouse it falls down. The problem is the lack of touch. Using the device to gesture to move things around or change views is fine but if you want to select, or attempt to move items with fine control, the lack of haptic feedback is a real problem.

Selecting requires a steady hand and the need to keep it in place to indicate that the item you are pointing at is the one you want to select. This requires the unnatural behaviour of holding your hand, pointed finger very still in 3 dimensions for a few seconds. Touch based interfaces faciliate near instant selection. But more importantly they provide a surface which your finger(s) can push against and rest upon. Keeping your hand of fingers resting in thin air is very unsatisfactory. If you try and do this for anything but a short time you find gestures and selections make you arm tired. The old human computer interaction term, coined in the 80's, "Gorilla Arm" applies. This was originally used to describe the fatigue induced from extended use of vertical touch screen interfaces, but it sets in even earlier where there is no surface to resist or use to gain some support from.

So, rather than be a device I use habitually, it has already become another novelty article I pull out to briefly wow others, and then put away do real work. Another technology which fails to work for my daughter. A bold and interesting technology, but fundamentally flawed. So what is the real future of interaction? How will we interact with devices in ten, twenty years time? I suspect there will be two. One is already here and well established, the other is in its early stages.

Touch based devices will become increasingly more common. We forget that decent responsive, natural touch screens have only been in the consumer market for a few years. Prior to the first iPhone most touch screens used resistive panels as still used in some cash machines. They required you to actually register a push and only allowed single finger interaction. The iPhone brought in high resolution capacitive screen which supported multi touch.  While the technologies to deliver these intuitive experiences are several decades old, the commercial versions in smartphones are already so common they seem old hat.  Touchscreens in devices and objects will continue its rise in use against the old guard of the mouse. As tablets and internet of things devices proliferate, these will exceed those devices which still use a mouse or track pad. However despite the success of IOS and Android, touch user interfaces are still subordinate to the mouse in highly functional tools such as professional image and video editing. The mouse has been around since 1965 and still has life in it for at least another five or ten years.

Speech interaction paradigms will grow in complexity and ability to work as effective natural language interfaces. Speech input and feedback is an odd method for use in public spaces and while it possesses the potential to allow complex instructions, is useless at "house keeping" interaction. Asking a computer to move a window from one position to another or to swipe a virtual paintbrush across a screen along a given path, is not where speech input works. But instructions which contain meaning, "where is the nearest restaurant which my friends recommend", "find when my doctor can next fit me in",  "when is my current account likely to go into overdraft", are more useful scenarios. But these are not excel or word type applications. And here is the problem.

When predicting the future is that we tend to see it in terms of today's capabilities. In these predictions I am grounded in the historic models of the desktop and newer tablet application model, and the latest siri and Google now and Google glass models. As we have seen in the past, new interaction approaches often come as a part of new interaction paradigms. Command line gave way to direct interaction because of higher resolution displays, graphical interfaces and the computer mouse. Without the combination of these elements we could not have had spreadsheets or WYSIWYG word processing and later desktop publishing. At the same time it did not make command line programming impossible, just less usable and effective when compared to the new alternatives.

Until recently we interacted with computers at the desktop, they were go to devices for specific uses. The world has radically changed, we now use smart phones, tablets, connected devices, as well as the PC. New interaction models are appearing, most focusing on context.  Many contexts are focused around direction and making a computerised device provide a service in the form of information, or rudimentary functionality. Google Now is based on a lack of explicit interaction, on an anticipation model based on individual and aggregated big data. While this may work in environmental and lifestyle applications, they fail to hit the mark in productive situations where we generate new content or documents or rich creations. The traditional territory of the PC. Tablets try and provide some tools for content production, but they have yet to reach the productivity levels one can achieve with a mouse at a PC. Kinect shows how one may interact with a living room computing system. Again, applicable in one context but not another.

Leap Motion has it tough. Put aside the flaws I have already identified. Leap Motion is trying to slot into an existing system, rather than the much, much harder task of creating a complete new paradigm. It may be that my disappointment with it may stem from the fact that it does not fit, and it is unclear what the supporting technologies are required to create a new effective paradigm. One where, like the mouse,  Leap Motion is just one element. A new model where new forms of applications will be discovered which may transform our ways or working as much as spreadsheets, word processing and the like have done in the past.

Predictions for 2013

Windows 8

Windows 8 will be seen as a failed digital mashup. The Jeckle and Hyde nature of attempting to combine two UI paradimes does not resonate with users. Those that upgrade find they live in either one UI or the other, but rarely switch between the two. The old school Windows users find it does not significantly improve Windows 7 and removing the start menu frustrates many. Microsoft eventually relent and launch a campaign telling users how to reinstate the start menu. Those that looked forward to the bright new world of touch are equally disappointed by the limited functionality of the core apps, and the low number of 1st rate apps available in the app store. The effort required to learn the charms metaphor or hidden functionality proves too annoying a hurdle for those used to the iPad's on screen approach. The minimalist design attracts while the minimal functionality confuses.

As a result PC sales continue their decline. Hundreds of millions of copies will be sold via OEM sales and a trickle of expensive touch enabled laptop hybrids will cater to the business market. Despite Microsoft's best efforts Windows 8 and Surface and the undelivered promise of Courier will further hasten the decline from what was once global dominance.

Second Screen

Sports will fuel the second screen revolution. The use of the second screen as an integrated supplement to TV broadcasts has started to arrive in the US but has yet to make significant inroads in the UK. 2013 will be the year when it goes mainstream. Data rich events such as sports events are the ideal content  for the dual screen approach.  Sky will continue to lead the charge. They will extend the functionality of their second screen enabled iPad apps to include data on a wider range of sports. There will be a closer integration with social media, and the ability to share realtime self commentary with your social circle. More and more people will augment their viewing experience of live sports events both at home and in the stadium by using a second screen. Making sense of all this data and deciding what stats to display and the level of information required by different user types will prove a significant UX challenge.

3D rights

Staples are set to launch a 3D printing services in two European cities with plans to roll the service out over the course of the year. But in order to print people need models. 3D modeling is still a specialist skill, even tools such as Google Sketchup require a significant effort to become adept at producing complex printable models. The 3D clip art market will expand as witll portable scanners. 3D smartphone based scanning apps such as 123D Catch, will proliferate. Now with a  smartphone and an app anyone can create a facsimile of a 3D object. Admittedly the intial offerings will be low fidelity but 2^nd and 3^rd generation apps will make significant increases in quality. The mainstream press will latch onto the dire concequences of such a capability. Stories will abound about who owns the rights 3d rights to objects. Museums will have to consider if they allow visitors to scan their artifacts. Brands will try to protect the 3d rights of their products, as a whole and in component form.   Celebrities will charge for 3d files of their bodies. Questions will be raised in the house.

Bot Menus

Tesco and other data rich retailers will employ more proactive and intelligent shopping bots. Regular grocery shopping will automatically take into account calendar events, school holidays, festive periods, birthdays. Weather predictions will inform alternative choices based on personal and aggregate and regional tastes. Users will be able to specify how the shopping bot can assist them. Any shopping list can be analyised to deliver, cheaper, healthier, and ethical or celebration alternatives. Users will be incentivized to employ the bots by preferential prices on staple goods. Brands will have to pay the supermarket chain to be part of the schemes and to feature their products in the initiative. Some will go as far as to offer "surprise me" features for new menu selections.

Eyes in the sky

Low cost aerial drones fitted with video cameras will be the top geek presents for Christmas 2013. Just as remote helicopters have moved from the domain of hobbyist to the general public so will quad copter. In 2013 these four and six motor devices which range from the cheap and cheerful with a low power video camera to robust versions capable of carrying the owner’s own camera will find a mass market. As these will be designed for non commercial purposes they will escape the existing legislation. Even so privacy issues will be raised and the public will be wary of these toy eyes in the sky. Digital services will spring up to allow users to geotag their movies as we all get up close and aerial with our, and our neighbor’s world. Police and security forces will employ them in a more Orwellian fashion. In 2013 it will be the norm for strikes, public demonstrations and even small scale public events to be monitored from the skies.

Mobile is not an island

Forrester predicts the mobile wave will engulf us all. One billion smartphones by 2016. As smartphones increase in numbers so does their effect. Who betide the company that does not make ready for the coming deluge.

The mobile user is not a specific type of user, a distinct breed form the “standard” PC user. In many cases the mobile user is the user. They will only engage with you via their phone. For many the principle form of engagement will increasingly become via mobile. The principle analyst at Forrester, Ted Schadler makes the point perfectly:

“Mobile is the new face of engagement,” Mr. Schadler said. “Businesses should stop thinking about it as a small Web site on a tiny computer, and start thinking about mobile as being deeply embedded systems of engagement. That turns out to have huge implications.”

It is no longer enough, or even right, to think of the mobile user as only requiring a subset of the functionality and content of your standard digital offering. If the only way someone will interact with you is via a mobile device then that does not mean we should limit what’s on offer.

The mobile user does have hurdles to overcome, the UI has to accommodate the restrictions of smaller screens, in the case of IOS they cannot access Flash content, which while it may be in its death throws it is still a common delivery medium. The smartphone user has distinct needs and usage patterns. We need to accommodate their on the move mentality.

But the mobile user gives us so much more than their desk bound counterparts. We need to reward the knowledge they give us about their location and time of interaction. We need to deliver meaningful and useful services. In some ways they are a super set of the traditional user. We should deliver the full web site package and then some.

We should see mobile as part of the complete interaction spectrum. A spectrum which reaches from the actual engagement with the brand product, through the social consideration and interpretation of the brand through all forms of digital interaction. So while the mobile experience should not be the poor partner of the PC experience, neither should it exist in isolation from it. Interaction is a continuum where the historic engagements play their part in current and future actions and reactions. The digital offering which is not only cognisent of this history but adaptive to it is far more powerful than an island of interaction.

When developing today’s mobile strategy it should weave its way through all parts of the digital strategy and indeed, the complete marketing and sales strategy. Assigning responsibility for the mobile deluge to one department will not work.  As Karl Heiselman the author of a his report "Game Changers" puts it in a recent Fact Company article:

"Companies are only valuable if they prove themselves useful, time and again. Viewed with that lens, most companies come up short. "What’s interesting is that, on the client side, it’s never anybody’s job to own the customer experience," says Heiselman. "Some people think about pieces of it, but it’s nobody’s job to think about it in any kind of joined-up way."

Joined up thinking is such a tired expression, but from a user's experience they are dealing with one entity through many touch points and medium. They do not see or expect a mobile interaction to differ in quality and value from a marketing engagement or a sales experience or an after sales encounter. They increasingly expect the company or brand to know them and know about them and previous encounters. As mobile is taking a leading role we have to rethink our digital priorities and no longer consign it to a niche activity but make it a prime element in the spectrum of engagement. Make ready to take advantage and ride the wave, or else get soaked.