Mobile is the New Mandatory
Success-minded businesses are already there, but only a minority have tested their apps or sites.
"The old model – a stationary customer sitting at a stationery desk – no longer applies." That's how The New York Times tersely describes the dilemma "old school" companies like Intel, Google and Microsoft are facing as they try to transition their old, desk-bound revenue spigots to the new mobile world. 1The New York Times, “In Mobile World, Tech Giants Scramble to Get Up to Speed,” by Claire Cain Miller and Somini Sengupta, 10/22/12
It's hard to think of any new technology that has had as swift and profound an impact as the smartphone. The Web took a decade or more to achieve a truly transformative effect. But it took just five years for smartphones to land in the hands of more than 100 million Americans – fully one-third of the total population, and nearly half of adults – and the pace of adoption is still accelerating. Worldwide, smartphone users just passed the one billion mark, and it's projected to take just three years to reach the second billion. 2ReadWriteWeb, “Know What’s Cool? A Billion Smartphones. And They’re Changing Everything,” by Dan Rowinski, 10/17/12
A slew of statistics validate the importance of mobile channels, from time spent on devices to data consumption to shopping behavior and on and on, and the numbers keep moving aggressively in a positive direction. So it begs the question: Why is it taking so long for businesses to get serious about mobile? And why is it taking those who do have a mobile presence so long to organize a testing program to make sure their apps and mobile websites work the way users expect them to work?
Mobile MIA, mobile missteps
Some observers estimate that half of the Fortune 500 do not have a dedicated mobile presence. In an Internet Retailer survey, almost 58 percent of the top 300 online commerce operations reported that they have neither a mobile app nor an mobile commerce website. 3Internet Retailer Mobile Commerce Top 300, “Internet Retailer Survey: Mobile Commerce,” pp 38-39, 2012
It's baffling that so many businesses are dragging their heels to stake their mobile claim, when consumers are so enthusiastically embracing mobile connectivity. It's perplexing, too, that those businesses that have taken the mobile plunge are doing it with their eyes closed. One study found that, worldwide, just 31 percent of respondents test their mobile applications; in North America, it's just 25 percent. Nearly two-thirds report not having the tools for testing, and more than half do not have access to the devices needed to check their applications. 4World Quality Report 2012-13, “Mobile Testing: Behind The Curve,” Capgemini/Sogeti/HP, 2012
Lack of proper QA in mobile is in some ways understandable. Some businesses may be rushing their mobile properties out the door because they do feel the pressure to be where their users are, and where their competition may not yet be. And the reality is that mobile testing is a much more messy and complicated process than testing for desktop browsers or operating systems.
Too many phones!
For practical purposes as of this writing, the smartphone world is coalescing around the iOS and Android operating systems; Blackberry is an ever-diminishing factor, and it's wait-and-see for Windows Phone. Yet within those two major camps, the permutations for what could actually be in a user's hand are mind-numbing.
There are more than 300 different Android phones worldwide. There’s some version of a sweet tooth-satisfying operating system on each, whether it’s the latest Jelly Bean, last year’s Ice Cream Sandwich or Honeycomb, or the now-stale Gingerbread or one of the five or so earlier OSs. The phones have varied screen sizes and different resolutions, processors, etc. It’s a sticky mess.
“The matrix of permutations is so incredibly huge that it’s just fundamentally challenging,” says Rohan Chandran, head of consumer products for YP, a huge local search, media and advertising company that offers apps on all the major platforms. “And I’m pointing the figure squarely at Android, to be honest.”
“It’s a lot of the very marginal differences that are there in handsets,” Chandran says, “Particularly with Samsung and HTC – they have their own skins and layers on top of the OS. Very often we run into a device that’s ostensibly the same version of Android, that has exactly the same screen resolution and everything else, and we’ll go through it and notice behavior that’s different. The widget won’t size correctly on a particular device because of the skinning that one of the OEMs has put on it. Or some text will somehow wrap around a particular feature, where again, theoretically, everything on that device is identical, but something in the skinning has had an impact that we can’t define and we can’t predict. The key problem for us is that it’s not predictable.”
Even the Apple ecosystem has grown more complicated, with multiple iOS versions in use, retina and non-retina displays, and now the larger iPhone 5 and smaller iPad. It’s easy to understand how daunting it can be, when a team is ready to test a mobile app or website and finds itself staring at an overwhelming array of device variations. But the choices and the process can be simplified.
A little planning keeps the devices at bay
“A lot of people, when they’re selecting devices, either go by the top set in the market or by data from their application analytics,” says Rachel Obstler, senior director of product marketing for Keynote’s DeviceAnywhere platform. “That’s fine, but it may not be the most efficient way to get coverage. If instead you can identify the likely parameters that could affect your application – for instance, OS version, screen resolution, manufacturer – you can arrive at a test panel of devices that may not be the same as the top 10 you’ve identified, but that get all your parameters covered. You reduce risk by criteria, not by the specific phone make and model itself.”
The reality test
Why all the sweating over devices? Because real devices are simply the only reliable way to test. There’s just no way to emulate with software the device permutations and effects of user interactions on a site or app, not to mention the vagaries of cellular network connections.
So mobile testing rule number one is: Test on real devices. Few operations, though, can manage acquiring and maintaining enough devices and contracts on enough carriers to fully vet their mobile website or app. For initial builds or development, testing on the devices scattered among staff in the IT or operations department may suffice. But for adequate marketplace testing of release candidates, the services of an outside partner with a deep device pool and proven testing infrastructure are indispensible.
“We have, at any time, half a dozen different capabilities that are under development,” says YP’s Chandran. “And we’re unit testing those individually on each platform as they go out. We’re testing those continuously and then, at any point in time, we may decide OK, it would make sense to draw down from these half a dozen tracks and four of them will come together. And we can actually bundle a release that would go out to market and at that point, we’re doing really comprehensive testing.
“And that’s the point at which we start to leverage, for example, the DeviceAnywhere platform, where we’ve got to be really thorough for the experience across all the potential devices that are in our audience out there.”
No exceptions for enterprise mobile
Mobile has been a hard nut for corporate IT departments to swallow, because it means they have to compromise on one of the things they hold most precious: control. Life is a lot simpler when workforces are armed with centrally controlled Blackberrys – what devices are used, what goes on them, and data security practices are strictly managed. Testing is much more manageable, too, with a small, defined pool of devices, carriers, and applications.
The iPhone changed all that, ushering in the BYOD (Bring Your Own Device) era, as employees demanded the usability, functionality, convenience and sometimes even fun that the new touchscreen smartphones enabled. You can still see the occasional worker in Starbucks juggling the corporate Blackberry in one hand and the personal iPhone or Android device in the other. But progressive organizations have opened their doors (within reason) to employees bringing their own phones for official work use.
On the one hand, it’s a plus for the enterprise: they don’t have to buy devices and their employees are happier. On the other hand, they now have to validate their enterprise app or website’s functionality on a menagerie of devices. As discussed above, it quickly becomes unmanageable to gather and test a wide-ranging pool of devices internally. A dedicated testing partner solves that problem.
“Companies really appreciate our shared system for ‘untrusted device’ testing,” Keynote’s DeviceAnywhere Obstler says. “When an executive wants to carry a particular device, IT can log into our test system and test it and make sure everything works according to the company’s standards. Since they’re testing on our shared system, the device is virtually ‘in the wild,’ outside the corporate ecosystem, just like the device an employee would bring in on their own. So IT can make sure the company’s data and apps are adequately protected.”
Building something to test: A website, an app, a hybrid
It’s clear that, to be competitive, just about every type of enterprise – consumer-facing, B-to-B, as well as corporate internal in many cases – needs to have a robust, well-functioning mobile presence. But what about the half, give or take, of businesses that don’t have any dedicated mobile presence at all? There are three basic paths to choose from to get started.
Native app. These are built for a specific OS – iOS or Android, typically – and live on the device. They offer the best performance, a true native experience, and access to all the device’s functionality, including the camera, GPS, accelerometer, contacts, calendars, etc. On the minus side, they are usually the most expensive to develop – a separate app has to be built for each OS – they are available only in an app store (and therefore are not discoverable by search), and they are the most challenging to test.
Mobile website or HTML5 app. HTML5 was heralded as the magic that would allow developers to build websites that would work on any and every platform. While some of that promise has been realized, it’s not proven completely to be the hoped-for panacea. It does generally work well, though not without the need for tweaks to ensure proper functionality across many devices. HTML5 can be used to make a cross-platform mobile website, or a mobile website that masquerades as an app. Its advantages are that it is cheaper to build; largely cross-platform compatible; easy to update; available through any browser, and therefore discoverable through search; and easier to test than apps. The downside is HTML5 mobile websites are slower than apps and cannot leverage all of the device’s functionality, such as camera, GPS, etc.
Hybrid. Is it an app or isn’t it? Sometimes only the developer knows for sure. A hybrid app takes an HTML5 website and wraps it in a thin native wrapper. It attempts, with some success, to combine the best of both worlds: the easy updatability of an HTML5 site and some of the speed, offline capabilities, and native functionality of an app. It’s more affordable to build than a fully native app; is distributed through app stores; and requires similar testing to a native app.
Build it, test it, and they will come
For any business or organization that interacts with anyone online, mobile is mandatory. It’s not a nice-to-have, not something to put in next year’s budget. It’s where users expect to find you. And for a mobile site to be viable, whatever shape it takes – app, website, Web app or hybrid – it has to be thoroughly tested on the real devices users hold in their hands.