Testing and the Art of Mobile ALM
Expanding the application lifecycle management platform to include mobile.
Expanding the application lifecycle management platform to include mobile.
Mobile has changed everything. Smartphone users don't live, work, or play the way they did before holding that little powerhouse in their hand, and very few would want to go back. It's a new world, indeed.
So, too, it is for the enterprise trying to keep up with its fast-moving but always-connected employees, customers, and partners. Mobile is demanding changes to the way enterprises conceive, build, test, deploy, and maintain their applications and websites. It’s a disruption that has been brewing for years, and yet many enterprises still have not fully sorted out how mobile fits — or doesn’t — into their existing application lifecycle management (ALM) process.
Putting a mobile ALM strategy in place is an increasingly urgent priority for a variety of reasons. Revenue impact is at the top of the list, in terms of sales, customer satisfaction, and employee productivity — with every passing day, a satisfactory mobile experience becomes more critical.
For IT and development teams, effective mobile ALM is a practical necessity. Gartner predicts that as soon as 2015, mobile app projects will outnumber PC projects by a ratio of four to one. 1MobilityWatch, “Are Agile development & mobile apps the perfect match?” by Steve Habermas, 7/29/13 And to support an agile development process for those projects , a viable mobile ALM solution is an absolute must, given how rapidly and continuously markets and functional requirements are changing .
Gartner defines application lifecycle management as “the practices, processes and tools that aid in the management of the application lifecycle, specifically the workflow of producing or maintaining an application." 2“MarketScope for Application Lifecycle Management,” Jim Duggan, Matt Light, Thomas Murphy, Gartner, Inc., 12/17/08 It’s a broad concept that includes just about everything involved in producing an application — all of the organization, all of the process data, the testing and QA, revisions, and the record-keeping and accountability for the whole project.
It’s important because ALM helps to unify geographically dispersed teams, and helps them to meet deadlines that keep getting shorter and shorter. Properly implemented, ALM brings agility, efficiency, consistency, accountability, and greater quality to the entire process. But it’s a process, or platform, that has until recently been focused primarily on traditional desktop applications.
The mobile complication: Testing
Many of the workflows, processes, and milestones are the same for mobile as they are for any other app development project. But there are key differences that make integrating mobile into existing ALM a challenge, particularly in the area of testing.
One of the most significant differences that impacts testing is OS and device diversity. There are hundreds of OS-device permutations in mobile, compared to just a few for desktop, and that has tremendous implications for testing and the ALM practices needed to manage it. Another testing complication is the diversity of user input types, vs. the simple keyboard and mouse input for desktop. User interaction is typically more complicated and therefore harder to test, as it can include various finger gestures, voice, camera, GPS, barcode data, and other inputs.
Another complication is that mobile devices spend a lot of time connected via cellular networks, which has its own set of implications in terms of data usage and performance.
An effective mobile ALM platform needs to accommodate the three types of testing that come into play in the mobile lifecycle: emulation, manual testing, and automated testing. Most experts agree that emulation has only limited use in the mobile testing cycle — for developers to explore what a change or feature might look like, or to help create automated test scripts to run on real devices. Manual testing, sometimes called interactive testing, has made up the bulk of mobile testing until now. But with the increase in release frequency of mobile applications, enterprises are moving towards automated testing solutions.
Manual testing involves real human beings holding the devices in their hands and using the app. It is most useful for feature testing — just making sure the “new thing” works; for exploratory testing, adopting the mindset of the user and trying everything out to see how it works; and negative testing, trying to break the app.
“Manual testing does serve an important purpose by providing a mechanism for obtaining crucial ‘usability’ feedback for the app,” writes Leigh Williamson, an IBM Distinguished Engineer and member of the IBM CTO team. “Usability, or user friendliness, is even more important for mobile apps than it is for other kinds of software. And it is pretty hard to get an idea about how usable your app is without a real human being interacting with it on a real physical device. 3The Invisible Thread, “The economics of mobile application testing,” by Leigh Williamson, 6/30/11
There comes a point very quickly, though, where testing is too extensive and involved to be conducted by human beings holding smartphones and tablets. That’s where automated testing comes in.
“A lot of the work that goes into testing every release is regression testing, because you need to validate all the functionality you already had,” says Rachel Obstler, senior director of product marketing at Keynote Systems. “You’re building on top of an existing product. Regression tests are always the biggest targets for automation, because you run them for every release and typically multiple times within a release. You’re going to do them over and over again.”
Performance testing that’s unique to mobile.
Functional testing makes sure the app works with the desired level of quality. But once functional integrity is established, a number of performance characteristics unique to mobile need to be vetted as well, and this has to be done on real devices.
“There’s a whole range of performance testing necessary for the app on the device itself, where you’re trying to understand how well or not well the mobile application is using the resources of the mobile device,” Williamson says. “There is a bunch of stuff that you can do badly. The application may be functioning perfectly fine, but it’s keeping the connection to the network open too long, it can drain the battery down because the radio chip is running too much…or if your application is not set up with a really long time-to-live for the cache, you’re going to be reloading the same data more and more frequently and that’s going to suck up the user’s cellular data plan and they’re not going to be happy about that. So there’s all of those performance aspects of the app on the device itself that are important aspects that are unique to mobile.”
Managing real devices is a real headache. It’s vital to test on real devices. But no matter how large the organization, it’s a daunting task to manage a pool of test devices in-house. Just between iOS and Android, there are literally hundreds of OS and device combinations. And Windows Phone and BlackBerry add incremental complexity.
With new devices being released practically daily, it’s a constant challenge to keep a sufficiently representative inventory of phones and tablets available and on active contracts. And it’s impossible to give geographically dispersed team members equal access to the device pool.
“It’s an unrecognized cost and complexity to manage all of those devices for the purposes of testing,” Williamson says, “and to ensure that you get maximum utilization of those test devices across everybody who’s going to need to perform testing.”
This is a problem that solutions like Keynote DeviceAnywhere can solve. Enterprises can have access through the cloud to a pool of the latest devices for testing, accessible to team members anywhere there is an Internet connection. Options include private device pools in the cloud or on-premise when security is a paramount concern, but all of the same automation and accessibility features apply.
Rolling mobile testing into ALM
Enterprises are of necessity beginning to more tightly integrate mobile testing into their ALM process, and an effective way to do that is to use a mobile testing solution such as Keynote DeviceAnywhere in conjunction with enterprise ALM tools such as HP ALM/UFT and IBM Rational Quality Manager. Such combinations enable enterprises to leverage the work that they have put into automated testing and immediately apply it to mobile.
For QA engineers, this means they can access all the devices in their pool, and operate them as if they were holding them in their hand: pressing buttons, pinching and swiping, sending and receiving messages with other devices, downloading and installing apps — everything.
It is difficult to overstate the advantages of an entire team having access to all the devices, all the testing, and all the details, all the time. It’s more efficient, troubleshooting is more effective, there is less duplication of effort, and it provides the data for better decision-making. Even for manual testing, when it would be tempting to use a local physical device if it were available, it makes sense to opt for testing using a pool of real devices in the cloud.
“The benefit of doing manual testing of any sort within a system like ours is that, one, you actually record what you’re doing,” Obstler says. “That means you’re halfway towards getting it automated the next time you want to test it. Another benefit is that all the screenshots are being saved, so if there is a problem, it’s much easier to put together a bug report or to send someone a screen shot that shows them the problem.”
Mobile ALM for agile development and testing The pendulum is swinging with more and more momentum toward a mobile-dominated development process. Enterprises can stay ahead of that shift by leveraging the tools and skill sets they have in place now for desktop ALM, and integrating the critical testing functionality for mobile. Testing early and often on real devices is the key to creating apps that address the needs of users, deliver a satisfactory (if not delightful) experience, and perform to expectations day in and day out. Automation is the key to successfully delivering with both quality and speed.
“With mobile, you have a lifecycle that requires you to be more agile, which means faster, and to react to changes,” Obstler says. “But at the same time, you have to test more platforms than in the desktop world. So you’re squeezing it on both ends — do it faster, but test on more devices. That leads you even more so than in the desktop world to look at how you automate.”
Williamson sees automation reaching beyond testing to cover more of the development process.
“We do have an idealized view of the mobile application development lifecycle and this would be a process that’s highly automated,” Williamson says. “We think more and more, you’re going to see these DevOps capabilities brought to bear. Developers will still be producing the creative innovations just like they do today, but the activities that come after they produce the code — the activities that process that code like build it, test it, scan it, deploy it, and so forth, will be all highly efficient compared to how things are done today. You can really see this trend accelerating all around, not just in mobile, but mobile will really get it because of the time-to-market expectations placed on mobile apps.”
It’s clear the future is mobile. So there’s no time like the present to get a robust mobile application lifecycle management platform in place.
To learn more about mobile testing click here. To get a personalized demo of Keynote’s ALM integration with IBM Rational click here. For a personalized demo of Keynote’s ALM integration with HP UFT click here.