Mobile App Performance Measurement: Thinking Inside the Box
By Product Management | June 17, 2013
Web application performance monitoring is a stable and sophisticated beast. There are tools out there—single systems right out of the box—that run every critical performance and functionality test web apps need. But let's look at what they test—and what they don't.
What they monitor is how browsers and servers get along. They send a request, wait for the requested page to load, and then report on that interaction. They keep you up to date with waterfall graphics and timeline videos. If they spot a problem, it's analyzed and alerts are generated. Great stuff: everything that happens between request and response watched like a hawk.
What don't they test? Essentially, anything that has to do with the computer or the mobile device that launched the browser. To them, the device is just a black box. Their job is to test outside that box.
Mobile apps are a whole different kind of animal. Now, the device is very much a concern: your code's running there. That device isn't just launching a browser. It's where your application is executing. And you need complete visibility into how well it's doing.
It's a new kind of toolkit that monitors mobile apps: that provides this visibility. One that tests how many connections your app makes, how long it holds them open, and how much data it downloads. That monitors how well your app uses native functionality, like cameras. That tests login and logoff. That monitors HTTP and API calls, and cache and buffer management, and that tests for the full range of user interactions—swipe and pinch and tap—that your apps use to deliver the best user experience.
A toolkit that tests your application where it runs—on real smartphones and tablets.
Inside the box.
Just the logistics are daunting. Mobile apps don't enjoy the luxury of write once/run everywhere. For you to make sure your apps perform well, you've got to know how they work during rush hour on the iPhone 5 in Tokyo and the Samsung Galaxy S4 in New York.
So, that's what I'm going to talk about in the upcoming blogs: the toolkit that tests your application inside the box.
I'll talk about scripting. It's a lot more intricate than web app scripting: you're not just simulating browser "clicks and types": you're running a live application.
I'll talk about the snapshot view: the tool that tracks and graphs the performance of your app at any instant and over time—and presents everything in an online portal.
I'll talk about the detailed view: the tool that opens visibility into application details at a network level. This includes what connections were made, how long they were maintained, and the impact of expired cache performance.
And I'll talk about the video timeline view: the tool that plays back a video of each test so you can see exactly what was happening when an issue occurred.