Keynote Tech Talks: Become a Mobile Testing Superhero with Automated Scripting
About the Webcast
For mobile testing teams, time is scarce and resources are stretched thin. The ability to write one test script and use it multiple times to validate content across multiple devices, on both iOS and Android, is priceless. Automated testing based on object-level scripting allows this kind of flexibility and provides testing and QA professionals with appreciable time and cost savings.
In this webcast, Darren Madonick, a Keynote test automation expert, discusses object-level scripting and how it can improve the efficiency of your testing operations.
This webcast includes:
- How to employ object-level scripting for automated testing of native and hybrid mobile apps
- Ways to quickly and easily build one script that works across all devices and on any given platform
- How web object-level scripting enables you to take advantage of powerful, time-saving tools such as the WebForm and the OpenBrowser commands to speed automation execution, providing you with full access to DOM, both real-time and in results, to analyze issues
Hello, and welcome to Keynote’s webcast on how to become a mobile testing superhero with automated scripting. My name is Josh Galde, and I am a Senior Product Marketing Manager for Quality here at Keynote.
Today’s live demonstration will be lead by Darren Madonick, a Customer Relationship Manager and automated testing expert here at Keynote.
Today’s webcast will help you understand first how to employ object-level scripting for automated testing of native and hybrid mobile applications. Second, ways to quickly and easily build one script that works across all devices and on any given platform. And third, how web object-level scripting enables you to take advantage of powerful, timesaving tools such the web form and the open browser command to speed automation execution, providing you with full access to both real time and end results to analyze issues.
These topics will be covered as part of our live demonstration later on in today’s presentation.
So, what’s going on in the mobile market today? What is this mobile app imperative we want to talk about today? Clearly users are demanding access via mobile. With today’s announcement from Apple, along with one billion apps being downloaded in 2013 according to Gartner. Second, Gartner predicts that there will be $77 billion in anticipated revenue generated through mobile apps by 2017.
Mobile is also the number-one technology priority for retailers in 2014. And two out of three banks predict that 100 percent of their customers will be using their mobile services by 2017. Whether we are talking about consumer apps or internal business applications, users want access on demand via their smart phones and tablets. And this is easily seen in the increase in traffic and sales in recent years.
And users have become not so accepting of poor quality of their mobile experience. They may have made allowances in the past, but now they expect sites that are designed for mobile and perform well for mobile. And the lack of this quality impacts sales and perception.
One survey found that only 16 percent of consumers would try a failing app more than twice before dumping it. Another study by Shunra found that app failures have cost businesses over $60 billion. And yet another survey said almost 80 to 90 percent of mobile apps were eventually deleted from users’ phones in 2013 alone.
With so much at stake, including in many cases one’s brand, testing these apps has become more than a nice-to-have. It has become an imperative in providing quality mobile applications to your users.
When developing a mobile app, you run into many headaches. You need to run it on major OS’s, on as many devices as possible, and you don’t have time to learn new technologies. You also need to be able to iterate quickly. Testing for mobile has truly become a moving target. Not only do devices change, but operating systems change, carriers change, phone specifications change, capabilities, features, browsers – everything is constantly changing. And guess what? With agile development your mobile app itself will need to iterate rapidly, which means – you guessed it – more change.
The picture you see is an actual mobile QA desk at one of our customers. This kind of unstructured environment is what many QA organizations are dealing with. So you need to be able to deliver and manage across all that dynamic complexity. Unfortunately there’s no time for your team to learn all these new technologies, and of course there’s not enough time to do all the testing you need. So there’s a lot of anxiety about whether the application will actually perform at scale when deployed in production.
To get started, we wanted to ask you a question. Just to gauge your experience in what you are seeing in the market, and also the challenges that you are facing. So first, here’s our question on the presentation, we want to be as interactive as possible. What do you see are the biggest challenges in testing mobile apps?
Please select all that apply. It could be device fragmentation, availability of skilled mobile testing experts – very common, implementing the right testing method process for mobile. Obviously there’s different processes and different methods to apply, and as mobile is much more complicated than desktop, it makes sense that this would be the case.
Availability of proper testing tools. Obviously there are different testing tools in the market from Selenium to device clouds, et cetera. Which one to use, how to use for which one, for which type of app, whether it’s native or web – it all can vary. You have access to real mobile devices to test on. Obviously you have devices that are sitting at your desk versus being able to use a remote cloud and having to shift devices around, or maybe not having the right firmware you need on the right device can be a problem.
And then lastly, having enough time to test is obviously – I think it could impact a majority of people, as this could always be the problem no matter how much testing you’re doing.
Looks like we’re getting some great responses. We’ll just give it another minute or so and we’ll go ahead and we’ll close the poll in a second. It looks great. It looks like we’re going to have a clear winner here.
Great. We’ll go ahead and close the poll and we’ll broadcast the results.
Excellent. It looks like availability of proper testing tools was the winner with 31 percent. In second place was implementing the right testing methods or process for mobile. Fantastic.
And this is very indicative of what we see in the industry as far as, you know the problems – the challenges that people face when testing. A lot of the problems that they run into do include this idea of not having enough testing tools – or the right testing tools, I should say, and being able to have something that can work across all devices. I think it’s been a big distraction, a big frustration for people when developing for mobile regardless of the app or how they’re deploying the experience.
So, we’ll go ahead and hide the poll and continue on the slides.
Excellent. Well, let me just talk a little bit about what Keynote does, because I know, you know, it’s important to address these critical issues and you wouldn’t be here today if you didn’t want to learn something about that. So we wanted to talk a little bit about that and then we can get into object-level scripting because this is a nice setup for that.
Keynote helps you solve these problems. First, we provide a global cloud-based testing environment with the broadest range of devices and testing capability on the planet. The picture shows our testing environment, which is replicated using service all over the world by contrast with the cluttered desk in the previous slide. Our global network is the most comprehensive in the industry, and we work closely with all major carriers so that you can test in every possible configuration of connectivity and platform.
Of course we know that you don’t use Keynote in isolation, so it’s critical that we integrate with and leverage your existing investments and development and testing tools from major partners such as SAP, Jenkins, IBM, and HP.
With using Keynote’s mobile testing, you get a common scripting environment for both quality assurance and performance monitoring so you can move an application from functional testing to monitoring reusing the same scripts to cross different parts of the organization.
And this is where test automation can be of the most value – the ability to run hundreds of tests repeatedly on a large number of devices.
And part of our discussion today will be on how we do that. And so we’re going to focus later on in the demo on our object-level scripting, particularly our object-level scripting with IOS. This is a newer version that we have that we want to share with you. We’ll be able to go into detail.
We also talk a little bit about Android as well and how you’re able to reuse these scripts for your functional and regression testing.
Before we get to that we just want to ask you another poll question to keep the conversation going here, so please feel free to respond as appropriate. When it comes to test automation – or mobile test automation and specifically testing mobile native applications – how much of that is done using automated scripting?
Would you say 0 percent, would you say 1 to 25 percent, would you say 26 to 50? That’s about mid-range. 51 to 75 is a majority of your testing when testing these applications is done using scripting, or I should say using automated scripting? Or over 75, meaning all of your testing is done – or you don’t know, and that’s actually a great question because a lot of times people aren’t familiar with some of the ability to script and that you actually can script a lot of this. Your standard manual functional testing that you’re performing on a daily basis can actually be scripted to be – so you actually get more testing done in less time.
Excellent. It looks like we’re getting some great response, and we have a clear winner.
It looks like very few of you are actually doing any mobile testing using scripting, so that’s great. Also makes sense why you’d be on the call today and want to learn more about that. It looks like first place was zero. Second place was 26 to 50, so it looks like some of you, about 21 percent, are doing some form of automated scripting. And a portion of you just don’t know. And that makes sense because I think depending on where you’re doing – if you’re on the developer’s side or on the QA side it’s important to understand, you know what you have available to you to be able to do the scripting and some of it may not make sense or may not be feasible. So this certainly is a good way to learn.
Excellent. Actually I think that we actually have one more poll we wanted to ask. This kind of gives you – helps us understand scripting and how it works and if you’re using it today, and how you’re using it today.
So, specifically talking about how you’re performing in mobile app automated testing today, mark all that apply. So again, this is kind of if you are using a testing framework, whether it’s a device cloud or maybe you’re using Calabash or MonkeyTalk or Robotium, please feel free to select the one that makes the most sense and that you’re the most familiar with. It would be great. And you’re not automating is a good answer as well.
Excellent. Looks like we’re getting some good responses. And it’s looking very consistent to the last poll, which is great – so, as expected.
Why don’t we close the poll and we’ll broadcast results. Super. It looks like 48 percent of you are not automating today, and this would be consistent with the last poll that said 0 percent of you are using scripting, so that’s great. And 34 percent are using device cloud, which is also interesting since we are expecting that several of you are current Keynote customers, so you may have heard of it from a prior webcast and/or are a current Keynote customer of our DeviceAnywhere – or I should say Keynote mobile testing platform.
And now at this time I’d like to introduce Darren Madonick, and he’s going to talk a little bit specifically about object-level scripting and how it works. And I’ll let Darren take it away.
Thanks, Josh. So, obviously the goal with object-level scripting is to be able to maximize your reuse across either the same platform or multiple different platforms. So our object-level scripting gives a high reusability across the same type of content, like web, or across the same kind of platform for native applications.
While object scripting is certainly one of the most important parts of automation, it’s also important to understand that it’s not the entire picture. Object scripting can certainly drive a script based on elements and objects, but it can’t truly tell if the object rendered properly.
Obviously it makes sense to script as much as possible with object, but then of course there is some piece to the script that you may want to validate. And that may be an image or it may be some specific text that is pixilated, and catching that with an object script – or a fully object script, isn’t something that’s necessarily very easy. You want to make sure that things show up appropriately, there’s no pixelization, proper alignment.
Obviously with the new iPhone 6 coming out there’s two different form factors, two different sizes. Apple has made mention that the scalability is going to be automatic, and of course no testing needs to be done. That’s straight out of Apple’s mouth. However, as a QA organization, I’m sure we’re all familiar with understanding that even though a manufacturer may guarantee that there’s no additional testing necessary, that doesn’t always end up holding true.
And so with our object-based scripting we give you the ability to not only script native and web elements using our object-level scripting, we support our native web and hybrid applications throughout all of our different tools, including our UI-based scripting, which includes our OCR capabilities and our image-based capabilities. And we can use states to manage that repository.
We also have capture playback, programming, or you can use our WYSIWYG scripting options, which give you a visual recorder right built into our study. And we’ll go into that in the demo that I show in just a little bit. For those of you who are already using the Keynote device cloud, you may have absolutely seen our scripting interface, so it’ll look very familiar to you.
We also have multiple scheduling options. So we give you the convenience of being able to either run ad hoc or schedule your test using our scheduler, which I’ll go into a little bit more into detail a little bit after the demonstration. We certainly have integrations that allow you to kick off these scripts from some other tools – very popular tools in the market including ALM, UFT, as well as Jenkins and other build tools for command-line execution and automated continuous integration.
We also provide that scalability to allow you to build, create, and manage thousands of test cases, and have the ability to have unlimited concurrent tests and unlimited concurrent users in the system.
So now I would like to take a minute and go into a live demo of some of the new tools that we have in the Keynote portfolio, specifically with the IOS object-level scripting. So I’m going to go ahead and share my screen.
And while Darren’s doing that, let me just – a quick reminder to go ahead and submit any questions you have during this demo, or have up unto this point. You should see the Q&A panel on the bottom part of your screen. And just as a note, if you are in full-screen mode, you will need to click out to be able to submit questions. And back to you Darren. Thanks.
Thanks, Josh. So yeah, here is the DeviceAnywhere studio interface. For those who are familiar with DeviceAnywhere, this will obviously look very familiar. For those of you who aren’t, I think one of the most important takeaways from this is our visual scripting interface.
There’s no requirement to understand any sort of programming language or have to be a developer in order to get started with our scripting interface. And it’s extremely easy to get a device up onto our DeviceAnywhere devices, so what you’re looking at here on the right-hand side is a real iPhone 5s. The device is hooked up using our DeviceAnywhere hardware solution.
And so if I scroll over to the right here, you’ll see that we have a device console to the right-hand side of the device. So uploading your app to the device is as easy as clicking on the little wrench here, clicking on our applications tab, and then choosing Add Application. This gives you a whole range of different applications that we support, including of course IOS. So I’ll select iPhone from the list. And I can give my app a name. And I can also append a version to it as well.
And then when I’m ready to add my IPA file, I can simply add that file from my local file system – I can browse to it. And then once I upload it, it will be added to the repository here on the right-hand side. At this point anybody who is within my account has the ability to click on that app and would have the ability to upload it via the data cable.
We’ve gone ahead and short-circuited that a little bit our demonstration. We’ve added a couple of popular apps onto this iPhone 5s. You’ll notice that we have a couple of Direct TV apps as well as Met Life and a Kroger app. We also have a Regions banking app that we’ve installed onto this device. All of which have been enabled with our object-level scripting.
Now the beauty part of our object-level scripting is we use built-in APIs that are provided by the Apple SDK directly using UI automator. So anything that you would see as far as objects that are identified on the screen would show up and appear the same as if you were using UI automator in the X code SDK.
So, I’ll go ahead and click my dropdown list here. I’ve automatically dragged in a launch application command directly into my scripting interface.
And so this is the first step that I would need to do in order to test my native application. Obviously in order to use my native app and perform some tests, I need to be inside of it and I need to launch it.
Instead of having to find this icon on the screen of the device and have it launch via matching, let’s say pixels or matching text, with our new IOS object-level scripting, we now just have a simple dropdown list that allows us to see all of the different applications and the texts that have been implemented or uploaded to this device.
So, I’m going to go ahead and pick the Met Life app here. And I’m going to go ahead and save this application – or save this command here. So what happens when I run this script is we’re going to automatically find – identify the Met Life app, and then we would be able to automatically launch that? So I’ll go ahead and run this script from here, and you’ll see that our execution updates on the top side here and then you can see the execution actually occurring in real time on the device.
So once we get into the device we do certainly have the ability to use any of our DeviceAnywhere commands. So it’s important to recognize as I mentioned a little bit earlier, there may be certain elements of this application that you may want to identify with a little bit more consistency than just an object-level identification.
The first thing I will do is drag in an object-level identification though. So I’m going to go ahead and take that from the list, I’m going to drag it right underneath my launch app.
So there’s a few things on this page that you’ll see show up right away. We have the ability to see the Wi-Fi 3G connection, the date and time, as well as certain icons like viewing and paying my bill, contact us, as well as the four options for auto, dental, home, and life insurance. And then you can select a button from there.
One thing that you’ll notice is not identified here is the background image. What ends up happening as you compile your apps inside of IOS is there are certain background images that may just be identified in the back layer or just as a background, such as this Met Life banner and our little friend, Snoopy dog, walking around on the side there.
And so it’s important to recognize that even though everyone loves to be able to script with just object-based script to maximize the reusability, there are certain elements that you would want to ensure a tight and a very crisp user experience. One of those being pixelization or the lack of pixelization for both your banners and your icons, as well as all of your different buttons and actual interactions on the app.
And so it’s very easy for us to switch back and forth between object-level scripting and then an image-based validation that gives us the ability to take a screenshot and actually identify a visual object on that screen as opposed to an object wire frame.
Now, I did mention that we utilize APIs that are available in UI automation and UI automator inside X code. So I can pop open my object tree and I have the ability to see all of the different class names and all the different nodes that are identifying the images and creating this native app that’s available on the device itself.
Now of course, for those of you who are familiar with object-based scripting, or at least programming it with an object-oriented programming, you’ll recognize that these objects will be the same on an iPhone 4s, and iPhone 5, a 5s, the new 6, and obviously both screen sizes. If the app is the same, the objects that are underlying inside that app are going to be the same. So with great confidence you know that you have the ability to script one script that will then be reusable across multiple devices, no matter the resolution or the screen size.
Now one question that usually comes up is well, how do I handle any sort of image or non-object item that I want to validate or verify? And the way that we handle that is with what we call states. Now because we’re focused on object-level scripting I’m not going to go into too much detail about our states. But we do have the ability to identify by device type a particular image or a particular icon or a particular state.
So, I’ll go ahead and call this Snoopy, and then I can create a state for my iPhone 5. Now if I had multiple devices inside my project, and I had multiple different screen resolutions and screen sizes, I’d be able to identify this at a per-device level, and then use this generic state as an object inside of my script that I’m creating. So instead of having some sort of device dependent action, I can provide this layer of abstraction with my state, create an object even from my image-based scripts, and drag that directly into my object-based script.
So this gives me a high level of reusability and a high level of customization, while still maintaining one set of scripts across multiple different devices of the same platform type.
And so that’s really the power of the object-based scripting. Once you define multiple different actions in multiple different devices inside of your application, you’ll have a project structure that starts to look like this.
So you’ll have different functionalities within your device, within your different screens, within your different apps. And then you can have one set of test cases that utilize these actions that you create as building blocks to identify and create one large set of scripts that you can then run as an executed test cycle either using our continuous integration with our command line or a Jenkins integration.
You can schedule it using our own internal scheduler, so this gives you high capabilities within our own visual scripting to provide an actual schedule that will give you flexibility to run these scripts at a given time and automatically upload the app and execute your set of test cases, as well as our integrations with other popular testing tools like UFT and ALM.
And so all of these scripts that you saw today, and everything that we’ve built today can easily be ported and used in any of our testing frameworks and even utilized using our job API that we have with our object-based scripting as well.
Great. Thanks, Darren. Let’s go ahead and switch back.
Excellent. Now we’d like to take a few questions from you. Again, please feel free to submit any last minute questions in the Q&A panel on the bottom section of the window. Looks like we’re getting some questions here. First question is for Darren – so when it comes to app testing, why is it important to test in real devices versus – and can you do object-level scripting on emulators, et cetera?
Yeah, so with both the Android SDK and Xcode to – I hate to leave out Windows, but market share’s not quite there yet. But what we’re focused on today is object-level scripting with Android and IOS. And so I’ll answer the question within that framework.
So with Android SDK you do get an emulator, with Xcode you’re getting an emulator as well; you can bring your own local IOS device and tether. What you don’t have necessarily access to is every single different flavor and every single different OS version that’s available on the market. So you may get an emulator that has the IOS operating system, but you may not have the ability to test that on an iPhone 4 running IOS 6, IOS 7, an iPhone 4s running IOS 6 through IOS 7, and so on and so forth.
Also with Android there’s a lot more fragmentation than IOS, especially when it comes to manufacturer differences, carrier differences, and third-party applications that take over default functionality of the device.
And those are things that an emulator just can’t catch. So Samsung doesn’t release an emulated version of its AT&T Samsung Galaxy S3, it’s Verizon Samsung Galaxy S3, and so on. And so there are APIs and default applications that have been identified on those devices that conflict with what is normally built out and normally running in an emulator.
So it is important to not just test within your emulator for basic functionality of the app, but also to see how they’re going to play on different carrier networks, how they’re going to play with different manufacturers, different screen sizes and different processing power, and different amounts of RAM as well.
And so having a real device makes a whole lot of a difference when you’re talking about a user experience and your ability to identify customer support issues before releasing that app into the field.
Great. Thank you.
Another great question here is, if we have any examples – does Keynote have any examples of object-level scripting in the Help/Documentation? I believe we have some documents available that we could send out.
We do have examples – visual examples within our Help and Documentation. Due to the nature of the DeviceAnywhere product, you do kind of need to have at least a device in order to start running or interacting or playing around with some of our examples. But there are definitely scripting resources that will help you get a leg up and a quick start into our DeviceAnywhere object-level scripting.
Okay. And then another kind of site question is, can you use Keynote’s mobile testing/DeviceAnywhere to write voice automated tests?
So with voice you certainly – since it’s a real device and we do provide Wi-Fi and cellular coverage, you can certainly test the handoff. One of the new features in the iPhone 6 is the handoff between Wi-Fi calling and then voice over LTE as well as 3G voice as well. So that handoff that occurs between making and initializing a Wi-Fi call and then switching over to let’s say a 3G or an edge call, it’s very important to be able to test that. Not just for the purposes of testing the call quality, but also testing app interrupts, how your application is going to handle an incoming call – things like that.
Because these devices are real devices, there’s absolutely no limitation to testing incoming calls. We provide phone numbers and active data plans on all of our DeviceAnywhere devices. We also do have streaming audio so you can hear ringtones and audio calls coming off of the handset and off of the speaker.
Okay. And I know you showed a little bit about object-level scripting and uploading and application, does Keynote support IOS 8 yet for object-level, and what is the plan for that, if not?
We do support it in Beta. We have a number of customers that are using their developer accounts to identify their own private devices and upload their IOS 8 Beta apps to our devices. And they are currently doing that testing. We unfortunately can’t release any Beta IOS 8 devices prior to its release, but we will be fully supporting IOS 8 on its release on September 17.
Okay. And – yeah, that’s another thing worth mentioning is that one of our planned support about IOS 8 on September 17; is it to update all of our existing devices?
Well, we won‘t be updating all of our existing devices. We certainly recognize that we have a great deal of customers that will want to be able to test differences between IOS 6, IOS 7, and the new IOS 8. And so we’re going to work to maintain a good mix within our shared environment so that our customers have access to all flavors and versions of the IOS operating system so they can truly test to ensure that their app is working across multiple different versions of the operating system.
Great. And is there a way to use Keynote’s DeviceAnywhere platform in conjunction with UFT – HP UFT without specifically using the add-in for UFT?
No. You do need to use our DeviceAnywhere add-in in order to get access to the devices, and log in using one of our DeviceAnywhere licenses in order to authenticate you against the device cloud. There’s no way to use UFT with our device cloud without using our add-in.
And then last question – or second-to-last question is, do you have a set of pre-defined connectivity tests? Do we have anything that we – like a pre-set test that they could run? Or I guess we just help them build it?
Normally – we’ve tried to do that in the past providing general actions or connectivity tests. And what we’ve found is every single one of our customers has a little bit of a different set of tests that they want to run or they want to be able to customize those in just a very particular way for their own applications or their own websites or their own functions that they’re doing on the devices.
And so the way that we handle that is with our Professional Services Team. And so we engage with our customer in understanding exactly what it is they want to test, so that we can ensure that the scripts that we deliver are of high quality and meet the needs of our individual customers.
So we’re not interested in providing blanketed scripts that kind of just sort of do what our customers ask, we’re really attempting to engage with our customers directly, understand their critical tests, and provide specific tests for them that accomplish the testing that they want to get done.
Another question here is, is it possible to change OS on the device during the test?
It is possible to update and change the OS on our devices. We encourage on our shared device pool our customers not necessarily do that, or engage with support in order to do that. We do have a number of devices that are labeled for certain OS versions. So what we recommend in the shared pool is to find devices that are like models with different OS versions using our filter capability and then test on multiple different devices instead of testing on the exact same device.
We’ll keep multiple copies of those devices inside of the cloud so that our customers have the ability to jump around. And you can certainly acquire more than one device. So the idea is maybe you would want to test an iPhone 4S on IOS 6 and IOS 7, and then IOS 8 when that comes out. So there’s absolutely nothing stopping our users from logging into DeviceAnywhere using our filter system, looking up IOS 6, 7, 8, for an iPhone 4s, acquiring all three devices and running through the set of tests.
For our private customers we absolutely allow them to update, re-flash, reset any of their private devices, and our support team is ready to help facilitate that as well.
Great. And the last question is if we jailbreak our devices for IOS?
We do have a number of devices that we label in the pool as jail broken devices. We do have a couple of customers that do appreciate being able to test with jailbroken devices to understand the difference. As a rule we don’t jailbreak our devices and we make sure that any devices that are jailbroken are very well labeled inside of our pool so that our customers don’t get confused between the two.
Excellent. Well, thank you everyone. We are going to proceed with our discussion. Darren mentioned a little about continuous integration ALM support, so we just want to talk a little bit about that as well as how you’re able to reuse the scripting – your automated scripting for performance monitoring and post production.
Thanks, Josh. Yeah, so there’s definitely a couple of users on the call today that may not be familiar with Keynote’s offering with the DeviceAnywhere system and our automated testing and remote access manual testing capabilities. And what I showed today was just a small piece. It’s our UI; it’s our framework that we’ve developed to make mobile testing easy.
So we’ve developed that from the ground up to provide a very easy and kind of self-starting interface in order for users who may not be familiar with scripting languages to be able to come in and build scripts very quickly and easily.
We do recognize however that there are other tools in the market that our customers might be more familiar with, or more comfortable using, so we provide a lot of different integrations. So we provide many ways to access these real devices. From our Java-base client we can use our visual scripting as I demonstrated today, we also have a full-qualified Java API that we have documented examples that you can utilize. And so this gives you the ability to integrate with your own jars as well as integrate with a command line.
We also have integrations with leading testing tools such as HP UFT, IBM Rational as well as the ability to integrate our Selenium scripting as well. So you can take your existing Java Selenium scripts and run those against our devices.
We also have an integration with Jenkins to help with continuous integration and also the billed acceptance testing, so that way you don’t have to necessarily use our command line interface. You can also use Jenkins if you happen to subscribe to Jenkins, and be able to utilize that to kick off DeviceAnywhere tests. So we do have a visual plugin that actually is installed and configurable from the UI in Jenkins. Nothing needs to be done inside any sort of configuration files or anything like that.
The other piece that we want to talk about is not just the preproduction and testing before you go live, but we wanted to also spend a minute just posing the question on this webinar here. What do you do after your app goes lives and is sitting in production, and you start having users access the devices – or excuse me, accessing your application using their device to interact with your app?
And so obviously preproduction testing helps get your application into your hands of the users. But once the app goes into production you need to be able to ensure the quality and that requires monitoring of your application and not just regression testing.
Ideally the work that you’ve gone into creating those automated scripts during your development and testing phase should be able to be reused in a monitoring capacity across those same or similar model devices to help better understand your user experience throughout the day.
So you may have always run your regression tests in the middle of the night, and then you’ve got users who maybe use your app quite regularly towards the evening or during their commute home, or in the middle of the workday. And there are certain environmental differences with cell coverage, saturation of the network, different device behaviors that you may not have been able to take into account during your preproduction testing. And so catching this in your production monitoring is something that’s extremely valuable and very important to be able to do.
And with the Keynote platform you can take the scripts that you’ve built in your preproduction and reuse those for your monitoring of your applications to ensure your user experience is of high quality.
And so I’d like to take a minute to just talk about our Keynote mobile app monitoring. So it allows you to quickly and easily understand mobile app performance and availability, it’ll improve your mobile application quality and reliability, and it can assure the return on your mobile initiative investment.
Mobile app monitoring is the only 100 percent cloud solution requiring no modification of your app to be monitored. So you can simply upload your app, our devices are not jailbroken. I know we fielded a question earlier – do we have jailbroken devices?
The answer to that is well, yes, we traditionally don’t use those jailbroken devices for our mobile app monitoring unless our customer wants a very specific feature, or a set of features that would require that jailbreak to occur. And it automatically integrates into our DeviceAnywhere cloud of real smart phones and tablets with the industry’s largest global test and monitoring network and leading monitoring service.
Mobile app monitoring continuously interacts with your apps 24/7. So the script that you write will continuously run in a scheduled capacity, exactly like a customer would do in the real world. So you can build this script to interact with the device in your app exactly as your user would. So using either the wireless carrier or you can use Wi-Fi for your true end user experience.
So by fusing together the best on-demand mobile testing and monitoring platform, Keynote can now offer a holistic solution that delivers unmatched value across development, QA, and your operations team.
Excellent. Thanks, Darren. So here’s the big question that we’ve been getting is, how to get started? And I think, you know, you might have been hearing a lot about this today as far as, you know from a lot of things just around what’s happening today via Apple, and a lot of it – you know, the need to be able to do testing, more functions, more features to test, lots more things to test. And so the need, desire, and demand to perform more testing in less time or the same amount of time is even that much more critical.
So just a quick – before we go we’re just going to give you a couple actions. One is if you’re a current customer of Keynote we encourage you to consult your account rep or your solutions consultant. They will have the ability to set you up with a demo and get you started, and also walk through with you on a dedicated environment.
If you’re new to Keynote and have not had the opportunity to experience this, we’re happy to sit down with you and encourage you to talk to an inside sales rep and they’ll give you an in-depth customized demo catered to your application and catered to your needs and customized to what you need to support your application.
And if you have any other questions, please feel free to email us at email@example.com.
And that wraps up today. Thank you for attending. We hope this webcast was both educational and informative, and we’ll see you next time.
Duration: 44 minutes