Visitor Monitoring Testing at the Speed of Mobile: Adopting Continuous Integration with Agile | Keynote

Testing at the Speed of Mobile: Adopting Continuous Integration with Agile

About the Webcast

Developers, testers, and managers are moving away from traditional testing late in development and toward early, agile testing practices, with this shift being immensely more evident in the mobile sphere. Many teams are adopting continuous integration (CI) to speed up and streamline their development and testing processes in order to meet the demands of this condensed, mobile-centric timeframe. 

Keynote’s Joe Lewis and Josh Galde explore how developers and testers can become more closely aligned than ever before with easily deployable and configurable tools such as Jenkins CI. Testing on real mobile devices through this integration tool gives you the most accurate view into how your mobile app or website will perform in the real world, all in a pre-production environment. 

The speakers discuss:

  • What the mobile boom means for today’s developers and testers
  • How to test at the speed of mobile
  • Who uses Jenkins and how Jenkins supports mobile testing
  • How to test your mobile app or website on real mobile devices

Webcast Transcription


Welcome to today’s web seminar, Testing at the Speed of Mobile, Adopting Continuous Integration with Agile featuring Joe Lewis, Solutions Consultant at Keynote and Josh Galde, Testing Evangelist at Keynote. I’m your host and moderator Josiah Renaudin.

With that said, I’d like to introduce and pass the controls over to our first speaker, Josh.


Thanks, Josiah. Today’s webcast will help you understand what the mobile boom means for today’s developers and testers. Also, how to test at the speed of mobile. We’ll cover who uses Jenkins and how Jenkins supports mobile testing. And of course, we’ll show you how to test your mobile app or website on real mobile devices from within Jenkins through Keynote’s mobile testing demo.

First, we want to talk a little bit about the mobile market. And also, what we mean by this mobile imperative that we’re talking about. Clearly, users are demanding access via mobile. For example, there were 1 billion apps downloaded in 2013 according to Gartner. And it was predicted that there will be $77 billion in anticipated revenue generated through mobile apps by 2017 also according to Gartner. In the retail sector, mobile has become the No. 1 technology priority in 2014. And in financial, two out of three banks predict 100 percent of their customers will be using their mobile services by 2017. Whether we are talking about consumer apps or internal business applications, users want access on demand via their smart phones and tablets. And this is easily seen in the increase of traffic and sales in recent years.

And users have become less accepting of poor quality of their mobile experience. They have made allowances in the past, but now, they expect their mobile experiences to be designed for mobile and perform well for mobile and not just accommodating mobile. The lack of this quality impacts sales and perception. For example, one survey found that only 16 percent of consumers would try a failing app more than twice before dumping it. Another study found that the app failures have cost businesses over $60 billion. And yet another survey found almost 80 to 90 percent of mobile apps were eventually deleted from users’ phones in 2013 alone.

With so much at stake, including, in many cases, one’s brand, testing these apps has become more than a “nice-to-have.” It has become what we call a mobile imperative in providing quality to mobile apps to your users. When developing a mobile app, you run into many headaches. You need to run it on major operating systems, on as many devices as possible, and you don’t have time to learn new tools or technologies. You also need to be able to iterate this testing quickly. So testing for mobile has truly become the moving target. Take, for example, the recent Apple and Google announcements.

Not only do devices change, but operating systems change, carriers change, phone specs, capabilities, browsers, everything is constantly changing. And guess what? With Agile development, your mobile app will demand rapid iteration and reiteration, which means, you guessed it, more change. The picture you see here is an actual mobile QA desk at one of our customers. This kind of unstructured environment is what many QA organizations are dealing with. So you need to be able to deliver and manage across all of that dynamic complexity.

Unfortunately, there’s no time for your team to learn all these new technologies. And of course, there’s not enough time to do all that testing you need. So there’s lots of anxiety about whether the application will actually perform at scale when deployed in production.

So we just wanted to take an opportunity to gather some data and some insight from you all that are attending today. So we want to ask you some poll questions. Our first poll question, what do you see are the biggest challenges in testing mobile apps? For example, device fragmentation, form factors, OS versions, etc., everything I just spoke about. It could be availability of skilled mobile testing experts.

It could be implementing the right test method process for mobile, availability of proper testing tools, access to real mobile devices to test on, a common hassle for people, and/or having enough time to test. Please feel free to select all that apply. It looks like we’re getting some great feedback. Thank you for submitting your answers. Now, we’ll go ahead and reveal the results of the poll. It looks like our winner is device fragmentation. As expected, we feel that is understandable as mobile is very complicated and complex, unlike desktop. What desktop has is maturation over the years where it grew, and it had its own complexity issues. But mobile has become what a lot of our customers tell us is 10 times more complicated. And that we see about 61 percent of the vote. Availability of skilled mobile testing experts, about 41 percent. Implementing the right mobile testing method or process, about 49 percent. Availability of proper tools, 46 percent. Access to remote devices, 51 percent. And having enough time to test, 56 percent.

So how does Keynote help you solve these problems? First, we provide global Cloud based testing environment with the broadest range of devices and testing capabilities on the planet. The picture above shows our testing environment, which is replicated using servers all over the world by contrast with the cluttered desk in the previous slide. Our global network is the most comprehensive in the industry, and we work closely with all major carriers so that you can test in every possible configuration of connectivity and platform. Of course, we know that you don’t use Keynote in isolation – and/or wouldn’t use Keynote in isolation. So it’s critical that we integrate with and leverage your existing investments in development and testing tools for major partners such as SAP, IBM, HP, and Jenkins, which will be showcased on today’s webcast.

And this is where test automation can be the most value. The ability to run hundreds of tests repeatedly on a large number of devices. So speaking of test automation, we’d like to hear from you on this particular poll question and specific to automation and types of application regression. What percentage of your mobile native application regression test is currently automated using scripting? This would be any form of scripting. It could be open source or using a device cloud. Would you say 0 percent, 1 to 25, 26 to 50, 51 to 75, over 75, the majority of it, or you just don’t know? And that’s totally acceptable.

Feel free to submit your response now. Great. It looks like we’re getting some great responses. And this question is really about if you understand how much of scripting – of automating your test is being done today. It could be none. That would be as expected because a lot of testing is done manually. So the idea of automating it I think, while be it ideal, can be difficult. So it definitely would be understandable. A majority of it is not achievable. But certainly, that’s why we’re here today is to learn how to do that, especially if you’re currently using Jenkins. Great.

We’ll go ahead and close the poll. It looks like the winner was 1 to 25 was 30 percent, 0 got 28 percent, and don’t know 23 percent. So again, as expected, just very, very informative. And thank you for responding. And this will feed into our discussion today. So it’s very, very helpful.

And I believe we have another poll here. And this is specifically to mobile apps automated testing and how you’re doing it. So the previous question was more about what percentage of testing is done. Now, we’re going to talk about how you’re doing it. So again, feel free to answer all that apply. You can say I’m not automating today at all, and that’s kind of why I’m here. Understandable. You can say I’m using a device cloud such as Keynote DeviceAnywhere or other vendors out there that have device clouds that you can automate on. And then we’ve got some open source platforms such as APM, Calabash, Monkey Talk, and Robotium. You can also say Other. A lot of – there’s a whole host of other ways to perform automated scripting for mobile applications. So that certainly would not be surprising.

Feel free to go ahead and enter your response, and we will let you take the time to do that. Excellent. It looks like we’re getting some great responses. I know this is a very interesting question, and a lot of people may not exactly know what they use. So I would not be surprised if not automating or other were the top winners here because a lot of companies aren’t sure how they’re doing this. And that’s kind of why they’re here to learn about that and also learn about how they could use a Jenkins to do that presumably if they’re using Jenkins to do their builds or their apps.

So this is very informative. So let’s go ahead and close this poll and get the response. It looks like not automating is the winner with 43 percent as expected. And Other is 25 percent. Third would be Appium, and fourth would be Robotium. So again, very interesting. And this will kind of feed into our discussion today. So thank you for providing your answers.

Excellent. Well, I want to take the opportunity to have Joe Lewis, one of our solution consultants who is on the call, to talk a little bit about object-level scripting. The reason we wanted to talk about this is because it feeds into how you can automate.

And especially with Keynote’s platform, using object-level scripting really gives you the ability to perform these scripts and perform this regression testing at the object layer versus the image- or text-based layer, which is a pretty common standard in the industry today. So this gives you a new and more efficient way to do it. And hopefully, the intended way would be to provide the efficiency and give you an easy way to do it. So I’ll let Joe talk a little bit here, and we’ll go from there.


All right. Thanks, Josh. So when we talk about running automated test on devices, it’s typically two approaches that people take. They either test on real devices, or they use emulators or simulators, and then they work at more of an object or code level. And this is kind of a divide that’s been in automated testing for a while. Open source automated testing versus real device automated testing, using object scripting can give you high reusability across the same type of content or processing platform such as Android or iOS. And while object scripting is certainly the most important part of automation because it’s so portable across different types of hand testing models, it isn’t the entire picture.

Obviously, scripting can drive a script based on elements and objects, but it can’t truly tell you if the object rendered properly if the picture actually showed up but there was no pixelization and clear and proper alignment, things like that. And oftentimes, when you’re running those object-level scripts, you’re running them against simulators or emulating devices. And the problem with that is that those aren’t always going to even render things or operate tasks the same way that a natural device would. Plenty of times you hear oh, it works fine in the emulator. But once we put it on an actual device, it just crashes.

You need to be testing on real devices as well. So with automated mobile testing, our object-level scripting ability, we support both native and web element object-level scripting using native web and hybrid applications. We also offer UI-based scripting, so you can also automate versus either image verification or text recognition to assure that your pages are rendering properly, that your app is actually displaying properly, and that the content that you want to be displayed is what’s being displayed. In terms of scripting options, we have several options as well.

You can either capture and playback, so you can record scripts by running through your test cases on a device, and then have a script be generated while you test. You can also use a Java API to create scripts using just an IDE or even a text editor. And there is also a WYSIWYG that allows you to just drag and drop plans into a timeline. Tests you created can be scheduled. This is really a benefit of automated testing. When you schedule tests, you can have them run overnight while you’re gone. You don’t have to wait for a bunch of QA resources to run your regression test. You can have that happen overnight and then just come in the next morning and review the results.

You can also run ad hoc tests. So you can take off automated test cases manually and use them in real time if you need to do a little bit of a deeper dive into an area that you’re seeing where you just want to make sure that everything is running properly and looks all right. The system is scalable, it’s very easy to add more devices to the system, and there’s a limit to how many different devices you can add and test again. You’re also able to create adequately different test cases, manage them all through the same tool, and everything is going to be in one centralized location. You’re also are going to be able to run concurrent tests and have as many users as you want to set up for that system.

So as I mentioned, we have many different ways to automate on real mobile devices. We provide our own Java-based client, which you can download, and do that WYSIWYG UI-based scripting. We also have a Java API or integration with leading test tools, including HPUFT, which used to be called QTT, IBM Rational, and Selenium. We will also discuss today how we integrate with continuous integration tools such as Jenkins.


Great. Thanks, Joe. So now, I want to talk a little bit about Jenkins. First, let’s talk a little bit about the history of Jenkins if we can. So it’s very interesting. It was built by Sun Micro about eight years ago. And it takes Agile development to the extreme. So there has been a release of Jenkins every week over the past eight years totaling over 500 releases. It’s the number-one continuous integration and continuous development server. The key to Jenkins is that it’s very easy to get started. All you need to do is to run the Java -Jar Jenkins.war file, and there you have it. Jenkins will be up and running. For more information on Jenkins, you can easily go to And we will also provide a link to your at the end of this webcast.

Jenkins has close to 78,000 installations around the world. A 60 percent increase in the last year alone. What has contributed to this massive adoption is the ease of use we spoke about previously, the ability to easily write a new plugin to support a new tool or resource, leveraging its inherent architecture. Several years ago, it had about 50 plugins contributed. And today, it has over 900 plugins by over 500 contributors with 1 plugin per day being added. Part of the success of an open source community is to see how many people contribute back.

And if you look at this graph, that’s about 12 percent of those using Jenkins who have contributed back into the open source with almost 25 percent who are interested in writing a plugin. In fact, it’s this easy plugin capability that has allowed Keynote to build its plugin to support mobile testing from Jenkins. Also, part of the success is the staying power of Jenkins. It’s one thing to launch something. But it’s a whole other issue to maintain the satisfaction and usability. And today, Jenkins enjoys an 80 percent satisfaction rate. So we’ve got another specific question for you around Jenkins. It’s a very unique question. It’s a yes/no question and just to gauge interest and/or use of Jenkins within our audience today.

The question is do you use Jenkins to do continuous integration for mobile? Yes or no? Please select one, and we will give you a few seconds to respond and answer the poll. It looks like most of you have responded. And this is really helpful for us and also telling to really gauge how many people are using it and also can feed into why you even want to use something like Jenkins for your builds or even how you can test the mobile devices, which obviously is a key part of the strategy. Excellent. I’ll go ahead and close the poll.

So 64 percent of you are not using. But it looks like a good number of you, 36 percent, are. So that’s a good balance. It looks like a lot of people are here to learn about Jenkins, and hopefully, we’ve covered a few slides on that. I do have a few more, so we will dive into it a little bit here. This is specific to Jenkins and mobile. So most of us are now quite new to continuous integration of mobile, right? So let’s try and understand why this is so important and how Jenkins and mobile can work well together. If we look at the chart here, we can see that internet traffic from mobile is skyrocketing.

Over 85 percent of mobile traffic is in direct result to mobile apps not mobile web or browsers thus supporting the need to understand the importance of investing in mobile app development, which I don’t think anyone doubts. But it’s certainly a good reminder. Setting up a continuous integration tool for mobile means that developers can focus on developing the application itself or the code without worrying about building or testing the application. This makes it much easier for developers. There are many plugins, as shared earlier, that you can use for continuous integration of mobile.

You can also scale out your workflow to distributed build and create different iClients, and you can also have matrix projects for cross-device tests. Today, when we talk about mobile, we are really talking about Android and iOS. Windows is growing but more in Europe than in the US and has not reached the level of saturation in the marketplace to warrant significant support. There are several approaches to building mobile apps. The most common approach is via native, i.e., native tools from Google and Apple. For iOS, it’s X Code, and for Android, you would use Java and its compilers.

To do this approach, you will need a build environment to support both iOS and Android. And as you may know, iOS needs Mac support and hardware, and Android is Linux-based, open-sourced, and passed by Google. Keynote’s mobile testing integration with Jenkins enables you to execute test cases directly from Jenkins and to upload applications available in the Jenkins’ workspace onto Keynote’s devices. This ability to automatically deliver your ability devices and run a set of test cases to validate the build increases productivity and streamlines your testing process. As you can see by the screen shot, you are able to run any existing Jenkins built script on real mobile devices—all from your desktop and without creating a new script.

With Keynote’s integration with Jenkins, you can perform automated sanity testing of each mobile application build to support earlier defect identification, build and schedule mobile applications, automated regression suite directly from build machine to increase confidence, enhance test coverage, and shorter test cycles. This gives immediate feedback to developers on the quality, functionality, or system-wide impact of the code they are writing. It also allows you to create an automated test suit, which will deploy the mobile application onto the device and run the sanity/regression test cases when the new mobile application build is available.

The key benefits that you’re able to access include, obviously, saving QA time. You’re able to run at any time. Reusability to increasing coverage, and this ensures full regression test coverage within the shorter timeframes. And a reduced cost. I’ll now turn it back over to Joe who will transition to the demo.  


Thanks, Josh. So what you folks are seeing here is an install of Jenkins that’s going to be utilizing our DeviceAnywhere plugins for Jenkins to kick off test cases and upload builds directly to devices automatically as part of our build process. The plugin is just like any other Jenkins plugin. So you would be able to install it using an HPI file directly through this interface. And then you would configure it in Jenkins using the system information or your DeviceAnywhere account.

Once I’ve got that set up, I would just start up one of my projects here and configure actions that can be triggered on builds. So I’m able to upload the application. And let’s see if I’m uploading the application called Stars via an ATK, to a device. I can also acquire that same build process through your test cases to run. So what I can do here is I can select what project I want to run, my test case, and what device I would like to run that test case against. Now, when a build happens as part of this project, these things will happen automatically.

So any time there’s a build now, I build my version of in my application, it’s going to get pushed through the device in my DeviceAnywhere system, and then it’s going to run automated test cases against it. So let me go ahead and kick off the build process here. Now, we’ve got a build started. Go ahead and watch the console output here, and I’m going to upload an application to the device. And then it’s going to run a test case against the device. This is going to allow me to run regression and smoke tests for a new build automatically as part of my build process. It can happen overnight, and it can happen without any human intervention.

So I don’t need to worry about having this happen, tying up human resources. It’s all going to be just happening at the same speed as the rest of my build process. It will allow me to remain as agile as possible when I’m working with this type of product deployment and development model. So here, we can see I was able to upload my application. Now, it’s running some tests against an application on the device. And this is actually happening on this real live device in the system, so you’re getting end user experience.

And this can be done using the object-level scripting that we talked about, or it can be done using integer text space verification as well. It’s however I want to script it. It doesn’t matter what type of automation I have chosen to use for this particular test case. They can all be triggered through the Jenkins plugin. I’m going to go ahead and view the details of my build. And now, we can see my application upload path. I can also see that my test cases run successfully. I can even go in and view results. So I can do this in the DeviceAnywhere portal alongside any other automation test cases that I’m running whether or not they contribute through Jenkins.

I can go through step by step and see exactly what was happening on the device during this test case. And I can find out what happened if there were any problems. I can view screen shots in full size. I can step through each point. So here, we’re closing the application from the device. And then here, we’re launching an application. So verifying that we’re at the home screen of the app and then performing a search for a movie. In this case, we’re searching for gravity and verifying that that movie is actually found in the search results. Once the test case is successfully completed, the success is reported and passed back to the Jenkins portal to be stored there.

And so as you can see with this tool, it’s very easy to automate my build process right down to making sure that there weren’t any catastrophic failures with my build. With that, I’ll go ahead and pass it on back to Josh. And we’ll start –


Thanks, Joe. And we’ll let Josiah – do you want to ask the questions?


Yes. The first question does Keynote processor performance?


So that’s actually going to depend on how many devices you’re working with. Overall, we recommend at least a Pentium 4 processor and a minimum of 2 gigabytes of RAM, although 4 gigabytes is preferred.


Thank you. Next question, how do you keep clients’ apps build secured if they have to upload them to your devices?


That’s an excellent question. So when we’re talking about automation, generally speaking, what you’re automating against are what we call private devices. So these are devices that we host for you alongside our cloud or public devices that are private and specific to you and your team. So any applications that you upload, any tests that you run on those devices, are going to remain on those devices and not be accessible to the general public. That being said, any testing that you do on those public accessible devices, when you access a device, it’s only one user per device per session.

So while you’re working on that device, everything that’s happening on that device is confidential. Once you’re done with that device, and you release it back into the pool, we have clean up scripts that attempt to run that device to clean up anything that you left on there, applications that you may have installed, websites that you may have entered in, restore the device back to its default state. That being said, since they are public devices, it’s an automated process. The general best case practice is to just as part of your overall workflow, make sure that you’re removing those items from the public devices before you release them. With the private devices, it’s not as much of a concern.


All right. Next question. How do you set up test devices in your test lab to make them visible in Jenkins?


All right. So these are all devices that are actually a part of the DeviceAnywhere system that we’re either hosting in our data center, or they can also be hosted at a customer site at their data center, and there’s devices that we specifically built out to be used with the DeviceAnywhere system. If you remember, Josh showed that slide that showed a bunch of devices in boxes and racks. And those are actually devices that we built out and integrated. They’re wired into the battery, into the video, into the buttons of the phone, so you have full control over that device remotely.

And so when you set up for Jenkins plugin, you’ll actually point it at your DeviceAnywhere account information, and it’s going to be able to get information on your projects that you’ve built for devices and then what devices you’re running those projects against, and those devices will be available through Jenkins.


Thank you. Next question, if you wanted to complement the emulator or cloud testing with devices, how would you decide how many of the latest devices or versions to actually purchase?


That’s a great question, and there’s no really great answer for it, unfortunately. So there are a lot of different devices out there. There are a lot of different OS versions. And it can be a real headache. What most people do is they take a look either through using some sort of – if they have a service that people are hitting, they keep stats on the user agent strengths, what are the most popular devices that are hitting their page that they’re seeing? You can also look around. There’s a stack out there. Android’s developer site has a lot of information about what the most popular versions of Android are, what’s most prevalent out there in the wild.

The benefit to a service like DeviceAnywhere is that we actually take a lot of that worry out of your hands. You don’t have to worry about buying all these different devices. You don’t have to worry about what the most popular devices are. We do a lot of that legwork for you, and we’ve actually got all those devices already in our public cloud or devices available for testing. Those have multiple copies of different devices across different carrier models, different OS versions, and different hardware revisions to make sure that you’re going to be able to get the test coverage that you need.


All right. Next question, what languages are supported for writing test cases?


So there are a couple of answers for that. The first one is that the basic scripting is going to be done, it’s actually all in Java. It can be done using our Java API. So if you want to code it all by hand, you could. But you also have the option of using our script building tools, directly in the tool. They’re going to let you do drag and drop scripting. And it’s actually a very deep and feature-rich type of scripting. I know a lot of people that started out saying well, I want to do it all through Java, and they’ve actually switched over to mostly using the UI. So it’s a very easy-to-use and easy-to-learn UI.

Fortunately, it also means that you don’t need to be a programmer to be able to create automation script. But we also then support now using Selenium. But that’s if you’re doing web testing on Android, you’re going to be able to use your Selenium script that you have already against the devices that you’re going to be automated testing on. In terms of other languages, unfortunately, we don’t really support those. So C+ or C# or any of those or Python. If you’re going to be doing coding of test cases, you’re going to be using our Java API predominantly.


Next question, do you have the same system for iOS devices?


We do. We actually just launched – I think we either just launched or are launching object level scripting very soon for iOS like they have for Android. All of the scripting stuff that’s based around image or text-based recognition is universal is going to work on iOS, Android, Windows Mobile. Almost every tool that you see is going to work on that platform. And the Jenkins plugin itself doesn’t really care what platform it is as long as the scripts have been implemented for it in the devices system. You’re going to be able to upload your application whether it’s Android or iOS.

With app upload for iOS, it’s making sure, I think we just mentioned it because someone had asked, no, you do not have to instrument or provision that application to work with our devices. You don’t have to add any additional devices to your app or mobile developer account. We’re actually going to assign that IPA file for you when you upload it with our Enterprise certificate that will work on all of our devices. So you don’t need to use a third party tool or text byte, or anything like that. You can just upload an IPA and be done with it.


Next question, what mechanism do you use to install and run applications on an iOS device?


All right. Well, very good. So I just kind of covered that a little bit. And we actually – behind the scenes, we’re doing an install of the device through a Mac machine that’s hooked up to the system to do that app loading on the device via the USB cable. And it’s also signing and provisioning the IPA file with our certificate. So behind the scenes, it’s actually going to be doing the exact same thing that you would be doing on your end loading it through X Code or something similar. But we’re taking that hassle out of your hands. You don’t have to worry about adding anything additional or adding extra devices to the mobile developer account profile. I think the current limit is 150 per year, and that will go a lot quicker than you think it will once you really start doing testing. So you don’t have to worry about any of that with the system.


Can scripts be run on physical devices locally rather than on the cloud?


So that is possible in a couple of different ways. So as I mentioned, we can do what we call on-premises deployment. So that would be our fully integrated hardware devices in your test lab. They hook up to your wi-fi, and you can have them right there. Or it could be also for a number of handsets, and this is a much more compatibility list. We also are able to offer what we call local tethered devices where you basically plugin a handset that you have directly into your USB port. Those are only really for local, automated testing. So it’s not really anything you run on a schedule.

Those, you wouldn’t work with Jenkins like you’ve seen here. For that, you would need the integrated devices that are always available via a specific server. But those can be located either in our data center or local to a lab.


All right. Next question, is it possible to performance-test different devices with Keynote?


So it is. We have some other products. So this is more focused around functional testing, regression testing, and automated testing, this particular product. We do also, however, offer similar tools more focused on performance. So what we can do is we have what we call a mobile application monitor. And that’s going to allow you to run scheduled scripts at intervals against your application from different locations around the world to get an idea about how it’s performing, different service thresholds, how long things are taking. So maybe it doesn’t take more than 30 seconds to log in to my app.

It will alert me, send me an email. But that’s part of a slightly different product and not a core part of the DeviceAnywhere test automation framework.


All right. Next question, if I have to automate iPhone and physical devices, will it be possible to run locally?


It’s going to be dependent on which handset and OS version that you’re looking to – generally speaking, the newest version of iOS isn’t available on local software devices. It’s only available in our integrated devices because there is some work that needs to go into developing software that they will interface with that OS. I know with our hardware integrated devices, those are available Day 1. The new OS version comes out. It’s ready to go right on those devices with the software devices. It’s going to depend on OS version. And a lot of times, it’s just availability.


Thank you. Next question, is your Jenkins plugin available through the Jenkins plugin pool? If so, what is the name of the plugin?


So the plugin is not in the Jenkins plugin pool publically accessible. It is something that, if you’re interested in using it, we will provide you with an HTI file, which you would then use in that same interface instead of, it’s like the fourth tab over. If you click Advanced, and then part way down in the options, one of the options is to install a file. You drop the HTI file and click upload, and it will install the plugin without any restart or other things necessary.


All right. Next question, would it be possible to test different localization and different network providers?


So it is. We have devices in a couple of different locations. We have most of our devices are going to be in California. We also have some devices in the UK, Germany, as well as I believe Texas. So we have a few different locations. And again, there’s also the option of having devices hosted on premises or, if you have a data center that you’d prefer to have the devices hosted, you can also do that as well. That’s something we support. So a lot of our devices are out here in California. The rest of them are going to be in some of the places around the globe.


Thank you. Next question, can we use any programming language to run the tests, or are we locked into a specific language such as Java?


All right. So we talked a little bit about this before. This is actually – you’re going to be using Java to write the test cases if you’re going to be programming. Or you can also use Selenium, which is a slightly different form similar to Java. But things like C#, Python, those would not be available to attach the script. And you’d have to use our Java API.


All right. Thank you. This is going to be the last question. Can you test via browser installed on the devices such as Chrome, Firefox, or default installed browser?


Absolutely. So the ability, since these are real live devices, to install applications via the play stool or by pushing directly to the device, just like you would with any other device. So what you can do is install pretty much any browser you want to test against, and since it’s a live device, it will install just fine, and then you can do your test against it. The only limitations if you’re using our Selenium integration, you have to use the Selenium web driver browser. But that’s the same as if you were learning the Selenium test in general. And so that’s the one limitation. But otherwise, you can use whatever browser you choose.


Thank you very much, Joe. And I’m going to hand it back over to Josh for our last slide.


Great. Thanks, Josiah. And thanks, Joe, for taking the questions. It’s a great demo. We want to talk to you guys a little about how to get started. And if you have any further questions, we actually had a lot of questions about where can I get more information. So that’s what why we wanted to show you this before we finish our webcast today. First, you can click on the link here to get started, if you just want to contact Keynote directly, and you want to get access to the integration. You can also, if you want to learn more about the integration, click here.

You can download the presentation to share with your team.

And to learn more about Jenkins, as I mentioned earlier in the presentation, you just go to Thank you.

Duration: 44 minutes

Back to Top