Results from the DeviceAnywhere Test Center Customer Survey
By Rachel Obstler | April 12, 2013
A few months ago Keynote sent out a survey to our DeviceAnywhere customers and we wanted to share some of the results.
The survey first asked a few demographic questions “What is your role in your company?”.
It’s somewhat surprising that only about half of users are in QA. More developers and product/project managers were using our product than expected. Also interesting were the “Other” responses—mostly because they indicate that this was a poorly worded question! Many of these responses were “Manager/Director/VP of Engineering/QA.” The word “role” was interpreted as job title, when it was intended to mean functional area. That’s one of the good and bad things about surveys – when you get the question wrong you don't get the best results, but it’s also very easy to identify sub-optimal questions so you can improve them for the next time.
The survey also asked users “What type of mobile products do you test using DeviceAnywhere?” and allowed multiple answers.
It's interesting to see such a high % of customers testing Websites, where in the past Native and Hybrid Apps has been the most popular responses for our product. It does highlight the priority that companies are placing on providing quality mobile optimized versions of their websites. Here’s a good reason why in this quote from a Forbes article: "Two-thirds of smartphone users say a mobile-friendly site makes them more likely to buy a company’s product or service." You can see the full article here.
Now perhaps the most important question in the survey – we asked about 5 device dimensions, including Reliability (do the devices work as expected), Cleanliness (do the devices have evidence of past use), Speed (interaction with the devices), Availability (are the devices available when you want to use them, and not offline or busy) and Model Availability (do we offer the device models you need). Users were asked to rank the importance of improving each of those dimensions. The chart below shows how many respondents said a given dimension was the most important to improve.
The only surprise here is that so few respondents said Cleanliness was the most important item to improve. However, in thinking about this, it makes sense. Getting on a cluttered device is a big annoyance; however, it will not completely block testing, unlike just not having the model you need. So, this is not an endorsement of “dirty devices”—we are still working on improving the cleanup process! However, it does mean that Keynote needs to place a very high focus on the other items on the list, most important of which are device interaction speed and new model availability. We are in various stages of working on all of the above items, and future blog posts will have more details once we are closer to making improvements available.
Thanks to everyone who gave us their feedback; this information is invaluable to the product, engineering, and operations teams in helping us prioritize efforts that will provide the biggest benefit to our customers. We always want to hear from our customers, so please either respond to this post or send us an email anytime at firstname.lastname@example.org.