Good, Okay, and Not so fast... a 3 screen look at holiday retail websites.
By Aaron Rudger | January 6, 2014
As we look back over the holiday retail period, we can draw some conclusions around how retailers are adapting to consumer needs.
Keynote monitored top retail sites over the holidays – from Thanksgiving Day through December 31 – to see how performance looked across the three screens of desktop, tablet and smartphones. We looked at the speed and success rates for transactions – entering the home page, searching, selecting and adding it to a cart.
Was 2013 the year retailers finally got to grips with delivering optimum performance across three screens? Not quite.
On the desktop we saw pretty consistent, good performance. The average time was 22.79 seconds to complete a transaction with just a slight dip during the third week of December, with the sites taking an average of 23.16 seconds. Not an amount a consumer would really notice. The fastest week was the first week of December, taking on 22.65 seconds. The bottom line is that the weekly averages stayed within a ½ second range.
Throughout this period Apple ruled as the fastest site, taking only 13.97 seconds for the transaction experience. Impressive.
The slowest sites were hampered by a large amount of images and third party tags. In some cases we saw nearly 7 MB of data being transferred, with pages reaching over 200 elements. By comparison, Apple’s site resulted in a trim 2 MB of transaction data, and never once saw a page with more than 100 elements on it. These are the things that make a huge difference to site performance.
The best success rate – meaning the ability to successfully complete the entire transaction - over this period was HP, hitting 99.62%. But overall all of the sites saw good success rates throughout December, with no major outages. The only sustained downtime – when a transaction could not complete - was a period between 03:00 – 05:00 am EST on December 27th, when the Dell site was unable to return search results. This downtime had limited impact due to the off-peak hours and because it was after the peak holiday season.
When it came to individual pages, the slowest page across the measured sites was the Search Results, coming up as the slowest across 46% of the sites measured. Why the slowdown? Long search application calls and a large amount of products showing up in search results leads to a large amount of image files being returned.
The second slowest section was the Home page, appearing as the slowest page across 30% of sites. This seemed surprising – you’d think sites would if nothing else, get the home page right. But often if it hasn’t been optimized – again, things like too many images and third parties – then that’s where the problems show up.
Over the holiday period the average transaction speed for tablets was 61.35 seconds. That’s a pretty painful amount of time for users to deal with.
Why so slow? We saw that all the sites we were measuring were serving up the desktop site on tablets. No-one offered an optimized tablet version, which like the smartphone sites would be mindful of the amount of images or third parties being served up and pay attention to the potential cellular connection and its limitations.
Our measurements were made over air cell connections, which is noticeably slower than a fast Wi-fi connection. With desktop pages not optimized for an over the air connection, some retailer pages contained hundreds of elements and reached over a megabyte per page, resulting in longer download times and a slower experience. If the tablet was connected through a broadband connection, we would expect the performance to be comparable with that of a traditional desktop experience.
Over on the smartphone side, we saw an average performance time of 32.13 seconds.
We saw that most of the sites, with the exception of the Apple Store, had page designs specifically optimized for the smartphone screen size and mobile network (3G), with fewer elements.
The fastest smartphone site we measured was Sony Style store, coming in with an average time of 19.72 seconds. Its site, optimized for a faster experience, regularly reduced the amount of new HTTP requests on multiple pages, resulting in speedier load times.
The slowest sites were coming in at a painfully slow average of 54.75 seconds. These speeds were primarily driven by the search page occasionally taking over 20 seconds to return results over the cell connection.
The slowest week in December was the first week, with an average performance time of 35.66 seconds. The fastest week was the fourth week in December, with the performance dropping to only 28.58 seconds.
The slowest page on the mobile side was by far the Home page, with it appearing as the slowest page on 76.92% of measured sites. That may sound surprising but in many cases the home page takes the biggest hit with DNS lookup and TCP connection delays compared to most other pages.
There were no significant performance spikes or outages during December for the mobile sites.
The lack of tablet-optimized sites is breathtaking given the surge in that and mobile devices. Maybe 2014 will be the year the three screens finally get their due. If time is money, then we have to assume all the online retailers want to address slow tablet and smartphone experiences.