By Aaron Rudger | February 11, 2013
The performance community has grown quite a bit over the past few years. Web Performance meetup groups now can be found in almost every large metro in the U.S. and around the world. The Velocity conference continues to grow with another venue added this year. And vendors have been providing better technology to help site owners monitor and improve performance. The result: many of the top websites we visit regularly deliver a pretty solid user experience.
It’s heartening to see that web performance seemingly matters to web businesses, and that the value of high performance has become understood.
And then the Super Bowl happens.
Each year’s edition of the big game represents a huge opportunity for advertisers to capitalize on their significant investment in air time by engaging consumers beyond the television—on their laptops, tablets and smartphones. Advertisers know that this produces huge surges in traffic, and without proper planning, can create a very poor customer experience.
This year, we closely monitored the advertisers across all 3 screens to see how well they prepared to engage with consumers. Here are the results, and some observations which should serve as reminders of how web performance considerations are just as important today as they were 5 years ago:
1.Mobile performance is tragically misunderstood
Over in our sister blog Mobility, we’ve talked about the differences between traditional desktop Web performance, and mobile performance. What seems clear is that most companies still do not understand how forcing the desktop experience down onto a smartphone impacts the end user experience. With no other Super Bowl advertiser is this more clear than Century 21 Real Estate. Their promotional site was not optimized for the smartphone, sending a staggering 94 objects to the browser. Because our smartphone testing agents connect to wireless carrier networks, the average response time during the game was over 54 seconds, compared to the 2.97 second average measured by our Internet Explorer agents. Such a huge difference illustrates how important it is to treat the mobile experience uniquely—even for tablet users.
On the other end of the spectrum, Axe presented a nicely optimized site for smartphone browsers. Coming in it nearly 7 seconds, Axe maintained a low page weight of 270KB. In contrast, one image on the Century 21 page was over 1,000KB. Axe also decided to provide their tablet visitors with the optimized smartphone site, instead of the fuller featured desktop site—emphasizing their focus on a high performing experience.
2. 40 million viewers… you should probably run some web load tests
While the 49ers repeated failures on defense and special teams were shocking to many fans, the most surprising crash was that of the Coke website.
You can see how the site availability deteriorates rapidly once the ads aired.
Fans of the iconic brand took to Twitter to vent their frustrations:
Coke reported that its campaign was a success by many measures. However, their public statements also reveal how significantly the brand probably under-estimated capacity for the website component of the campaign. Keynote worked with other Super Bowl advertisers to test their sites in advance of the game by generating real traffic from the Internet—a practice known as web Load Testing. Those advertisers planned for 1.5 – 3.0 million site visitors. Coke’s estimate was less than 1 million. Doing a bit of ‘Monday morning quarterbacking’ here, this low estimate likely explains much of why Coke’s sites performed so poorly during the Super Bowl.
By contrast, the online retailers (having weathered many a Cyber Monday) such as Best Buy performed flawlessly with 100% availability. Best Buy also delivered some of the fastest response times across all 3 screens.
3. The (good) old rules still apply
It was surprising to find that many the advertising sites did not make use of CDNs and common optimization techniques. It’s clear many site owners can benefit from taking a fresh look at their pages. So revisit those web performance best practices and take an inventory of how well your site is delivering end user experience.