Holiday Readiness Lessons: How to Avoid the Dreaded “Check Box” Load Test Trap
By Dave Karow | June 25, 2015
At Keynote, we have tested some of the largest and most ambitious web and mobile properties ever created. We’ve also tested smaller, but no less strategic, sites. Usually, the testing is “successful” meaning it proves the system is ready or, far more likely, it finds problems (big or small) with time left to fix them before revenue is put at risk.
When load testing fails and systems degrade unacceptably or crash as a result, it’s often for a very simple reason: someone tried to “check a box” to say load testing had been done, but didn’t have enough information, time or budget to do what it would take to make the testing valid.
At the highest level, this is just common sense. To be “valid” a load testing campaign must succeed in modeling the same performance challenges your digital channel will face in the real world. The benefit is huge: creating the same problems your paying customers would have encountered gives you an opportunity to fix those problems before customers arrive. Valid load testing is the closest you can come to a time machine (jumping ahead to your next big day) and a “do-over” button.
Picture it. Hundreds, thousands, or (if you are really big), millions of consumers streaming to your digital properties. Traffic records are being set and you see two, three or 10 times more sessions than you have ever seen in a single hour. Visitors are delighted with your offer and are eager to spend money. Just as everything is going right on the demand side, your site slows down or falls over.
This is the conversation you don't want to have:
Angry executive: “I thought we tested our site for this level of traffic?!”
Digital channel owner: “Yes, yes we did.”
Angry executive: “Why didn’t we find these problems?!"
Digital channel owner: “Our tests weren’t designed to test the conditions we encountered.”
Whether you choose our load testing service, or go it alone, we would like to save your reputation and revenue from harm by sharing nine tips for getting real value from your testing.