Visitor Monitoring A Bright Spot in a Dark Economy | Keynote

A Bright Spot in a Dark Economy

Buffeted by a torrent of economic bad news week after week, month after month, pundits and average citizens alike are straining to find the slightest glimmer of good news — housing sales that don’t decline as much as last month, or a month when job losses fall below the 600,000 mark. Reports of even the most modest profits for financial institutions — though much of those profits are more the result of creative accounting than actual market performance — is enough to send their admittedly dismal stock prices on a major upward spike. As the economic crisis drags on and on, the metrics by which we judge our economy’s health have undergone a major recalibration.

At least one sector, though, continues to be a bright spot of performance and promise, even without adjusting expectations or financial reports: online retailing. While comScore reported that online sales for the 2008 holiday season were down three percent — the first decline since they started tracking e-commerce in 2001 1comScore: Q4 and Holiday Online Retail Sales Decline, 1/6/09 — overall sales for the year grew 13 percent. Forrester Research is projecting 2009 sales to rise 11 percent, to US $156 billion, growing to 7 percent of overall retail revenue (from 6 percent in 2008) 2BizReport, “Forrester: Growth forecast for 2009 online retail sales,” by Helen Leggatt, 1/30/09; Forrester projects US $229 billion in online sales by 2010 3Forrester Research, US Online Retail Forecast, 2008 to 2013 by Patti Freeman Evans with Vikram Sehgal, Cristina Bugnaru, Brendan McGowan, 2/2/09 (updated 3/4/09). Amazon continues to be the darling of Internet retailing, posting a first-quarter revenue increase of 18 percent overall, to US $4.89 billion, with a 21 percent surge in North American sales.


Traffic to the major online retail sites is growing at an even faster rate than sales, according to some reports and anecdotal data from Keynote analysts. Speculation is that consumers are cutting back on shopping trips outside the home — perhaps as a belt-tightening effort to resist impulse buys and stay “on the list.” At the same time, they are increasing their online trips, and visiting more sites to search out a better deal for the items they do want to buy.

ComScore reports visitors 4comScore: Q4 and Holiday Online Retail Sales Decline, 1/6/09 to Amazon sites grew seven percent, to 76.2 million visitors, for the shopping period of December 1–24, 2008. Wal-Mart was up four percent for the same period, to 51.5 million visitors, and Apple jumped 19 percent, to 35 million visitors.

“There’s a larger jump in volume than in revenue,” says Paul Kohler, marketing manager for load testing at Keynote. “There’s a dramatic increase in the number of people visiting sites, because they’re jumping around and comparing prices and taking a look at their various options before they make a purchase decision. The conversion rate per page click has decreased quite a bit, however, for a number of retailers.”


Conventional wisdom says that an economic downturn presents an opportunity for businesses to gain market share from weaker competitors, and thus be poised for major growth as the economy rebounds. Online retail may prove to be the test case worth watching. Unlike brick-and-mortar retail, which is subject to the vagaries of geographic markets, store locations, and other fixed variables, the playing field online is comparatively level. One online storefront can be as interesting and compelling as the next, and all are equally accessible to every Internet visitor, anywhere in the U.S. or the world.

Today, with record traffic levels and more and more shoppers visiting multiple sites before making a purchase, online retailers are in a position to make or break their business prospects for years to come. It is an urgent time for site operators to ask themselves the hard, critical, fundamental questions: How does my site’s user experience compare to my competition? How easy is it for visitors to find what they’re looking for? Are we making suggestions to boost total ticket? How well does my site perform overall…and for critical processes including search and checkout? Is my fulfillment operation delivering complete customer satisfaction?


“The base of the performance pyramid is availability,” says Dan Richards, senior product manager at Keynote. “It’s that question we used to always ask in the old days, ‘Is the dang site up?’ The next level is site performance. How long does it take a user in London to browse, buy, and check-out? And then at the top of the pyramid is content that resonates with customers and is easy to use.”

In the fast-paced world of retail, sites change even more quickly than the fashions or electronic products they may be selling. Site changes and updates are an ongoing process, and always have the potential to negatively impact performance.

“The reality in the business world is that most people are doing a three- to six-week release schedule,” says Keynote’s Director of Global Testing Services Donald Foss. “And I don’t mean tiny little patches going in. We’re talking about the actual releases. And even if you’re not doing that kind of release schedule, if you’re just doing patches — once you drop enough patches in, it could be worse than a major release because you have patch upon patch and you really don’t know how well they all interact together.

“So for your major sites that are more active and constantly rolling out new features, we recommend testing on a monthly basis.”

On the other side of the equation, site traffic has the potential to significantly impact performance, an especially important factor given the increasing traffic levels discussed above. Stress tests have been making headlines this spring, as the U.S. government tries to evaluate the viability of large financial institutions. Web site stress testing, more typically referred to as load testing, is a routine practice for high-performing sites — not something done just for Black Friday or some other busy season.

“The best practice is definitely to have it all year-round,” Foss says. “One of our client companies who does a huge business on Black Friday runs their first load test of the calendar year in February. Then we have other companies that habitually come in October. A couple of years ago, we were actually load-testing a customer two days before Thanksgiving. We had to tell them, you guys are going to be down Friday morning and you might as well just get a good night’s sleep the night before. There’s not a thing we can do for you because you waited way too late.”

Effective load testing is a scientific, systematic process that requires outside agents. Because no matter how well internal servers are performing, no matter how good the internal testing numbers look, it’s not a real picture of the performance and experience for the actual user on the other side of the city, the country, or the world. It requires a statistically reliable sample of virtual users dispersed in the appropriate geography, conducting virtual sessions that involve all the navigation and transactions encountered on the site.


It can be overwhelming being an online shopper. More sites are selling more products, beyond their core brand franchise. As it so often is, Amazon was at the front of this trend when it expanded from books into a host of consumer product categories. Zappos, once known simply as the online place to buy shoes, now markets every type of wardrobe item, as well as kitchen gear, electronics, and more.

Guiding visitors through large and diverse inventories — helping them to find what they want, and offering suggestions that might interest them, as a retail clerk would do — falls to the search function of the website. Search has grown in sophistication and relevance in an effort to keep up with ever more demanding consumers who are just a click away from looking for a better experience.

Zappos has recently re-tooled its entire search function using open-source technology (see accompanying interview with Zappos’ Brent Cromley). Users are able to specify more criteria and drill down further, and results are better-ranked for relevance. Zappos is also testing another site,, that uses like-product association to serve visitors a whole range of product suggestions that are similar to the specific product they’ve searched for. And in still another channel to deliver a satisfying store experience, VIP Zappos offers personal shopping service real-time with an online associate who is able to provide suggestions and steer the visitor through a variety of product.

Another tactic that some online retailers are successfully adopting is faceted search, where every page is drawn or run by a search engine. In addition to the main results that take up the majority of the screen, a side rail presents options to refine the search, or presents other suggestions. Best Buy was an early adopter of faceted search.

Sears has taken still another approach to enhance the search experience. “For performance reasons, what Sears started doing is caching a lot of the search on the client side,” explains Foss. So while you may only see ten products on the screen, it actually brought in 200 products behind the scenes while you were looking at the first ten — and they keep coming. So that when you start looking at other products and moving through them, there’s actually no server interaction at all. It’s already been brought client-side for you. That delivers tremendous speed benefits.”


Product, price, performance, search — a host of factors contribute to the user experience, which ultimately determines whether a visitor fills their shopping cart and proceeds to checkout. These are the fundamentals. On top, add video and Flash features, customer reviews, wish lists, product configurators, and the gamut of Web 2.0 functionality. And factor in the offline components, including fulfillment, return policies, and return shipping.

It’s a complex, multi-faceted challenge to build and maintain a retail Web site that delivers a satisfying customer experience that keeps them engaged all the way through checkout. But in these times of increased online retail shopping traffic, those sites that can deliver stand to capture a bigger share of the still-growing online retail pie, and an even bigger slice when the economy turns around.


Keynote Retail Apparel Index, Week Beginning 4.13.09

Week Starting 13 April 2009

Rank by Speed (seconds)
Rank Target Response Time (seconds) Rank Last Week
1 Abercrombie 2.53 1
2 Macys 5.25 2
3 J Crew 6.92 5
4 Eddie Bauer 7.30 7
5 Foot Locker 8.27 8
  Retail Apparel Index 8.51  
6 Zappos 8.76 9
7 Neiman Marcus 9.46  
8 Nordstroms 9.92 10
9 Saks Fifth Avenue 11.17 11
10 Sears 16.21  
Rank by Success Rate (percentage)
Rank Target Success rate (%) Outage Hours Rank Last Week
1 Abercrombie 100.00 0 3
2 Eddie Bauer 99.87 0 1
3 Nordstroms 99.87 0 7
4 Zappos 99.79 0 2
5 Foot Locker 99.75 0 7
6 Sears 99.74 0  
7 Macys 99.49 0 5
8 Neiman Marcus 98.86 0  
  Retail Apparel Index 98.86 0  
9 J Crew 98.13 1 9
10 Saks Fifth Avenue 93.59 6 11

The Keynote Retail Apparel Index tracks performance for ten leading sites. It measures the time to log onto the site, search for an item, add it to the cart, and check-out, without actually logging in as a shopper.


Back to Top