Press Room: 2003


Eddie Bauer Camps Out at the Top for Performance on New Keynote Consumer E-Commerce Transaction Index; Staples and JC Penney Tops in Availability

Initial Results Released by Keynote Show Dramatically Variable Performance for Highly Competitive Retail Vertical


Etail 2003, Boston, Mass., — August 12, 2003 — Keynote Systems (Nasdaq: KEYN), today released the aggregate results of two weeks of data from its new Keynote Consumer E-Commerce Transaction Index, the retail industry’s first and only Web transaction performance benchmark index that measures the response time and success rate for executing a typical six-page online retail transaction on 10 leading retail Web sites. The results show dramatically varying performance and response times leading to the failure at times of almost 20 percent of transactions.

Leading the pack the weeks of 7/28 and 8/4 in average time to execute the transaction were Eddie Bauer and Amazon in the number one and number two spots respectively. During the weeks of 7/28 and 8/4 Eddie Bauer’s average response time was 7.18 seconds and 7.09 seconds respectively. Amazon’s execution speed averaged 11.07 seconds and 11.59 seconds respectively. Best Buy ranked tenth on Keynote’s Consumer E-Commerce Index both weeks at 24.20 seconds and 23.91 seconds respectively. The Consumer E-Commerce Index average for performance was 14.59 seconds the week of July 28. For the week of August 4 the index average was 14.87 seconds.

Staples held the number one spot in availability at 100 percent the week of 7/28. The week of 8/4, JC Penney held the number one spot in availability at 99.55 percent. Sears was in the number two spot in availability the week of 7/28 at 99.64 percent. Eddie Bauer was number two in availability the week of 8/4 at 99.46 percent. Best Buy was tenth in availability both weeks at 83.33 percent and 80.2 percent respectively. The E-Commerce index average for availability was 96.69 percent the week of July 28. For the week of August 4 the index was 96.21 percent.

Best Buy appears to have suffered a capacity-related outage on August 4th. Between 08:00 AM PDT and 3:00 PM PDT, the Transaction ran at a total availability of 10.94%. The majority of errors during this period were Navigation Failed errors on Page one. This was caused by the normal Best Buy home page being replaced by a page announcing that there were too many shoppers on the site and that customers should try again later.

If visitors did manage to get past the main page, most of the remaining transactions timed out attempting to login as a returning visitor using an e-mail address and a password.

“Our founder Eddie Bauer's commitment to customer satisfaction laid the foundation for this company to grow from one Seattle location to the tri-channel authentic American lifestyle brand it is today. Eddie Bauer's Web site has become a key channel for us in delivering outdoor-inspired product to our customer,” Troy Brown, divisional vice president, e-commerce said. “With the recent launch of our redesigned site, we are confident customers will have an even more enjoyable shopping experience. An early indication of this is the exciting recognition we received from Keynote's index.”

The Keynote Consumer E-commerce Transaction Index, introduced in May, is composed of the most active e-commerce sites on the Web, as determined by industry experts and analyst reports, and is the only industry benchmark that compares and contrasts the performance of a similar six-step Web transaction by a consumer purchasing a similar item on each site.

The 10 companies that make up the index include: Amazon (Nasdaq: AMZN), Best Buy (NYSE: BBY), Eddie Bauer (OTC: SPGLA), JCPenney NYSE: JCP), Office Depot (NYSE: ODP), Office Max (NYSE: OMX), Sears (NYSE: S), Staples (Nasdaq: SPLS), Target (NYSE: TGT), Wal-Mart (NYSE: WMT). These companies receive a significant majority of their revenue from online sales and a high volume of Web site visitors flock to them every day to complete transactions similar to the ones being measured. As such they are the de facto benchmark against which retailers doing business on the Web should measure themselves.

The data used to create the index is taken from actual online retail transactions Keynote performs and is measured from 10 U.S. cities (Boston, San Francisco, Detroit, Washington, D.C., Los Angeles, New York, Chicago, Philadelphia, Houston and Dallas) on an hourly basis Monday to Sunday from 5:00 AM to 9:00 PM Pacific time.

Performance (or “response time”) is defined as the aggregate time in seconds it took during the week to complete a multi-step transaction from the 10 U.S. cities. A transaction consists of six steps (that may represent six or more pages in a retail transaction): accessing the home page, searching for a product, viewing the product, adding the product to a shopping cart, proceeding to check-out, logging in to an account for purchase.

Availability is the “success rate” for completing the entire transaction.
The following tables show summary data of the measured performance (in seconds) and availability rates for the first two weeks from Keynote’s new Consumer E-commerce Transaction Index.

Week of July 28, 2003

RESPONSE TIME
Rank
Site
Time (seconds)
1
Eddie Bauer
7.18
2
Amazon
11.07
3
Office Depot
12.08
4
JC Penney
13.43
5
Target
13.77
Index
14.59
6
Staples
15.29
7
Walmart
15.89
8
Sears
17.81
9
Office Max
23.12
10
Best Buy
24.20
AVAILABILITY OR “SUCCESS RATE”
Rank
Site
Success Rate (percent)
1
Staples
100.00
2
Sears
99.64
3
Eddie Bauer
99.54
4
JC Penney
99.44
5
Amazon
98.81
6
Office Depot
98.81
7
Wal-Mart
98.18
8
Target
97.88
Index
96.69
9
Office Max
86.67
10
Best Buy
83.33

Week of August 4, 2003

RESPONSE TIME
Rank
Site
Time (seconds)
Last Week
1
Eddie Bauer
7.09
(1)
2
Amazon
11.59
(2)
3
Office Depot
12.18
(3)
4
Target
13.74
(5)
5
JC Penney
13.79
(4)
Index
14.87
6
Staples
15.53
(6)
7
Walmart
16.70
(7)
8
Sears
17.32
(8)
9
Office Max
22.95
(9)
10
Best Buy
23.91
(10)
AVAILABILITY OR “SUCCESS RATE”
Rank
Site
Success Rate (percent)
Last Week
1
JC Penney
99.55
(4)
2
Eddie Bauer
99.46
(3)
3
Staples
99.44
(1)
4
Target
99.01
(8)
5
Sears
98.96
(2)
6
Office Depot
98.93
(6)
7
Wal-Mart
98.73
(7)
8
Amazon
98.21
(5)
Index
96.21
9
Office Max
84.93
(9)
10
Best Buy
80.20
(10)

The Growth of Online Retailing and Consumer Perceptions

Forrester Research, Inc reports that sales made over the Internet will more than double to $229.9 billion and account for 10 percent of overall U.S. retail sales in 2008 as more people use the Web for purchases. About 5 million U.S. households will make their first online purchases each year for the next five years. At which time, 63 million U.S. homes will be shopping on the Internet the research group said. That level of explosive growth in sales makes it critical for transactions to perform at a high level in the fiercely competitive world of retail where a competitor is just a click away.

Enterprise Management Associates reports that up to 28% of attempted online purchases fail. Worse still, 42% of consumers experiencing a technical failure will move on to make their purchases elsewhere never to return to the faulty site.

“Certain buyers exhibit positive ‘retainability’ traits. Retailers who can find these customers and offer good (if not necessarily the best) prices, an easy to use Web site experience and great customer services will have the opportunity to profitably retain them,” said Jupiter analyst Robert Leathern. A new Retail Consumer Survey by Jupiter Research, Inc., found that as long as the merchant provides a good online experience, 29% of online consumers are inclined toward merchant loyalty as well as being relative high spenders.

The Keynote Methodology

Keynote pioneered the use of vertical indices and benchmarks for Web application performance over four years ago.

The new Keynote Consumer E-Commerce Transaction Performance Index is the retail industry’s most accurate Web transaction performance benchmarking index for improving the performance of the retail industry’s most critical online transactions. Keynote measures transactions from all the major metropolitan areas in the United States and places its transaction measurement computers on the most widely used, familiar and recognizable Internet backbone and service providers in the United States including Uunet/Worldcom, Sprint, AT&T and Qwest.

Additionally, Keynote’s global network of measurement computers run standard Microsoft Internet Explorer browsers on Windows/2000 to obtain the most accurate and representative transaction performance data possible in the banking industry today. It does not use an “emulator browser” as used by some vendors for obtaining similar-type index data.

About the Keynote Consumer E-commerce Transaction Index

The Keynote Consumer E-Commerce Transaction Index is the retail industry’s first and only Web transaction performance benchmarking index for optimizing the experience of its online customers.

The Keynote Consumer E-Commerce Transaction Index provides customers with a new and ideal method for evaluating and improving Web transaction performance relative to direct competitors in their market space, rather than simply to a cross-section of general business sites.

The aggregate results from all sites included in the Keynote Consumer E-commerce Transaction Index will be published weekly in The Ecommerce Times®, the leading publication for e-business executives and IT decision makers who need to keep up with breaking news and the latest analysis and business strategies to keep ahead of the competition. (See separate press release issued today.) Complete index data that includes not only the aggregate and limited time-frame results as published here, but every data point taken 24/7 along with drill-down error analysis and more, is available from Keynote by monthly subscription only.

The Index accurately measures the performance and availability of Web transactions that represent a typical consumer purchasing a popular product at the most active e-commerce sites on the Web today.

The companies selected for inclusion in the Consumer E-Commerce index were ranked in the top percentile of all e-commerce companies based on visitor traffic, online revenues and consumer interest. Each Web transaction measured is representative of the sequence of steps that a consumer would perform on each company’s Web site leading up to the purchase of a product or item. The product or item selected in the transaction represents a popular item that a consumer would purchase on that particular Web site. The number and type of steps that are measured in each transaction uses a consistent methodology, allowing for a direct comparison from one retailer to another in the same market.

What The Keynote Consumer E-Commerce Transaction Index Measures

Each transaction is measured using the following six components:

  • Accessing the home page
  • Searching for a product
  • Viewing the product
  • Adding the product to a shopping cart
  • Proceeding to check-out
  • Logging into an account for purchase

Keynote’s transaction methodology provides the most accurate view of end user experience by taking its measurements using the market leading browser coupled with the most comprehensive geographically distributed infrastructure of measurement computers and backbone representation.

Performance data of the sites measured by The Keynote Consumer E-Commerce Transaction Index is taken using Keynote’s Transaction Perspective® service from US10 agents. Each transaction is measured hourly.

Background on Keynote’s Web Performance Indices

Keynote’s Web transaction indices leverage Keynote’s long history and experience developing Internet performance indices. Keynote launched the world’s first Web performance indices that assess page download performance (as opposed to transaction performance). Known as The Keynote Business 40 Internet Performance Index (measurements over T1 and T3 connections) and The Keynote Consumer 40 Internet Performance Index (measurements over dial-up and broadband connections), Keynote also launched the world’s first transaction performance index, The Keynote Broker Transaction Index. All today are considered the ‘gold standard’ for Web performance benchmarking.

Now customers can use Keynote’s growing line of industry-specific Web transaction indices to assess Web transaction performance against internal benchmarks and against competitors. Customers can then leverage Keynote’s additional monitoring, performance management, load, content and usability testing services to improve the overall performance of their e-business applications and infrastructure.

To build and maintain its growing line of Web performance indices, Keynote uses its unparalleled global infrastructure of over 1,500 strategically and statistically located measurement, testing and diagnostic computers representing Internet performance from 50 metropolitan areas worldwide.

Information about all Keynote performance indices can be found at http://www.keynote.com/solutions/solutions_pm_performance_indices_tpl.html.

Availability and Pricing

Complete index data that includes not only the aggregated and limited time-frame results as published here, but every data point taken 24/7 along with page, network component, content and error detail is available from Keynote on a subscription basis for $3,495.00 per month.

About Keynote

Keynote Systems (Nasdaq: KEYN), The Internet Performance Authority®, is the global leader in Internet performance management and testing services that improve the quality of e-business. Keynote’s services enable corporate enterprises to benchmark, diagnose, test and manage their e-business systems both inside and outside the firewall. Approximately 2,300 corporate IT departments and 16,000 individual subscribers rely on the company's easy-to-use and cost-effective services to optimize revenues and reduce downtime costs without requiring additional complex and costly software implementations.

Keynote Systems, Inc. was founded in 1995 and is headquartered in San Mateo, California. The company can be reached at www.keynote.com or by phone in the U.S. at 650-403-2400.

© 2003 Keynote Systems, Inc. Keynote, The Internet Performance Authority and Perspective are registered trademarks. Other trademarks are the property of their respective owners.

Contacts:

Della Lowe, Keynote Systems, Inc., (650) 403-3233, dlowe@keynote.com
Dan Berkowitz, Keynote Systems, Inc., (650) 403-3305, dberkowitz@keynote.com

Press Archives

2007

2006

2005

2004

2003

2002

Contact us

Press Contacts