For further comments or questions about the testing methodology used for this index, please email us at public_services@keynote.com.
Topics
1. What is The Keynote Brokerage Web Transaction Performance Index, and how is it measured?
The Keynote Brokerage Web Transaction Performance Index shows the average response times and success rates for creating a standard stock-order transaction (log on, obtain a stock quote, create an order to buy stock, and log out before confirming the order) on selected brokerage Web sites from the ten largest U.S. metropolitan areas (Boston, Chicago, Dallas, Detroit, Houston, Los Angeles, New York, Philadelphia, San Francisco, and Washington, D.C.) on high-speed links attached to key points on the largest U.S. Internet Service Provider (ISP) backbones.
The published Index is based on measurements performed every 15 minutes between the hours of 9am and 4pm Eastern time, Monday through Friday. Keynote subscribers receive 24/7 measurements and aggregated measurement components (DNS lookup time, TCP connect round-trip time, etc.) Subscribers can also investigate performance based on city, backbone, or agent.
The Keynote Brokerage Web Transaction Performance Index is measured by Keynote’s Transaction Perspective. That service is used to measure download speed for well-connected major business users (T-1 to T-3 connection speeds) and as a way of measuring ISP peering performance. The measurement agents use standard Microsoft Internet Explorer running on Windows/2000. To ensure that any difficulties in downloads are not due to congestion on Keynote’s connections to the Internet, Keynote’s connections are always lightly loaded and low latency.
Measured performance depends on factors such as geographic location, backbone connectivity, and network infrastructure at each measurement location. The performance experienced by an individual user in one of the cities above may be better or worse than our measured average depending on how the user is connected to his ISP and how that ISP connects to the Internet.
2. How does the transaction work?
The standard Keynote Brokerage transaction begins by the measurement agents entering the brokerage's respective Web site through its home page and logging on to a brokerage stock account. (Keynote has established standard consumer stock accounts at all measured brokerages.) The transaction then obtains a stock quote and enter and preview an order. The next step in the transaction is to log out before confirming the order. Cancel order may be necessary before logout. Demonstration accounts or other non-standard accounts are not used for these measurements.
3. How are transaction measurements calculated?
Transaction success rates are the ratio of completely successful transactions to attempted transactions. Unsuccessful transactions include those in which any Web page in the transaction fails to download correctly and completely, those in which any error was detected on any page, or those that do not complete within a specified time limit. The time limit for a transaction is calculated by multiplying the number of Web pages in a transaction by 12 seconds. For example, a five-page transaction is allowed 60 seconds to run, while a seven-page transaction is allowed 84 seconds. The transaction times reported do not include any human interaction time. Since each of the indexes measure the ability to complete expected actions, actual number of pages may vary from site to site depending on site design, log-in requirements, or placing stock order process.
Transaction performance is calculated as the average of each measurement agent's median measurement over the entire reporting period; that ensures that a few abnormal measurements do not distort the statistics. (For each measurement agent, all of the measurements for the entire measurement period are sorted, and the median measurement is chosen. The medians for all of the measurement agents are then averaged together to obtain the Index.) Measurements from unsuccessful transactions are not included.
4. When is the Index updated?
Results are posted each Wednesday and represent the performance of each site during the previous week. The rankings for the sites in success rate and response times are published for the current week as well as noting previous rankings.
5. What happens when a software agent does not report data?
From time to time Web site changes and other factors may cause a site to be temporarily unable to be measured accurately. In this case the site may be temporarily removed from the published weekly reports. This typically indicates that the Keynote Transaction Perspective agents were unable to interact with the Web site for an extended period of time. This is not always due to troubles at the Web site and does not necessarily indicate real performance problems.
6. What changes have occurred to the Index since its inception?
This index was established on September 2002. For a complete list of the recent changes in the index please click here.