For further comments or questions about the testing methodology used for this index, please email us at public_services@keynote.com.
Topics
1. What is The Keynote E-Banking Web Transaction Performance Index, and how is it measured?
The Keynote E-Banking Web Transaction Performance Index shows the median response times and success rates for performing a standard home E-Banking transaction (log on, display account balance, log out) on selected banking Web sites from the ten largest U.S. metropolitan areas (Boston, Chicago, Dallas, Detroit, Houston, Los Angeles, New York, Philadelphia, San Francisco, and Washington, D.C.) on high-speed links attached to key points on the largest U.S. Internet Service Provider (ISP) backbones.
The published Index is based on measurements taken every hour between the hours of 8am and midnight Eastern Time, Monday through Friday. Keynote subscribers receive 24/7 measurement components (DNS lookup time, TCP connect round-trip time, etc.) for all the individual sites and the aggregated measurements. Subscribers can also investigate performance based on city, backbone, or agent using the advanced MyKeynote data portal.
The Keynote E-Banking Web Transaction Performance Index is measured by Keynote's Transaction Perspective. That service is used to measure download speed for well-connected major business users (T-1 to T-3 connection speeds) and as a way of measuring ISP peering performance. The measurement agents use standard Microsoft Internet Explorer running on Windows/2000. To ensure that any difficulties in downloads are not due to congestion on Keynote's connections to the Internet, Keynote's connections are always lightly loaded and low latency.
Measured performance depends on factors such as geographic location, backbone connectivity, and network infrastructure at each measurement location. The performance experienced by an individual user in one of the cities above may be better or worse than our measured average depending on how the user is connected to his ISP and how that ISP connects to the Internet.
2. How does the transaction work?
The standard Keynote E-Banking transaction begins by the measurement agents entering the bank's respective Web site through its banking home page and logging on to a banking account. (Keynote has established standard consumer banking accounts at all measured banks.) The transaction then looks at the individual checking account details page to obtain the balance. The next step in the transaction is to log out. Demonstration accounts or other non-standard accounts are not used for these measurements.
3. How are transaction measurements calculated?
Transaction success rates are the ratio of completely successful transactions to attempted transactions. Unsuccessful transactions include those in which any Web page in the transaction fails to download completely (including all of the page's content), those in which any error was detected on any page, or those that do not complete within a specified time limit. The time limit for a transaction is calculated by multiplying the number of Web pages in a transaction by 12 seconds. For example, a five-page transaction is allowed 60 seconds to run, while a seven-page transaction is allowed 84 seconds. The transaction times reported do not include any human interaction time.
Transaction performance is calculated as the average of each measurement agent's median measurement over the entire reporting period; that ensures that a few abnormal measurements do not distort the statistics. (For each measurement agent, all of the measurements for the entire measurement period are sorted, and the median measurement is chosen. The medians for all of the measurement agents are then averaged together to obtain the Index.) Measurements from unsuccessful transactions are not included.
Results are posted each Wednesday and represent the performance of each site during the previous week. The rankings for the sites in success rate and response times are published for the current week as well as noting previous rankings.
5. What happens when a software agent does not report data?
From time to time Web site changes and other factors may cause a site to be temporarily unable to be measured. In this case the site may be temporarily removed from the index. This typically indicates that the Keynote Transaction Perspective agents were unable to interact with the Web site for an extended period of time. This is not always due to troubles at the Web site and does not necessarily indicate real performance problems.
6. What changes have occurred to the Index since its inception?
This index was established on August 4, 2003. For a complete list of the recent changes in the index please click here.