For further comments or questions about the testing methodology used for this index, please email us at public_services@keynote.com.
Topics
1. What is the Keynote Credit Card Web Transaction Performance Index, and how is it measured?
The Keynote Credit Card Web Transaction Performance Index measures the performance and availability of going to a selected credit card site and logging in to conduct the appropriate intended actions and checking or signing out. All measurements are taken from the ten largest U.S. metropolitan areas (Boston, Chicago, Dallas, Detroit, Houston, Los Angeles, New York, Philadelphia, San Francisco, and Washington, D.C.) on high-speed links attached to key points on the largest U.S. Internet Service Provider (ISP) backbones.
The sites that appear in the Index were selected based on publicly available information published for the most commonly visited credit card account access Web sites from major industry resources and internal research data.
The published Index is based on measurements taken every hour between the hours of 8am and midnight Eastern Time, Monday through Friday. Keynote subscribers receive 24/7 measurement components (DNS lookup time, TCP connect round-trip time, etc.) for all the individual sites and the aggregated measurements. Subscribers can also investigate performance based on city, backbone, or agent using the advanced MyKeynote data portal.
The measurements in The Keynote Credit Card Web Transaction Performance Index are taken using Keynote's Transaction Perspective service. This technology uses the Internet Explorer browser and measures download speed for well-connected major business users (T-1 to T-3 connection speeds) and it can also reveal issues pertaining to ISP peering performance. The measurement agents are uniform in configuration and run Microsoft Internet Explorer on Windows 2000. To ensure that any difficulties in downloads are not due to congestion on Keynote's connections to the Internet, Keynote's connections are always monitored for acceptable latency by Keynote’s own Network Operations Center (NOC).
Measured performance depends on factors such as geographic location, backbone connectivity, and network infrastructure at each measurement location. The performance experienced by an individual user in one of the cities above may be better or worse than our measured average depending on how the user is connected.
2. How does the transaction work?
The standard Keynote Credit Card Web Transaction begins by the measurement agents entering the credit card’s respective Web site through its credit card home page and logging on to a credit card account. Keynote has established standard consumer credit card accounts at all measured credit card companies. The transaction then looks at the recent charge info in the account. The next step in the transaction is to log out. Demonstration accounts or other non-standard accounts are not used for these measurements.
3. How are transaction measurements calculated?
Transaction success rates are the ratio of completely successful transactions to attempted transactions. Unsuccessful transactions include those in which any Web page in the transaction fails to download correctly and completely, those in which any error was detected on any page, or those that do not complete within a specified time limit. The time limit for a transaction is calculated by multiplying the number of Web pages in a transaction by 12 seconds. For example, a five-page transaction is allowed 60 seconds. The transaction times reported do not include any human interaction time (or client time). Since each of the indexes measure the ability to complete expected actions, actual number of pages may vary from site to site depending on site design, log-in requirements, or log-out process.
Transaction performance is calculated as the average of each measurement agent's median measurement over the entire reporting period; that ensures that a few abnormal measurements do not distort the statistics. (For each measurement agent, all of the measurements for the entire period are sorted, and the median measurement is chosen. The medians for all of the measurement agents are then averaged together to obtain the results for the index) Measurements from unsuccessful transactions are not included to calculate the medians.
4. When is the Index updated?
Results are posted each Wednesday and represent the performance of each site during the previous week. The rankings for the sites in success rate and response times are published for the current week as well as noting previous rankings.
5. What happens when a software agent does not report data?
From time to time Web site changes and other factors may cause a site to be temporarily unable to be measured accurately. In this case the site may be temporarily removed from the published weekly reports. This typically indicates that the Keynote Transaction Perspective agents were unable to interact with the Web site for an extended period of time. This is not always due to troubles at the Web site and does not necessarily indicate real performance problems.
6. What changes have occurred to the Index since its inception?
This index was established on September 2004. For a complete list of the recent changes in the index please click here.
