|
September 14, 2006
PRESS RELEASE
Leading VoIP Providers Outperform Regular Phone Carriers in Overall Audio Quality, Keynote Reports
- VoIP providers still lag behind PSTN in important areas, including audio delay; 12 leading VoIP providers
studied and ranked in new competitive intelligence study
- Among VoIP providers, VoIP digital phone service ranks first for reliability and audio clarity
- VoIP service providers show overall improvement over the last year’s results in key
performance indicators such as service availability and audio clarity
- Competing VoIP providers in New York and San Francisco markets, including digital cable, VoIP
phone and PC-based software phone, included in study – all benchmarked against PSTN
- Keynote study is first to include last-mile – including media and VoIP adapters –
to provide true consumer perspective
- Voice service quality trend analysis included in study to help VoIP providers understand
impact of infrastructure changes over past six months
SAN MATEO, Calif., – September 14, 2006 – New insights into the performance of
leading VoIP providers was released today by Keynote Systems (Nasdaq: KEYN), the global leader in Internet and
mobile test and measurement services. Keynote’s third VoIP competitive intelligence study revealed that
overall VoIP quality has improved across the board since Keynote’s last study in December 2005 and that
the leading VoIP providers have actually surpassed PSTN (traditional phone service) in overall audio quality,
but still lag behind PSTN in audio delay.
Twelve leading VoIP providers are part of the benchmark study that includes AT&T (NYSE: T), Comcast
(Nasdaq: CMCSA), Lingo, Packet8, Skype (Nasdaq: EBAY), SunRocket, TimeWarner Cable (NYSE: TWX), TrueVoice,
Verizon (NYSE: VZ), Vonage (NYSE: VG), Vonics and Windows Live Messenger (Nasdaq: MSFT). The complete Keynote
VoIP competitive intelligence study, including detailed rankings, is available for purchase.
In order to benchmark and rank the quality of consumer VoIP services, Keynote measured the relative
performance of the leading VoIP providers in the New York and San Francisco markets, including digital
cable, adapter-based VoIP (hard phone) and PC-based software (soft phone) services, as well as the
performance of leading VoIP providers against PSTN service in those cities. Keynote then rated the
leading VoIP service providers on critical performance factors that influence the end-user experience
using Keynote Voice Perspective®, which is Keynote’s VoIP quality test and measurement product.
Survey Results Reveal Overall Improvement in VoIP Quality, Variations Between Cable and DSL
Based on the results of the survey, which was conducted over a one month period from Aug. 1-Aug. 31, 2006,
Keynote found that overall reliability among the various competing VoIP providers had improved across the board
and that the leading digital cable providers had in fact outperformed PSTN in overall reliability. Overall
reliability is a computed index score based on performance measurements in three performance factors:
service availability, average number of dial attempts and dropped calls.
Leading digital cable VoIP providers were also found to deliver better audio quality than the competition,
with the leading cable providers achieving excellent audio responsiveness (a measure of audio delay) and audio
clarity (measured by Mean Opinion Score, or MOS), two key contributors to overall audio quality. However,
there was still room for improvement among the rest of the pack, with 10 of 12 VoIP service providers studied
achieving less than a 4.0 MOS, which is considered to be “toll quality,” that is, comparable to
the audio quality of a toll call over PSTN.
Despite the shortcomings of the lower-ranked service providers, the overall average MOS of the VoIP providers
studied continues to improve over time, with the overall average MOS of 3.58 reaching levels comparable with
GSM mobile phone quality. In Keynote’s December 2005 study the overall average MOS among VoIP providers was 3.55.
The study also examined the relative performance variations of the various VoIP service providers (as well as against
the PSTN benchmark) during peak versus non-peak hours in terms of audio delay and Mean Opinion Score. It had been thought
that cable modem subscribers would suffer overall service degradation during peak hours (8:00 PM-1:00 AM EDT), however
the study revealed that while cable modem subscribers did experience greater instances of audio delay during peak hours,
audio clarity (as measured by Mean Opinion Score) was not affected by the increased traffic associated with peak hours.
DSL connections, on the other hand, were found to deliver more consistent peak versus non-peak audio delay performance,
but were less consistent as measured by MOS.
Although the top performers in the consumer VoIP services market have improved the quality of call audio, calls placed
on VoIP phones continue to exhibit considerably more audio delay than calls placed on traditional PSTN phones. This audio
delay can cause callers to talk over each other, leading to conversational disruption and missed information, which can
create frustration among users, especially in a business setting.
Study Focuses on End-User Experience and Includes Skype and Windows Live Messenger
The third Keynote VoIP Competitive Intelligence Study rates the relative performance of PSTN, Digital Cable, VoIP hard
phone and VoIP soft phone service providers (such as Skype and Windows Live Messenger) in the New York and San Francisco
metro areas. It compares VoIP service providers based on reliability and audio clarity over consumer cable and DSL lines
and evaluates network carrier performance on end user perceived call quality.
The study sought to identify industry trends in service level performance since the last Keynote VoIP study and evaluate
the range of performance between the best VoIP service providers and the worst. The study also examined variations between
the peak and prime-time performance of VoIP providers over various media and pinpointed the strengths and weaknesses of
each service provider and various voice service technologies.
The current study was expanded from eleven providers in the second study (results announced on January 25, 2006) to
12 providers (and 13 services). Data for the study was collected over a one month period, from August 1 through
August 31, 2006.
The Keynote study provides an objective assessment of the critical performance factors that affect end-user perception
of a VoIP service. While several providers and networks did well in certain areas, no single provider or network dominated
the study in all metrics considered. However, the results of the study indicate that in the last year, the industry as a
whole has shown marked improvement in the key performance indicators such as Service Availability and Average MOS.
Analysts estimate that residential adoption of VoIP service will grow to over 26 million homes in 2008 in the US, up
from 6.5 Million in 2004. The SMB and enterprise market forecasts are equally aggressive. Even so, VoIP reliability and
audio clarity remain important perceived factors that limit the widespread adoption of VoIP in consumer markets.
Additionally, a high rate of customer churn based on dissatisfaction with service levels makes it difficult for carriers
to break even. Some analysts estimate it can take as long as four years for some VoIP service providers to recoup the
marketing and other costs associated with luring a subscriber. While VoIP providers can take heart at the strides
they have made in improving overall audio quality since the last Keynote study, there is still work to be done in
order to address consumer perceptions about overall VoIP performance versus regular telephone service.
“As VoIP continues to move into the mainstream and challenge the incumbent carriers in major markets nationwide,
consumers have started focusing on two important differentiators, audio quality and pricing,” said Vik Chaudhary,
vice president of marketing and product management at Keynote. “As the results of the Keynote study indicate,
VoIP providers have overcome a major hurdle in the past seven months by addressing concerns about overall audio
quality, but they still have work to do to improve the consistency of their service levels during peak versus non-peak
hours and to decrease the variation in performance levels between the top performers and the rest of the pack.”
With this latest study, Keynote continues to extend its test and measurement expertise to embrace emerging technologies
such as VoIP, streaming and wireless, which are increasingly being adopted by both consumer and enterprise users. As
VoIP emerges as an influential technology that promises to cut consumer phone bills and enterprise communications
expenses, the Keynote rankings help assess overall VoIP quality and highlight market leadership among the various providers.
How the Study Was Conducted
The Keynote VoIP Competitive Intelligence Study was conducted using Keynote’s Voice PerspectiveŽ to evaluate
critical performance factors that affect the consumer’s experience with Internet telephone service.
Keynote placed local and long distance VoIP calls to destination phone numbers on a standard (PSTN) phone service.
Calls were placed from San Francisco and New York once every 30 minutes on every VoIP provider and network carrier
combination. A total of 125,000 calls were placed over a month-long period. Calls placed using competing VoIP
services were compared to traditional phone “toll quality” standards to determine what residential
customers can expect when switching from traditional phone lines to VoIP.
The full study, which is available for purchase from Keynote, includes detailed results, custom analysis of the
data and all raw measurement data.
About Keynote and Voice Perspective
VoIP service providers can leverage the competitive insight provided by Keynote VoIP Competitive Monitoring Solution,
an application of Keynote’s Voice Perspective product, to narrow the gap between their call performance in target
markets relative to PSTN and leading competitors. Combining Keynote’s competitive insight with internal network
monitoring tools will allow operations and quality assurance teams to adopt an outbound focus and assure a quality
experience for the end user. Keynote can design an ongoing VoIP Competitive Monitoring Solution in multiple markets
including multiple competitors. Service providers can leverage Keynote’s independent assessment and ongoing
actionable recommendations to align their resources and infrastructure investments with customer acquisition and
retention goals.
Keynote’s VoIP Competitive Monitoring Solution addresses the need of marketing and operations executives to
understand their performance relative to competitors, and to gauge the impact on the end-user experience of both
their infrastructure investments in new markets and their enhancements to services in existing markets.
Keynote VoIP Competitive Monitoring Solution provides ongoing performance monitoring on a monthly basis.
In addition to ongoing competitive monitoring reports delivered monthly, Keynote includes custom analysis to help
align operations teams with customer experience enhancement goals. To learn more about the Keynote VoIP Competitive
Monitoring Solution view: www.keynote.com/solutions/ci_voipStudies.html.
To learn more about Voice Perspective view: www.keynote.com/solutions/voice_perspective.html
Availability and Pricing
The Keynote VoIP Competitive Monitoring Solution is available immediately. Pricing is customized based on the
number of cities, providers and network carriers to be included in the ongoing measurements. For more information
email sales@keynote.com or call 1-800-KEYNOTE (1-800-539-6683).
About Keynote
Founded in 1995, Keynote Systems (Nasdaq “KEYN”) is the global leader in Internet and mobile test and
measurement services. Keynote helps approximately 2,600 corporate customers become “the best of the best”
online. The business premise supporting Keynote’s mission is: “Online businesses can’t manage what
they don’t measure.” As an independent and trusted third-party, Keynote provides IT and marketing executives
with unbiased benchmarking data, competitive analysis and operational metrics from the customer perspective. This data
measures service levels and customer experience of Web sites, broadband services and mobile communications.
Known as The Internet Performance Authority®, Keynote manages a market-leading infrastructure of 2,100 measurement
computers and mobile devices in over 120 locations and 70+ metropolitan areas worldwide that assess service levels and a
panel of over 160,000 consumers who participate in interactive Web site tests that assess user experience. These online
user experience tests capture customer attitude and behavior to answer the critical “why” behind the
“what.” Keynote’s geographically distributed measurement services, on-site monitoring appliances,
competitive intelligence and custom studies ensure that its customers outpace their competitors in online service levels
and overall user experience.
Keynote Systems, Inc. is headquartered in San Mateo, California and can be reached at www.keynote.com
or by phone in the U.S. at 650-403-2400.
Keynote, The Internet Performance Authority and Perspective are registered trademarks of Keynote Systems, Inc. Other trademarks
are the property of their respective owners. © 2006 Keynote Systems, Inc.
Editorial Contacts:
Josh Danson, Keynote Systems, Inc. (650) 403-3264, jdanson@keynote.com
Dan Berkowitz, Keynote Systems, Inc. (650) 403-3305, dberkowitz@keynote.com
|
