Visitor Monitoring High Stakes for the Rich Internet | Keynote

High Stakes for the Rich Internet

HTML5, Flash and the future of the rich Internet experience

Here we go again. Just when we thought we had Web 2.0 down pat, and all the marketing and C-suite folks were taught that "tall skinny latte no whip" is not what is meant by JavaScript, along comes HTML5. Actually, it's been coming for some time now. But ever since Steve Jobs picked a fight with Adobe over Flash, HTML5 has been a headline-grabber, dramatically boosting awareness for the language that will define how the Web looks and works. And it looks increasingly rich.

Rich media — the ubiquitous video, audio, and slide shows — and rich website applications with high levels of interactivity are now the lingua franca of the online experience. (Note the use of "online" instead of "Web," as more and more of these experiences are happening outside the browser — but that's a separate semantic discussion.) Up until now, plug-ins — Flash, Silverlight, and others — plus AJAX and regular old JavaScript have been the leading technologies for creating rich online experiences. Now, though, HTML5 has inserted itself into the mix, creating confusion, often conflict, and high hopes for a streamlined, unified way to deliver what users want.

So what's underneath all of the hype and battle cries? Is HTML5 the answer to our cross-platform, rich Internet prayers? Is Flash dead? Or for that matter, is the Web itself dead, as Wired recently proclaimed? How do apps fit into the mix? And most importantly, what does it all mean for delivering a dynamic, efficient, entertaining, high-performance experience to users?

The Shot Heard Round the Web

The great HTML5–Flash debate of 2010 started in the spring when Steve Jobs unequivocally (or so it seemed) announced that Flash would not be allowed on Apple's mobile devices. At The Wall Street Journal's D8 conference in June, Jobs explained, "Sometimes you just have to pick the things that look like they're going to be the right horses to ride going forward, and Flash looks like a technology that had its day but is really waning, and HTML5 looks like the technology that's really on the ascendancy right now."

Comments like these, and Jobs' open memo, "Thoughts on Flash," sparked an intense and sometimes impassioned debate on the relative merits of Flash and HTML5 — a debate that evidently will be simmering for some time to come.

Apple muddied the waters further in September when it abruptly lifted its restrictions on development tools — including Flash — that can be used to develop iOS applications for the iPhone, iPod and iPad (and presumably Apple TV, if rumors that it will run apps are true). So developers can now use Flash to create games and mobile apps on Apple devices, but Flash is still prohibited on Apple's mobile browser, so Flash-based websites are still left out in the cold. Adobe was "encouraged" by the announcement, and its stock jumped 12 percent on the news. Adobe also expressed its intention to resume development of its Packager for iPhone, a tool in Flash Professional CS5.

30-Second Update: HTML5

Jeremy Keith, in his new book, HTML5 For Web Designers, offers a good brief history of HTML. In a nutshell: In the beginning, there was Hypertext Markup Language, aka HTML, the creation of Sir Tim Berners-Lee, father of the World Wide Web. It governed the way pages looked and linked. Then there was HTML 2.0, published by the Internet Engineering Task Force. Then the World Wide Web Consortium, W3C, took over, revising and updating the standard; they published the official HTML 4.01 specification in 1999. From there we moved on to XHTML 1.0 (at about which time CSS started gaining traction with Web designers to get better control of the presentation of their Web documents). But when the W3C started work on XHTML 2.0, that's when the trouble began.

Unhappy with the direction the specification was going, and concerned about its real usability for Web designers, a group led by Opera, Apple and Mozilla split off and formed the Web Hypertext Application Technology Working Group, WHATWG. This group laid the foundation for HTML5 (explaining some of Apple's fondness for it). Ultimately, W3C adopted WHATWG's progress and got on the HTML5 bandwagon.

As it stands today, while HTML5 code is showing up in some Web pages and is supported to varying degrees by some, but not all, major browsers, there still is no published specification for HTML5, and projections vary widely as to when any of it will become official. The W3C's HTML5 working group expects it will be 2011, at the earliest, for HTML5 to reach "candidate recommendation" status, and that the ratification process could take until 2022(!). But the lack of official status is not likely to stop browser and Web developers from forging ahead and embracing HTML5's eagerly anticipated features.

Five things to love about HTML5

More than previous mark-up specifications, HTML5 separates content from presentation. It leaves many of the stylistic instructions to CSS. And it offers a simpler, more elegant solution to developers in five key areas: scalable graphics, animation, form input, browser-based caches, and multimedia without plugins. It is this last feature in particular that has caused so much stir.

No mere Flash in the pan

Flash is the most widely adopted plugin on the Internet, believed to be active on 99 percent of Web browsers. The reason can be summed up in one two-word word: YouTube. Users around the world depend on Flash for their daily dose of stupid cat videos, and they get to see them no matter what browser or operating system they're using. Everyone watching their favorite TV shows on Hulu is using Flash, too. Jeremy Allaire, who helped develop the Macromedia MX/Flash platform at Macromedia and is currently CEO of Brightcove, puts Flash's share of the online video market at 75 percent. Note that all of this refers to desktop, not mobile, video.

Flash was born as an animation and motion-graphics engine. By the early 2000s, though, it evolved into a full-featured, interactive audiovisual platform running in the browser. Creative Web developers began using Flash elements to add sizzle to their sites, and overachieving developers built (and are still building) whole sites in Flash. And Flash began powering a new generation of highly interactive rich Web applications, arguably ushering the era of Web 2.0. Corporate users, too, recognized the powerful capabilities of Flash for communications and application development. And Flash is integral to the browser-less Adobe Air application development platform.

The bottom line: Flash is a valuable technology that is not going to be eclipsed anytime soon. Adobe is clearly committed to Flash, and continues to develop, extend, and enhance it for a multitude of uses.

Decisions, decisions

So, Flash is a vibrant platform that offers a high degree of rich functionality, some of it best-of-breed. Sites and applications developed with Flash are available to virtually every non-mobile Web user, and likely to be available for most mobile users before long, except for those using Apple products.

HTML5 is an exciting, evolving Web coding language with extensive support from all of the Internet's heavy hitters — Google, Apple, Microsoft (albeit slowly), Mozilla, Opera, and yes, even Adobe. It promises to simplify and speed-up both the Web development and Web browsing experience, has native capability to handle multimedia and rich Internet functionality without external plugins, and has ambitions to be the technology that finally bridges all devices, mobile included.

A number of leading Internet voices argue that it's not an either-or scenario. Flash is the best platform for some applications, and HTML5 for others. But how is a developer to decide what the best technology is for the project that's in front of them? Assuming A) that the technology can perform the desired function and B) that the end product is available on user devices, it gets down to a matter of performance.

The proof is in the performance — if you can measure it

One point of agreement from everyone in the Web world, from technology purveyors to developers to users, is that faster is better. Users are universally impatient, and if you want them to stick with your site or application, it has to be fast.

The problem with Flash-based experiences, though, is that they lack the full transparency required for comprehensive operational measurement and diagnosis.

"You have the ability to detect when there are failures," says Vik Chaudhary, Keynote vice president of product management and corporate development. "You can detect performance problems — for example, the Flash application overall is taking 60 seconds instead of 15 seconds, But you have very limited ability to actually diagnose the failure. With interactive Flash monitoring, you have detection and alerting —'you've got a problem and it's taking 25 percent longer.' But diagnosis is very limited."

"When it comes to a proprietary player like Flash, it's really the Flash player that's running inside your browser that's running that application," Chaudhary explains. "So the testing challenge there is to be able to listen to all the chatter and events that are happening inside that Flash player. It's not an open standard; it's closed. Listening to the events happening in the application, you have to integrate deeply with Flash. That's a big technological challenge."

For example, many Flash applications on websites allow the user to click and manipulate objects or images. It's possible to track where users click, and to measure response times, but it's extremely difficult to know what Flash returns in response to a click, and therefore extremely difficult to diagnose the specifics of any problem that occurs. It is a test and measurement problem inherent with closed technologies.

"I can't think of any tools that are available to analyze the performance of Flash," says Steve Souders, Web performance engineer at Google and author of High Performance Web Sites and Even Faster Web Sites (visit his page at Amazon). "I've talked to people at large companies who have built internal tools, but none of them are published. And so we have all these tools — Firefox Firebug, Safari Web Inspector, Google Speed Tracer — all these tools that give us visibility into the performance of Web pages, and none of them give us any visibility into Flash."

See HTML5 run

Because it's an open technology (with the possible exception of video), HTML5 promises to lend itself to the robust performance monitoring, measurement, and diagnosis that's been available for other open Web technologies. And it is expected that the advances found in HTML5 will, if used judiciously, be a boon to overall page performance.

"There's a lot of wins in HTML5 for performance," Souders says. "There's app cache, local storage — I think those are going to be really big. There's some things, Web Workers, WebSockets, that I think are performance wins — I don't think they're useful for a large number of Web applications, but for the Web applications where they make sense, it's going to be a performance win."

Aside from any inherent speed gains, another big advantage of HTML5 vis-à-vis Flash is that, because it is an open technology, real browsers can be deployed in the field to measure and record virtually all of its activity and performance. As has been the case with HTML in all its flavors, and JavaScript, AJAX, etc., performance issues are transparent, and problem sources are more readily visible. There are far fewer black holes that require deep diagnosis.

VHS or Betamax?

One of the most anticipated features of HTML5 — and one of the most debated — is the ability to handle audio and video natively in the browser. A few simple lines of code is all it takes to embed A/V files, and, once browsers are updated to handle HTML5, the user would need no special plug-ins to run the files.

As mentioned above, Flash is by far the dominant force in online video. But HTML5 is already making inroads. Vimeo has introduced HTML5 video capability in a new universal player, enabling its videos to be viewed on mobile devices, including those from Apple. And Google has introduced a new, HTML5 mobile version of its site,, which is promoted as delivering better video with a better interface than Apple's own YouTube app.

The goal, of course, is to arrive at a standard that will operate across all platforms and devices. And that's the problem. It's VHS vs. Betamax all over again, this time between codecs competing to be included in the HTML5 spec.

On one side are the H.264 proponents. H.264 is the dominant video codec on the Web, used by most of the major video sites (and supported by Flash). Apple is leading the push to make H.264 the HTML5 video codec, and Microsoft has thrown its support, perhaps less vocally, behind H.264 as well. Coincidentally, patented technology from both Apple and Microsoft is part of the H.264 codec, so they both have a financial stake in the game.

In the other corner is the open-source WebM project based on Google's open-sourced VP8 codec, supported by Mozilla, Google, Opera, and more than 40 other publishers; Ogg Theora, another open-source codec, has also had the support of Mozilla, Google and Opera. Among the arguments for open-source is freedom from royalty payments, but now even those waters are being muddied. MPEG-LA, which manages patents and royalties for A/V codecs, announced this summer that it will offer a royalty-free H.264 license for sites that offer free video. That still leaves a lot of royalty liability, though, in terms of encoders, decoders, and commercial content.

So it's really difficult to say which way the standard will go. H.264 is the most widely used, but anything with Google behind it has the potential to be a contender. The good news is that, whichever standard wins out, HTML5 will make good on its promise to handle audio and video seamlessly across browsers and devices. A number of demonstration projects show off this capability (see photos).

Mobile: the driver behind the whole debate

It's likely that none of this discussion would be happening if not for the whirlwind ascendancy of mobile as an Internet access point. If we were just dealing with the desktop, the problems wouldn't be so complicated, the technologies so critical, or the stakes so high. The huge and still largely untapped mobile market is driving the train.

Morgan Stanley's Mary Meeker puts the tipping point when mobile will overtake the desktop Internet at sometime around 2014. This summer, George Colony, Forrester Research CEO, advised his fellow chief executives, "As CEO, start pushing your CIO and CMO toward App Internet. The Web was just a warm-up."

The "App Internet" will likely migrate back to the desktop as apps become blockbusters or indispensable productivity tools, but for now, it's a mobile phenomenon. And therein lies the all-too-familiar problem: rather than developing for a desktop that involves a few major operating systems and a handful of browsers, mobile brings in a number of additional, disparate operating systems and browsers, and literally scores of form factors for displays.

In many ways, native apps only perpetuate the problem. Today, for a marketer to cover the bulk of the smartphone universe requires building and maintaining an iPhone app, an Android app, and a Blackberry app — three different OSs, with Flash maybe or maybe not thrown into the mix. That's why a cross-platform solution is so sought after.

"In Asia, something like 18 percent of e-commerce on popular websites is done over mobile," Souders points out. "And for Amazon, it's something like 2 percent. For some reason, in the U.S., mobile users who are primarily going to U.S. websites are not doing a lot of e-commerce. How can those companies build an engaging experience to close that gap?"

"If you go down the path of, 'I have to build a native application for every mobile device that I'm targeting,' it's going to take years to close that gap," Souders continues. "I think if you build a mobile Web app that makes it easy and compelling for users to engage with your website and makes it easy for them to conduct e-commerce, we could close that gap a lot quicker. And so I think the way to do that is to build mobile Web apps as opposed to native apps."

Buying advice?

The question is not really if HTML5 will become standard, but when. Technologies develop rapidly these days, and once the major issues are worked out, particularly the video codec, and browsers are updated, widespread adoption could happen quickly. There is optimism that HTML5 will move us closer to a unified standard and a solid cross-platform foundation for rich Internet experiences. But the reality of technology evolution, with powerful competing stakeholders vying for dominance, suggests that unified, universal standards will remain elusive.

"There's a holy grail of 'code it once, and run it on any device or any platform, any browser, any desktop, any mobile phone,'" says Chaudhary. "And my prediction is, it'll never happen. There will always be a tower of Babel, where there will be multiple technologies used by website owners depending on the industry and the skills of the development team. And as a result, building a great user experience becomes all that much harder. And it increases the need for user interface designers, architects, and testers to be the ones who really ensure that the experience works really well. And that will continue to be a challenge as it has been for the past 20 or 30 years."

Chaudhary concludes with a most diplomatic characterization of the rich Internet technology future. "I predict Adobe Flash will continue to be a strong standard," he says. "As will the open Web technologies, as will the new mobile technologies."

Back to Top