Innnards: What audio and video formats are "standard equipment" on the web?


As the author of the WWW FAQ, I regularly answer questions about the workings of the Web. If a question is frequently asked, I simply add an article to the FAQ. But sometimes a question is more detailed, more in-depth— not really a FAQ, but still of interest to others. You'll find those questions, with my answers, here in Innards along with commentary on other web-technology-related topics.

2007-12-13

Q. What audio and video formats are standard equipment on the web?

A. The most broadly supported multimedia format is Flash, which is supported by nearly all operating systems and web browsers "out of the box," reaching the largest possible audience. The Flash player supports both streaming motion video and streaming audio. You can see this in action on popular sites such as YouTube.

Even though Flash is a semi-proprietary format, its extremely wide audience reach and excellent control of presentation make it the preeminent choice for audio and video presentation on the Web. So I'll discuss Flash first and then look at other formats.

Flash Video Formats

"Under the hood," Flash has historically supported two proprietary video formats known as Sorenson Spark (a variant of H.263) and On2 TrueMotion VP6. These are still the Flash formats that reach the widest base of users. You can convert video to these formats with Adobe's Flash development tools, or with various options (including freeware) which are detailed on the FlowPlayer FLV encoding page. FlowPlayer, by the way, is an excellent video player written in Flash which you can use on your site without owning a copy of Adobe's Flash development tools yourself.

In addition, the latest updates of Flash player also support H.264 video, which is an open standard (though patent licensing fees are still involved for creators of relevant hardware and software).

Flash Audio Formats

Historically MP3 audio was the only useful compressed audio format supported by Flash. While MP3 audio works well, it potentially involves patent licensing fees— not just for software and hardware developers, but also for high-profile web sites that use MP3.

However, the latest versions of Flash also support AAC, a superior compressed audio format also found in iTunes. While AAC is also patented, license fees are paid only by software and hardware creators, not by end users and web sites.

Non-Flash Audio and Video

If you prefer not to use Flash to embed your audio and video, there are several options. Unfortunately, none will give you the control over presentation that Flash provides. This is because web browsers embed the user's audio or video player of choice, which typically will not provide much control for JavaScript-based programming and will present its own proprietary user interface.

Non-Flash Video

For video, your major non-Flash alternatives are Apple's QuickTime and Microsoft's Windows Media (WMV). If you offer a choice of these two, most users will be able to play one or the other successfully.

It also true that nearly all systems support embedded playback of MPEG Level 1 and MPEG Level 2 videos. Unfortunately, level 1 is low-resolution and low-quality, and level 2 uses very high levels of bandwidth when compared to QuickTime, WMV or the video formats supported by Flash. Providing high-quality video in MPEG Level 2 would exceed the capacity of even a good-quality cable modem or DSL connection. At any given speed, MPEG Level 2 will look much worse than Flash video, WMV or QuickTime. For these reasons it is generally not used on the Web. MPEG Level 2 video, by the way, is the format used on DVDs.

The Ogg Theora format is an alternative video format which claims to be patent-free. The creators of Ogg Theora hope to promote it as an open and free alternative to other video formats. Unfortunately, much optimization work on Theora remains to be done, and it is difficult to say with certainty that any format is patent-free unless it based on expired patents or well-known prior art. For these reasons, Ogg Theora has not made much of a splash on the Web and is not "standard equipment" on most systems. Most users would have to install extra software before displaying Ogg Theora videos successfully.

Non-Flash Audio

MP3 ("MPEG 2 Level 3") audio playback is supported on nearly all platforms. You can link directly to an MP3 file or embed it with the object or embed elements. However, since this does not give your web site much control over the playback and typically displays a proprietary player user interface, most designers wind up using Flash-based players to present MP3 in an attractive way.

As mentioned earlier, MP3 does have licensing problems. High-profile web sites deploying MP3 files will probably have to pay licensing fees for the use of MP3, above and beyond what has been paid by the hardware and software creators involved.

AAC is a strong alternative to MP3. Used in both iTunes and the latest Flash player, AAC is technically superior to MP3 and is promoted as the successor to MP3 by the creators of MP3. Most importantly, AAC does not involve license fees for content creators and distributors. License fees are paid only by software vendors and hardware vendors. Unfortunately, freestanding AAC files embedded in a web browser are not yet supported "out of the box" on most systems.

Windows Media audio is probably the third best choice on the web, but it trails distantly behind MP3 due to the fact that only Windows users are certain to be able to play it. QuickTime audio faces similar problems. And the "open" Ogg Vorbis audio format, while more mature than Ogg Theora video, is unfortunately not supported "out of the box" on Windows or the Macintosh which makes it a poor choice for use on web sites today.

Uncompressed .wav and .au files can also be used, however these use enormous amounts of bandwidth if you wish to provide high quality, so they are not practical choices for most purposes. Still, since these are patent-free formats and easy to generate on the fly, low-fidelity "telephone-quality" .wav files are sometimes useful for short sounds and audio-based CAPTCHA systems. You can see an example of this (with source code) in my article how do I add a CAPTCHA to my web site?

Still, while .wav is a valid workaround at very low quality settings, it should never be used for music-grade audio delivery on the Web.

In Conclusion

Want to reach the largest possible audience and achieve decent control over presentation? Use Flash. Right now, nothing else comes close.

For more information about how to legitimately use Flash for audio and video presentation on your site without necessarily buying Adobe's Flash tools, check out my articles how do I embed audio in my web site? and how do I embed video in my web site?

If, however, you insist on not using Flash, your best video options are WMV and Quicktime video (I strongly recommend providing both).

For non-Flash audio, MP3 is the only serious choice for reaching a large audience with music-quality sound.

If you are philosophically opposed to patented formats, consider using Ogg Vorbis audio and Ogg Theora video. But keep in mind that almost all users will have to install optional (but free) software first before they can see or hear your work. So using these formats for "casual" sound and video in a web site just doesn't make sense.