The video tag mess, and why Google's interests are (mostly) our interests

Earlier this week, Google's Chrome browser project announced it was dropping support for H.264 video, and immediately there was an uproar as if Google had desecrated a sacred object and posted the video on YouTube.

Most people actually have no idea what this means. A lot of people have drawn conclusions that I think are fundamentally wrong. All of this is very important to the evolution of Web media, and I'm going to try to make some sense of it.

Let's start with the word "open."

The Web has never been the best technology for conveying and displaying information on computer screens, but it has always been the most open. That openness enabled "the rest of us" to go off and create pages, sites, applications and innovations.

Nobody had to buy a license from Tim Berners-Lee to use HTML or transmit information using HTTP. Nobody had to pay Vint Cerf and Robert Kahn to use TCP/IP. Nobody had to license Javascript from Brendan Eich or pay Håkon Wium Lie to use CSS in web pages. Their contributions were free and open, and the entire world benefited.

The Internet became popular because it's owned by no one, and open to all. All of us can be creators and publishers.

If you're the guy holding a government-enforced monopoly on an idea or a method of doing something that's important to the rest of us, then of course you're happy. You might even think you're entitled to ownership and that the rest of us should pay to enforce your property rights.

But if you're the rest of us, well, open is way better. And the Web is built not on patent law and payment of royalties, but on open, freely usable, standard ways of doing things that are part of our human commonwealth.

When the makers of various tools reach formal agreement on how they can interoperate, those agreements are called standards.

Making standards is a messy, contentious process.

Sometimes they prescribe things that haven't been implemented yet. CSS2 has a lot of that.

Sometimes they chase implementations. HTML5 tries to do that, following a pattern called "pave the cowpaths." Overall, this works pretty well, messy or not.

The trouble we're about to examine arises from a case where the cowpath is not yet defined, and a bunch of bulls are having a big argument over where it should lead. At least one of the paths is known to be dangerous. None of the bulls trust each other.

Sure, we've had video on the Web for years, so why the fuss?

None of it has been defined as a Web standard. There was no way to be sure of having the right extra tools installed on your computer -- maybe RealPlayer, maybe Quicktime, maybe some goofy format-of-the-year from Microsoft or from some company you've never heard of.

While there was no video standard, there was a standard for telling a Web browser to embed an external program in a page.

Well, that's not quite right. There actually were three competing methods for doing it, a real mess, depending on which browser and platform you were targeting, but there were ways to write HTML that would at least work everywhere -- if the external program was available. Each browser maker developed its own "plug-in" architecture that made it possible for third parties to add external programs.

Those external programs might be a video player, a "virtual reality" viewer, an interactive game, a viewer for an advertisement with moving images, a Java image editor -- any number of things, so long as the browser could figure out what to do.

The trouble was, there were no actual standards. If you were a content creator, you couldn't depend on any particular technology being available.

Eventually, Adobe "won" this battle of non-standards by making its Flash-based video player technology available, free, on Windows, Macintosh, and several versions of Linux, so it could be used by a Web browser as a plug-in.

But it wasn't a standard, isn't a standard, and -- this is important to understand -- it is not built into any Web browsers. Not IE, not Safari, not Chrome, not Opera. In every case, it's an external plug-in.

Because this non-standard was at least ubiquitous, Flash video helped a couple of former PayPal employees create a company in February 2005 that was worth $1.65 billion by the end of 2006. That company was YouTube, and it made Internet video commonplace. It changed everything.

Then Apple launched the iPad, and it chose not to support Flash. All hell broke loose.

Apple's reasons aren't relevant to this discussion. The simple fact is that since Flash wasn't specified as a Web standard, Apple could, with a straight face, claim to be supporting openness by excluding the one tool that nearly everybody was using.

Apple started talking a lot about HTML5, something most people hadn't heard of. HTML5 is good. HTML5 is open. It's full of benefits for everybody.

HTML5 also is unfinished. The committee writing the HTML5 standard has agreed that a video standard would be nice, and that a <video> tag that works pretty much like an image tag would be the right way to do it, but it has declined to specify what kind of video. And that's the problem.

At one point, a free and open technology called Theora was in the spec, but that's been removed. So now we know how to write the tag, but we don't know what to render.

For years we've had an <img src="blahblahblah"> tag in various versions of the HTML standard, and yet there is no required standard for what kind of image files are supported. So why isn't image handling a train wreck?

Well, actually it was, for awhile.

Let's go back to 1994. The Web was a new thing. Early browsers like Mosaic and Viola and Cello were letting you mix images with text. What format should be used? There was a handy image format called GIF that had been developed by CompuServe. It compressed flat colors well and made tiny files, which was important in a world of slow dialup modems, so it became popular.

Then in 1995, Unisys surfaced with a previously unknown patent on the data compression technique that was used in GIF. Big trouble ensued.

That patent led to the creation of the competing, patent-free PNG format. There was some chaos around this issue for a couple of years until the Unisys patent expired. Now both formats are widely supported, along with JPEG.

There are lots of other formats that could be used. But if you're developing Web pages, you stick to these because they're the ones people use -- standards or no. (The HTML4 spec mentions them, but only as examples.)

So let's return to the present and the future of video. We'd all like to see a simple HTML5 video tag that just works, and Web browsers that can play the video without having to fire up some external program.

So the browser makers -- Microsoft (IE), Apple (Safari), Mozilla (Firefox) and Google (Chrome) are all incorporating support for the video tag.

But which formats to support?

The world of video formats is incredibly complicated. There are competing methods of encoding/decoding, compressing, coordinating images with sound (which itself can be represented in many ways), wrapping all of it in a container and then feeding it across the network.

As you might expect when there are potential fortunes to be made, there are lawyers involved. Lots of them.

In some countries, software -- methods of doing things with a computer -- can be patented. A patent isn't a copyright, which covers a specific creative expression or design. A patent covers all possible implementations.

The United States is one of those countries. There are many who believe software patents are hindering innovation, and the reason patents exist at all is to promote innovation. But software patents are a fact of life.

At the moment, there are three proposed ways of delivering video that have gained some level of support among the people who create browsers.

One is called H.264 and has the backing of Apple and Microsoft. It's already in Safari, on the Mac and on IOS. It will be in Internet Explorer 9, which is still being developed. Until the recent announcement from Google, it was included in Chrome's feature set as well.

H.264 has a problem. It's clearly covered by a big pile of patents.

This is a big problem. At least until a court says otherwise, if you make a tool that encodes video in H.264 format, you need a license. If you make a tool that decodes video in H.264, you need a license. But it's worse than that: If you encode your own video into H.264 and distribute it, you also need a license.

The cartel that controls the H.264 patents and collects royalties can fix this, but it has not done so.

It has said it will not charge royalties to content creators (that's you and me) for "Internet Video" content "that is free to end users." Originally the term of that license expired in 2015; the group recently extended the offer.

But be very careful. If you charge for your website, or if you include your video in an iPad app that is sold, or if the vague phrase "during the entire life of this License" turns out to be something different than you thought, you could find yourself in deep trouble.

This is also a big problem for anybody making free software, such as Firefox, and is why Firefox will never support H.264.

And as for Apple -- let's just keep in mind that Safari would not exist if it weren't for Linux and the KDE project, which created the open-source HTML toolkit that eventually turned into Webkit and Safari. (And Chrome, for that matter.) Free tool licenses matter, too.

So, is there an alternative? Yes and no.

Ogg Theora is a competing format that has the support of Mozilla (Firefox), Google (Chrome) and Opera browsers. It's not supported by Safari and Internet Explorer.

Theora is widely considered technically inferior to H.264, even by some of its supporters. Google is supporting it in the browser, but won't use it for YouTube.

Like H.264, it is covered by patents. The company behind them has assigned them to a foundation and they're licensed freely, in perpetuity, for all uses, commercial and noncommercial. So it's open and unencumbered. However, the team on the other side of the fence has been spreading a story that "A patent pool is being assembled to go after Theora and other 'open source' codecs now."

The third alternative is WebM, developed by Google using patented technologies from a company it acquired. It's technically competitive with H.264 and its patents are licensed royalty-free to everyone. It's supported by Firefox 4, still in development, Opera, and many others.

There are two arguments against using WebM. I think both of them are wrong, but here they are:

WebM could violate somebody else's streaming patents and Google isn't indemnifying users. I'm not persuaded. Writing this sentence could violate somebody's patent; the USPTO hands out patents for idiotic things like swinging from side to side on a swingset, or cutting the crusts off bread. Submarine patents and patent trolls are a fact of life.

Google "controls" WebM. I don't see what "control" there is -- the specification is frozen, but the open code can be improved, subject to the creativity and persuasiveness of contributors and developers. Open tech has to start somewhere.

This isn't about trusting Google more than Microsoft and Apple. I understand a certain amount of paranoia about Google. It's spooky that they know my physical location when I'm using a wired Internet connection (but they screw up the address of the restaurant around the corner). Their data sponge soaks up everything.

And I understand completely that they're operating in their self-interest. It's not just a matter of saving money on patent licensing. It's a matter of minimizing the power held by their biggest competitors.

But in this case, Google's self-interest is also in the interest of the rest of us who create and consume content on the open Web.

I can't predict how this will end. Maybe WebM gets us there. Maybe MPEG-LA grants perpetual, genuinely free licenses for all of these usages. I really don't care.

My hope is simply to see free and open standards across the Web, including the toolkits we use to generate the Web, and consume the Web, and without regard to whether we're private citizens, small businesses or large corporations, artists or programmers or kids in high school. Open is open.