December 2002
The carrier and content industries aren't buying the Net's end-to-end argument. Here are a couple open source ideas that might finally win them over.
The Internet has been around for a long time, and the ideas behind it have been around even longer. Many of those ideas were shared with UNIX, which has co-evolved with the Net since 4.2BSD was released in 1983. Once that happened, Peter Salus says in Casting the Net (Addison-Wesley, 1995), "Solid protocols on a flexible operating system at low cost was an unbeatable combination. OSI never had a chance."
OSI was an abstract architecture for open systems that turned out to be no less monolithic, and far less appealing, than any vendor's "complete network solution". Oceans of ink were spilled in the 80s on the subject of OSI. Many vendors signed on to endorse and co-develop it (as the Corporation for Open Systems -- COS) in 1985: but ultimately the OSI model succeeded mostly by burning the concept of a "stack" into everyone's brains.
OSI failed, John Quarterman told Peter Salus, because
OSI specified before implementation. So specification took forever and implementation never happened.
What got implemented instead was UNIX and the Net. Together they're still getting implemented -- with Linux playing a leading role. But many of these early lessons still haven't sunk in. In fact, the very nature of the Net remains a mystery to many -- perhaps most -- people who aren't familiar with the seminal ideas behind it. Let's look at three expressions of those ideas.
The first is from End-to-End Arguments in System Design, the 1984 essay by Jerry Saltzer, Dave Clark and David P. Reed. This was the argument that framed the Internet's architecture:
The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.)
The second is something David Isenberg wrote in The Rise of the Stupid Network, the internal document that cost David his job at AT&T in 1997:
The Internet breaks the telephone company model by passing control to the end user. It does this by taking the underlying network details out of the picture.
The third is from the preamble to The Cluetrain Manifesto, which David Weinberger, Chris Locke, Rick Levine and I wrote in early 1999:
Through the Internet, people are discovering and inventing new ways to share relevant knowledge with blinding speed. As a direct result, markets are getting smarter -- and getting smarter faster than most companies.
After Cluetrain turned into a book and hit the streets in early 2000, we heard back from a surprising bunch of companies that seemed to be getting real smart, real fast: GE, Procter & Gamble, Ericsson, Wal-Mart, Prudential and Johnson & Johnson, to name a few. Unfortunately for the Net, they were the wrong companies.
The ones that most desperately needed a few clues were the big carriers whose job was to keep things stupid between an exploding number of smart ends. But Verizon, AT&T, Qwest, Time Warner Cable, Comcast, Cox and SBC are all still possessed by their old telephone and cable company mentalities. They still dream about metered traffic and subscription valves on content spigots.
This founding dream so upset Gordon Bell that he circulated an email in January, 1995 with the title "Building Cyberspace with One-way Streets -- bad idea? conspiracy? short-sightedness? incompetence?" In it he pleaded with his readers to put their "bodies in front of the backhoes that are installing asymmetric networks that simply mimic cable TV."
The big "content" companies from the entertainment industry shared those dreams. Unfortunately, they did get the Net's founding clues -- and hated them. The last thing they wanted to see was billions of dumb consumers turned into smart customers -- or worse, smart suppliers. Or, worse than that, both. Which is exactly what Napster showed them. That made Napster very unlucky.
But in the end-to-end ecology of the Net, Napster was just one big mutant animal. It wasn't a species, and it wasn't a habitat. That's why peer-to-peer (P2P) MP3 sharing didn't go away after the RIAA put Napster through the chipper. It couldn't go away, because the RIAA's problem was the Net itself. It defined a vastly social space where everybody could share anything with anybody. Like the assistant's broom in The Sorcerer's Apprentice, Napster's chopped up remains came to life as an army of clones, plus a brand new Internet protocol: Gnutella.
As of September, Slyck.com tallied 2,525,791 FastTrack (a proprietary Gnutella-like service) users online. Slyck say that's 25% more than Napster had at its peak. iMesh has 836,385. Gnutella has 130,556. Slyck also lists six Napster clones, plus 191 MP3 file servers on five networks that range from 3.4 million to 8.3 million songs per server.
The RIAA has taken some of these guys to court, of course, but even if the RIAA succeeds, they can't stop resourceful companies from taking advantage of the Net's end-to-end magic. They can lobby to outlaw certain kinds of businesses, or to regulate them out of existence, as they did with Internet radio in the U.S. But the Net is a worldwide phenomenon. The U.S. feds can't throttle the whole thing. When I asked Dean Landsman, a veteran broadcaster and programming consultant, for his view on the matter, he said this:
Smart netcasters are light years ahead of their nemeses. There may be a die-off today, but you can bet your next paycheck that lots of netcasters are already setting up operations offshore. And they're doing it cheap, so you can bet Linux is involved...
True, netcasting isn't much of a business yet, but in time it will be. Serious netcasters are going to want to make money, and they'll be very inventive and resourceful about it. They'll also be much more friendly to the sources of their goods than the RIAA or the Big Five record companies ever were. The opportunities for market symbiosis are too huge to ignore, especially given the old industry's legendary shaft-the-artist legacy.
All victories won by the RIAA and the MPAA on behalf of the artists they claim to represent will be hollow... The victories won by independent netcasters on behalf of those same artists will be meaningful and lucrative for many in the long term. Those victories won't happen tomorrow, but they will happen.
I agree. In the long run (and it might be very long), the Net will win. The question for the carrier and content companies will then be no different than it was in the first place: How can I win here?
David Isenberg sets up the answer with a few clues from his original Stupid Network paper:
The Stupid Network would let you send mixed data types at will - limited only by the knowledge and imagination of the application programmer community. One way voice messages, multi-way voice conferences, two-way video, email, documents, audio and/or video entertainment, whatever, could be mixed and interspersed at will, within and between sessions. You would not have to ask your Stupid Network provider for any special network modifications - its only function would be to, "Deliver the Bits, Stupid."
One thing about the Stupid Network is clear - the physical elements that comprise the network would be neither expensive nor scarce. There would be little profit margin in shipping dumb bits. There would be lots of high value Business Ideas supported by the Stupid Network, above and beyond transport.
The shift from scarcity to plenty is often the harbinger of new value propositions... The changes that now portend the Stupid Network are likely to shift the telecommunications value proposition from "network services" to something else. If I knew what it was, I would not be wasting my time writing these words.
The "network services" David talks about are ones owned by the carriers, integrated into the network in a highly proprietary way, and sold as value-adds to customers. They are not part of the Internet's infrastructure, which would not welcome them
The Internet, on the other hand, is comprised of many "stupid" protocols that each provide straightforward services. Mail routing. DNS lookup. Hypertext. File transport. Among those, thanks to Gnutella, we now have peer-to-peer file sharing as well. In fact, Gnutella is so peer-to-peer that crossloading seems a better term than of "uploading" and "downloading". Crossloading is a transport concept plainly implied by TCP/IP but not implemented until Napster raised the possibility. In other words, Napster, in spite of its centralized design, did a fine job of making the end-to-end argument.
This argument is so deeply anathema to both the carrier and the content industries that it's success at taking advantage of both is an irony of the first order. So is the fact that the Net withstands continued dysunderstanding by the Hollywood types who desperately want to regulate it away. Witness the paranoia in a list of panel topics from the most recent Digital Hollywood conference:
Feature Distribution and the Threat of Internet Piracy - Evaluating the Short & Long Term Implications
The Musical Dilemma - Has the Industry Been Permanently Napsterized or Is this the Road to Recovery?
Hollywood vs. Silicon Valley - Will the Federal Government Decide?
Digital Rights Management - Business Models, Secure Streaming, Secure Subscriptions & Download, Secure Pay Per View Technology Solutions
Thanks to years of constant lobbying, Congress now shares Hollywood's understanding of the Net, and the same paranoia. Tom Poe, who runs a free recording studio in Reno, recently sent a suggestion to Senator Harry Reid of Nevada. Here is the opening to Senator Reid's reply:
Dear Mr. Poe,
Thank you for contacting me to express your concerns about the intellectual property rights and the public domain. I appreciate hearing from you.
I understand your concerns about ensuring that. This issue is very controversial because Congress must protect intellectual property rights while still allowing ordinary Internet users to have access to public domain content. I appreciate hearing your suggestion for a "tag" system. I am carefully reviewing a number of proposals to address this issue, and as I do so, I will keep your views in mind.
Note the language. Senator Reid sees the Net as a space comprised primarily of "Intellectual property" that must be "protected". Stuff made unimportant by lack of ownership is "public domain content" to which "ordinary Internet users" should be "allowed" to have "access". To gauge the ubiquity of this default understanding in Congress, consider this: the DMCA was passed by a unanimous vote.
It's going to take a lot of counter-lobbying to turn that ship around. Four years ago, the authors of the original end-to-end paper (Clark, Saltzer and Reed) were already defending their argument from those who failed to understand it. In that follow-up paper, Active Networking and End-To-End Arguments, they said this:
This idea of "stupid operating systems", "stupid networks", and "stupid processors" hasn't yet had its full run. The telephone company still seems to think that all users want the illusion of a copper pair from the user's house to some ISP point of presence in another city. In its Internet access approach, the cable company places the real network one step closer, but neither architecture is really prepared for the household with three computers and two network access providers. Politicians want both the cable company and the ISP to filter packets for things children shouldn't see, and the FBI asks them to make copies of specific data streams to simplify wiretapping (these may be examples of non-cooperating endpoints). Political arguments aside, even if one accepts these requirements, the corresponding implementation proposals are sometimes stunning in the way they fail to consider end-to-end arguments.
In his keynote speech at the O'Reilly Open Source Convention last summer, Lawrence Lesssig made a pessimistic case against trying to reason with people like Senator Reid, by quoting departing Congressman J.C. Watts, who said "If you are explaining, you are losing."
Arguing end-to-end clearly works with technologists, but not with most businessfolk, and certainly not with Hollywood or Congress. So that's not what's going to do the job here.
What we need is implementation. Take away Hollywood, the carriers and the feds and you're left with the two populations:
And, of course, combinations of the two.
We need both these groups, working together, because --
These two last points are chicken & egg problems for anybody who wants to add a new service to the Internet.
Take instant messaging (IM), for example. The Net's founding hackers didn't come up with an IM protocol, but Jeremie Miller and the guys at Jabber.org did, starting in '99. Jabber is very widely used now. IBM and Earthlink both use it extensively on an internal basis. So do uncounted other organizations. Meanwhile, Microsoft, AOL, Yahoo and Apple all continue to lock their users into closed IM systems that don't interoperate with each other. But there's nothing stupid about any of them. Until one or more of them gets stupid, we won't see ubiquitous IM on the Net.
Identity (ID) services is another. Back in June I wrote about what PingID was up to in this new "space". The company (PingID.com) was founded by Andre Durand, who also co-founded Jabber.com. Like Jabber, PingID has a .org counterpart -- PingID.org -- that handles open source development. (Disclaimer: I am on the advisory boards of both Jabber.com and PingID.com.) At the helm of that effort is Bryan Field-Elliott. I recently asked Bryan how it's going. Here's some of what he told me:
We're still just getting started, and it isn't easy.
Looking back over the history of protocols and their implementations, I see two different routes we can take. One is the Gnutella route, where we trailblaze a new protocol. The other is the Apache route, where we do practical implementations of protocols that have already been developed.
We're trying to take both routes and see what happens.
So we're working on identity protocols, and on implementations for them. But so are Sun with Liberty Alliance and Microsoft with Palladium. Those are the 800 pound gorillas that are making the biggest noise in this space.
Can we realistically expect any traction in the real world if we trailblaze a new protocol? Or should we follow up on protocols developed by, say, the Liberty Alliance? Right now those protocols are not especially friendly to the individual, but someday they might be. There's a better chance of that happening if we're in the conversation.
I single out IM and ID because I think both can help the content & carrier folks start to make sense of the Net's end-to-end nature. Together those services will make it possible for sellers to have data-rich conversations with highly empowered customers. IM and ID together will support a number and variety of business relationships that verges on the infinite. Those new customers will deliver plenty of clues.
Nothing characterizes the mutant industrial value system we call "consumerism" more than the extreme asymmetry of power between producers and consumers. That value system -- which ought to be called "producerism" -- has been around for so long that we can hardly imagine one comprised of equally powerful vendors and customers.
But that value system is exactly what the end-to-end nature of the Internet was built to support. And there's a name for it, too.
It's called a marketplace.
Doc Searls is Senior Editor of Linux Journal.