December 2002
The carrier and content industries aren't buying the Net's end-to-end argument. Here are a pair of open source ideas that might finally win them over.
Here are two questions to mull for the couple thousand words or so:
1) Why are whole industries, including the biggest carriers and content producers, still clueless about the Net (and Linux along with it)?
2) What open source projects will help those industries not only understand the Net and Linux, but start innovating and making money with them?
The Internet has been around for a long time. The ideas behind it have been around even longer. Many of those ideas were shared with UNIX, which has co-evolved with the Net since 4.2BSD was released in 1983. Once that happened, Peter Salus says in Casting the Net (Addison-Wesley, 1995), "Solid protocols on a flexible operating system at low cost was an unbeatable combination. OSI never had a chance."
OSI was an abstract architecture for open systems that turned out to be no less monolithic, and far less appealing, than any vendor's "complete network solution". Oceans of ink were spilled in the 80s on the subject of OSI. Many vendors signed on to endorse and co-develop it (as the Corporation for Open Systems -- COS) in 1985: but ultimately the OSI model's only real success wa getting everybody to separate topics into "stacks" (with hardware on the bottom and applications on top).
OSI failed, John Quarterman told Peter Salus, because
OSI specified before implementation. So specification took forever and implementation never happened.
What got implemented instead was UNIX and the Net. Together they're still getting implemented -- with Linux playing a leading role. But many of the early lessons still haven't sunk in. In fact, the very nature of the Net remains a mystery to many -- perhaps most -- people who aren't familiar with the seminal ideas behind it. Let's look at three expressions of those ideas.
The first is from End-to-End Arguments in System Design, the 1984 essay by Jerry Saltzer, Dave Clark and David P. Reed. This was the argument that framed the Internet's architecture, and blew away the whole centralized concept of the "stack", where ownership of lower levels gave control to everything above:
The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.)
The second is something David Isenberg wrote in The Rise of the Stupid Network, a widely circulated internal document that cost David his job at AT&T in 1997:
The Internet breaks the telephone company model by passing control to the end user. It does this by taking the underlying network details out of the picture.
The third is from the preamble to The Cluetrain Manifesto, which David Weinberger, Chris Locke, Rick Levine and I wrote in early 1999:
Through the Internet, people are discovering and inventing new ways to share relevant knowledge with blinding speed. As a direct result, markets are getting smarter -- and getting smarter faster than most companies.
Here's a way of tying all three together: Markets are made where the Net's ends meet. Supply and demand, production and consumption, seller and buyer, vendor and customer -- all those polar forces whose dialog we call economics -- are put right next to each other by the Net's empty (stupid) middle.
The Net's end-to-endedness supports countless new efficiencies that go without notice because they happen in the background. GE Global eXchange Services (GXS). for example, processes over billion transactions worth over a trillion dollars a year -- all over the Net.
A couple years ago, GXS president and CEO Harvey Seegers told me Cluetrain's key insight was that the net evened the balance of power between supply and demand. In the past GE was in command of its relationships with its customers, he said; but that ended with the Net. Now customers have real power. Even giant companies like GE are coping with a new reality where they can't muscle their customers any more -- and liking it, because now they have much healthier relationships with those customers.
After Cluetrain came out we heard the same kind of thing from lots of other big companies, including Wal-Mart, Johnson & Johnson, Prudential and Ericsson.
But a conspicuous silence came from the companies that actually carry the Internet to homes and businesses. Verizon, AT&T, Qwest, Time Warner Cable, Comcast, Cox and SBC all try to talk Internet jive, but they all still seem to be dreaming about metered traffic and subscription valves on content spigots.
This dream has been a nightmare to Internet veterans ever since carriers first began to build broadband services around the same supply/demand power asymmetries established by consumer marketing. To them the Net was all big-to-small and few-to-many, just like retailing, and just like television.
It so upset Gordon Bell that he circulated an email in January, 1995 with the title "Building Cyberspace with One-way Streets -- bad idea? conspiracy? shortsightedness? incompetence?" In it he pleaded with his correspondents to put their "bodies in front of the backhoes that are installing asymmetric networks that simply mimic cable TV."
The big "content" companies from the entertainment industry shared those fantasies. Unfortunately, they did get the Net's founding clues -- and hated them. The last thing they wanted to see was billions of dumb consumers turned into smart customers -- or worse, smart suppliers. Or, worse than that, both. Which is exactly what Napster showed them. That made Napster very unlucky.
But Napster was just one mutant animal. It wasn't a species, and it wasn't a habitat. That's why peer-to-peer (P2P) MP3 file sharing didn't go away after the RIAA put Napster through the chipper. It couldn't go away, because the RIAA's problem wasn't Napster, but its habitat: the Net itself.
In the end-to-end ecology of the Net, everybody can share anything with anybody, including ideas. Or, in business parlance, innovations. Good ideas beget better ideas. One innovation suggests another and another.
That's why Napster's chopped up remains came to life like the army of brooms in The Sorcerer's Apprentice. These include not only countless Napster-inspired alternatives, but a whole new end-to-end file sharing protocol: Gnutella.
As of September, Slyck.com tallied 2,525,791 FastTrack (a proprietary Gnutella-like service) users online. Slyck says that's 25% more than Napster had at its peak. iMesh has 836,385. Gnutella has 130,556. Slyck also lists six Napster clones, plus 191 MP3 file servers on five networks that range from 3.4 million to 8.3 million songs per server.
The RIAA has taken some of these guys to court, of course, and they've talked Congress and the Copyright office into taxing the U.S.-based Internet radio business out of existence. But they can't stop resourceful companies from taking advantage of the Net's end-to-end magic, which ignores national boundaries. Here's Dean Landsman, a veteran broadcaster and programming consultant:
Smart netcasters are light years ahead of their nemeses. There may be a die-off today, but you can bet your next paycheck that lots of netcasters are already setting up operations offshore. And they're doing it cheap, so you can bet Linux is involved.
In the long run (and it might be very long), the Net will win. When that happens (hopefully sooner than later) the question for the carrier and content companies will be no different than it was in 1995: How can I win here?
David Isenberg sets up the answer with a few clues from his original Stupid Network paper:
The shift from scarcity to plenty is often the harbinger of new value propositions... The changes that now portend the Stupid Network are likely to shift the telecommunications value proposition from "network services" to something else. If I knew what it was, I would not be wasting my time writing these words.
The "network services" David talks about are "stacks" owned by the carriers. They are not part of the Internet's infrastructure.
The Internet, on the other hand, is comprised of many "stupid" protocols that each provide infrastructural network services. Mail routing. DNS lookup. Hypertext. File transport. Among those, thanks to Gnutella, we now have peer-to-peer file sharing as well. In fact, Gnutella is so peer-to-peer that crossloading seems a better term than of "uploading" and "downloading".
"Crossloading" is a transport concept that was plainly implied by TCP/IP but not implemented until Napster prototyped it. In other words, Napster, in spite of its centralized and proprietary design, did a fine job of making the end-to-end argument anyway.
This argument is still so deeply anathema to both the carrier and the content industries that its success at taking advantage of both is an irony of the first order. So is the fact that the Net withstands continued dysunderstanding by the Hollywood types who desperately want to regulate it away. And so is the fact that Congress almost unanimously lets Hollywood do the thinking for both of them.
Thanks to years of constant lobbying (going back to '95 and earlier), Congress today shares Hollywood's understanding of the Net, and its paranoia as well. Tom Poe, who runs a free recording studio in Reno, recently sent a helpful suggestion to Senator Harry Reid of Nevada. Here is how Senator Reid replied:
Dear Mr. Poe,
Thank you for contacting me to express your concerns about the intellectual property rights and the public domain. I appreciate hearing from you.
I understand your concerns about ensuring that. This issue is very controversial because Congress must protect intellectual property rights while still allowing ordinary Internet users to have access to public domain content. I appreciate hearing your suggestion for a "tag" system. I am carefully reviewing a number of proposals to address this issue, and as I do so, I will keep your views in mind.
Note the language. Senator Reid sees the Net as "Intellectual property" real estate that must be "protected". Stuff made unimportant by lack of ownership is "public domain content" to which "ordinary Internet users" should be "allowed" to have "access". To gauge the ubiquity of this default understanding in Congress, consider this: the DMCA was passed by a unanimous vote.
It's going to take a lot of counter-lobbying to even begin changing all those minds. The short-term prospect isn't good. In his keynote speech at the O'Reilly Open Source Convention last summer, Lawrence Lesssig offered little hope for reasoning with people like Senator Reid. Lessig told the story of Congressman J.C. Watts, who recently resigned his seat because "If you are explaining, you are losing".
So arguing isn't going to do the job with Congress, and it probably won't do the job with business, either. Hey, if they haven't grokked end-to-end by now, how many decades is it going to take?
What we need is implementation. The questions are who, what and how? The how part is easy: The Net is free and open, so the development process needs to involve free software and open source methods.
For who, I see just two populations:
For what I have two suggestions. One is instant messaging (IM). The other is identity (ID). Neither is in the suite of protocols that define the Net's infrastructure, and all kinds of business opportunities open up once those services are ready to support new kinds of functionalities and new business innovations based on them.
First, IM.
The Net's founding hackers didn't come up with an IM protocol, but Jeremie Miller and the guys at Jabber.org did, starting in '99. Jabber is widely used now. IBM and Earthlink both reportedly use it extensively on an internal basis. So do uncounted other organizations. Meanwhile, the familiar IM systems from Microsoft, AOL and Yahoo all continue to lock their users into closed systems that don't interoperate with each other. Until one or more of them gets into alignment with the Net's stupidity, we won't see ubiquitous IM.
Second, ID.
To guage the importance of ID, consider the matter of your own identities: those representations of yourself in the business world. How many of them are granted to you by outside organizations? How many are mostly beyond your control? Is it not close to 100% in both cases?
What new business possibilities would open if your suite of identity representations were fundamentally your own and under your own control? What if you, as a customer in the business world, were fundamentally autonomous in respect to every business, including all those businesses that relate to you as a name with a number that they assign?
What if your identity operated in the networked world according to protocols that grant your "end" just as much power as the ones that make and distribute the products you buy, and the ones that intermediate your transactions?
What new businesses would be possible if your identity suite had its own open APIs?
What new innovations would be called forth if business conversation about your personal information -- your location, your destinations, your interests, your preferences for anything and everything -- was primarily about what you as a customer want to do with that information, and about the voluntary relationships you want to have with businesses that provide desired products and services to you?
What kinds of existing businesses will suddenly find all kinds of productive new ways to relate to customers once the Net supports a public digital identiy infrastructure that gives all ends on the Net equal powers as peers?
What kinds of businesses would go away once markets turn into countless sets of real conversations between fully empowered customers and the companies that relate to them on a fully permitted basis?
It's easy to imagine countless new businesses -- and a lot less unwanted advertising, PR, promotion, junk mail and other annoying guesswork about what you, "the consumer", might want.
Making this happen is what PingID.com has been up to in the year since it was founded by Andre Durand, who also co-founded Jabber.com. Like Jabber, PingID has a .org counterpart -- PingID.org -- that handles open source development. (Disclaimer: I am on the advisory boards of both Jabber.com and PingID.com.)
PingID.org is working on a peer-to-peer browser or thin (embedded) client-based public digital identity infrastructure. It makes heavy use of XML, SOAP, XML-RPC and other standards that Jabber also trafficks in, which makes for many useful synergies. For example, the Jabber protocol not only handles instant messaging, but also presence. When you show up somewhere, for example, your presence can trigger an IM message in XML that could set all kinds of useful activities in motion.
At the helm of PingID.org's efforts is Bryan Field-Elliot. I recently asked Bryan how it's going. Here's what he told me:
In some ways, it feels like we're still just getting started, and it isn't easy.
Looking back over the history of protocols and their implementations, I see two different routes we can take. One is the Gnutella route, where we trailblaze a new protocol which perfectly suits our vision of identity rollout. The other is the Apache route, where we do practical implementations of protocols that have already been developed.
We're trying to balance both routes and see what happens.
So we're working on identity protocols, and on implementations for them. But so are Sun with Liberty Alliance and Microsoft with Passport and perhaps with Palladium. Those are the 800 pound gorillas that are making the biggest noise in this space.
Say that Pingid.org trailblazes a new protocol putting ultimate privacy control in the hands of the user -- priorities which are secondary at best for Liberty and Passport. Can we realistically expect any traction in the real world given the momentum behind the other identity protocols? Or instead, should we follow up on protocols developed by, say, the Liberty Alliance? Right now those protocols are not especially friendly to the individual, but someday they might be. There's a better chance of that happening if we're in the conversation. This is the fine line we're walking: a balance between idealism and pragmatism.
Again, the choice Bryan presents is about implementation. But motivation to implement gets a lot easier once people start imagining all the cool ways this stuff can be put to use. Here are seven that have come up for Andre Durand, just through his conversations with businesses that have taken an interest in building out ID infrastructure:
"In addition to this", Andre says, "I imagine that there will be tons of 'identity infrastructure integrators', identity security monitoring companies, identity theft insurance products, etc."
There are also existing business categories that can make estensive use of a public ID infrastructure: banks, cell phone companies, credit card companies, retailers... and our original two -- carriers and content providers.
With an end-empowering ID infrastructure in place, many new carrier services to customers suddenly become imaginable, because each customer is much more than a name, a number and an address -- a passive "consumer." Their identities are not only far more rich and detailed, but their behavior is far more active and involves far more choice. Rather than threatening carriers, it gives them an API to which they can address far more granular and flexible services, including bandwidth as symmetrical or asymmetrical as the customer requires.
When preference-rich and fully-empowered customers show up in the content marketplace, eager to develop relationships with the artists who make movies and recordings, all the DRM issues that hog conversational airtime today get exposed as desperate attempts by producers and distributors to control "consumer" behavior.
A sufficiently end-empowering digital identity infrastructure will make consumers evolve into customers. When that happens, all arguments about controlling consumer behavior become moot.
And the end-to-end argument finally wins, big time.
Doc Searls is Senior Editor of Linux Journal.