The January 17, 2005 issue of The New Yorker has a long report by Dan Baum titled Battle Lessons: What the Generals Don't Know. It tells one story of how U.S. armed forces learned to make war and peace — at the same time, on the ground — in Iraq. First, it's the story of a generation:
Gen Xers are notoriously unimpressed by rank, as Donald Rumsfeld discovered in December, when enlisted soldiers questioned him sharply about the lack of armor on their vehicles. This turns out to be a positive development for the Army, because the exigencies of the Iraq war are forcing the decision-making downward; tank captains tell of being handed authority, mid-battle, for tasks that used to be reserved for colonels, such as directing helicopter close-air support.
The younger officers have another advantage over their superiors: they grew up with the Internet, and have created for themselves, in their spare time, a means of sharing with one another, online, information that the Army does not control. The "slackers" in the junior-officer corps are turning out to be just what the Army needs in the chaos of Iraq. Instead of looking up to the Army for instructions, they are teaching themselves how to fight the war. The Army, to its credit, stays out of their way.
Second, it's the story of resourcefulness, and of do-it-yourself problem-solving. The above paragraph is part of a much longer montage of snippets from two websites created by Majors Nate Allen and Tony Burgess, a pair of West Point classmates who saw a need and filled it. Baum explains:
In March of 2000, with the help of a Web-savvy West Point classmate and their own savings, they put up a site on the civilian Internet called Companycommand.com . It didn’t occur to them to ask the Army for permission or support. Companycommand was an affront to protocol. The Army way was to monitor and vet every posting to prevent secrets from being revealed, but Allen and Burgess figured that captains were smart enough to police themselves and not compromise security. Soon after the site went up, a lieutenant colonel phoned one of the Web site’s operators and advised them to get a lawyer, because he didn’t want to see "good officers crash and burn." A year later, Allen and Burgess started a second Web site, for lieutenants, Platoonleader.org.
The sites, which are accessible to captains and lieutenants with a password, are windows onto the job of commanding soldiers and onto the unfathomable complexities of fighting urban guerrillas.
Here's part of a montage of snippets gleaned by Baum from the two sites:
If they accept you into the tent, by custom they are accepting responsibility for your safety and by keeping on the body armor, you are sending a signal that you do not trust them. If tea or coffee are offered, be sure to accept the items with the right hand. Do not look at your watch when in the tent. Have the unit invest in Wiley X’s—these sunglasses also serve as sun-wind-dust goggles. Supply each soldier with one tourniquet; we use a mini-ratchet strap that is one inch wide and long enough to wrap around the thigh of a soldier. Cotton holds water. Even with the best socks, and plenty of foot powder, your feet are likely to start peeling like you’ve never experienced. You’re more likely to be injured by not wearing a seatbelt than from enemy activity.
Third, it's the story of a surprisingly adaptive institution: the U.S. Army itself. Rather than fight these affronts to protocol, the Army brought the two sites in-house (both now have .mil addresses). They also all four of the sites' founders to grad schools to earn Ph.D.s, after which they'll come back to West Point and run the sites while working as professors there.
Naturally, other military sites, with similar purposes and methods, have been spawned. Together these efforts are reforming modern war- and peace-making — not from the top down, but from the inside out and from the outside in. One of the most tradition-bound organizations in the world, one that has always lived by top-down command-and-control, finds itself adapting to the same networked environment that is forcing private and public organizations of all kinds, all over the world, to adapt or suffer the consequences.
So, what does all this have to do with the title of the book you're holding? The answer comes in the middle of the story, between the time when the two sites were built and when the Army took them in-house:
"Hey guys," one captain wrote. "Remember this is an open-source Web site. Everything you type is being read by the enemy."
It's a fair guess to say this captain was not familiar with the OSI's Open Source Definition. He probably also didn't know what "open source" meant in the first place for software developers. The open source movement we know today was launched roughly, if not exactly, when Eric S. Raymond announced it in February 1998. (And that's not to subtract credit from the free software movement that Richard M. Stallman started fifteen years earlier — without which we wouldn't have had an open source movement.)
More likely this captain was using "open-source" in a sense that predates the open source movement. This likelihood was brought to my attention by Jeremy Impson, who says the captain
...was using the *original* definition of "open source", which predates Eric Raymond's usage by, well, who knows by how long. Possibly decades. In the intelligence community, "open source" refers to any public source of information. Newspapers are the canonical example, but it's clear that any web site on the Internet is an open source. This is the captain's point--everything they are discussing on the Internet-accessible web site is open to the public.
Jeremy was among a number of readers who wrote to tell me I neglected to credit that captain with knowing the older definition of "open source" when I wrote about the New Yorker piece in one of my SuitWatch newsletters (published bi-weekly by Linux Journal) in February 2005. Corrections of this sort are examples of "open source journalism" at work.
In addition to "journalism", "software" and "development" Wikipedia has listings for open source funding, games, politics, ethics and culture — though not yet for open source journalism. Of course, as an "open source" journalistic project, Wikipedia is subject to change.
As of today (March, 2005, as I write this), Google finds 22,000,000 results for "open source". That's more than for "word processing" (3,830,000), "spreadsheet" (4,940,000) and "user friendly" (6,800,000) combined. No doubt the majority of those pertain to open source software and develpment. But no doubt many other meanngs are also included. Just as "ground zero" is no longer confined to exploding bombs, "open source" is no longer confined to software.
Yet the values and practices of open source development are at the heart of a profound change in civilization — a change made possible by the Internet and by countless infrastructural building materials that build on the Net, and thrive in the world's internetworked environment. The Net and all those building materisls — the stuff we call free and open source software — share three of the Net's core virtues, abbreviated as NEA:
Exceptions can be found to each of these, of course. For example, MySQL is owned. Yet, as infrastructural building material, it almost doesn't matter whether MySQL is owned or not, because everybody can use it and anybody can improve it. MySQL's ownership applies matters to those that choose to have a commercial relationship with the company. In that respect, MySQL doesn't behave as a traditional "platform" vendor, trying to lock customers and third parties into a dependent role in a private environment. Instead, it behaves more like a provider of building material for a construction project. MySQL, like other open source components, is modular stuff. It's made to work inside a larger context — a job, a design, an architecture, whatever — where each component does its job and not much more than that.
The list of open source building materials grows constantly. When I started writing for Linux Journal, there were a handful of familiar names: Linux, Apache, Sendmail, Perl. That grew to become the LAMP suite (where the M represents MySQL and the P includes PHP and Python). Now the names dropped in a discussion of open source might also include Tomcat, Squid, Asterisk, Jboss, Eclipse, Squid, Jabber, ZeroConf, RSS, iPodder or any of the 80,000+ projects on Sourceforge alone.
Open source building materials are all commodities. In the old software business, commodities and commoditization were considered Bad Things. What you wanted was 80% profit margins and a role as a "platform" for dependent customers and third parties, both of which you controlled. That model still works for Microsoft, Oracle, SAP and a few other companies. But, except for Microsoft, nearly every large software vendor is busy adapting to a world filling up with commodity components.
In mature industries, commodities are Good Things. Thanks to open source, the computer and software industries are maturing, and coming to accept the goodness commodities bring.
For clues to what the software industry is becoming, now that it's growing up, just look to the one from which it borrows much of its vocabulary: construction. As with construction, we have "architects", "designers" and "builders". We build "structures" and "platforms" with "tools" and "components". On the Web we extend construction to real estate, with "sites" at "locations" and addresses that are "under construction". In fact, it's hard to talk about software development without borrowing the language of construction.
The International Federation of Building and Wood Workers (IFBWW) says construction is the largest economic sector in the world, with a market volume of 3.5 trillion dollars (U.S.) and a work force of 50 million people. It is filled with companies and customers of every size. It loves commodites — the more, the better. Open source is standard. If somebody figures out a better way to put up sheet rock or to lay a floor, they share it with the rest of the crew, and the knowledge spreads. The industry is also thick with all kinds of intellectual property. Manufacturers brag about their patented floor joists, insulated glass and door latches. Yet those intellectual property claims are rarely an issue, because they don't usually serve as a "platform" in the sense we have in the software business. The modular nature of building construction isolates IP claims to the component itself. Nobody builds a house on the "Georgia-Pacific platform". That would be silly, even though we've been accustomed to the equivalent in the computer industry since IBM cornered the mainframe business and became known for decades as "the environment".
The construction business has no Microsoft. Yet it has lots of companies of Microsoft's size.
Already Microsoft's monopoly looks anachronistic, if not doomed. I know two multi-billion dollar companies that are moving employees from Microsoft Office to OpenOffice on Windows, with plans to swap Linux for Windows later on the same machines. There is still huge lock-in around Microsoft Exchange, Outlook, Access and other products. But
Yet we are clearly living in a new world, described by Chris Locke, Rick Levine, David Weinberger and myself in the first words of The Cluetrain Manifesto:
A powerful global conversation has begun. Through the Internet, people are discovering and inventing new ways to share relevant knowledge with blinding speed. As a direct result, markets are getting smarter—and getting smarter faster than most companies.
That was in early 1999.
In 1995, John Perry Barlow perfectly described this millieu in his classic essay "Death From Above":
Over the last 30 years, the American CEO Corps has included an astonishingly large percentage of men who piloted bombers during World War II. For some reason not so difficult to guess, dropping explosives on people from commanding heights served as a great place to develop a world view compatible with the management of a large post-war corporation.
It was an experience particularly suited to the style of broadcast media. Aerial bombardment is clearly a one-to-many, half-duplex medium, offering the bomber a commanding position over his "market" and terrific economies of scale.
Now, most of these jut-jawed former flyboys are out to pasture on various golf courses, but just as they left their legacy in the still thriving Cold War machinery of the National Security State, so their cultural perspective remains deeply, perhaps permanently, embedded in the corporate institutions they led for so long, whether in media or manufacturing. America remains a place where companies produce and consumers consume in an economic relationship which is still as asymmetrical as that of bomber to bombee.
After deriding the shortsightedness of the telcos and cable companies that were then busy installing asymmetrical (fat down, thin up) service to homes, Barlow summed up the optimistic long-term case:
On one side you've got the monotheism of Control, the one-to-many system which has dominated the West at least since the Industrial Revolution, possibly since Gutenberg; possibly since Moses. And done a damned fine job of creating civilization, I might add. A necessary thing in its day.
Surging toward these battlements of God Above All are the galloping, barbarous hoards of pantheism, guerrillas all, from the Cypherpunks to Newt Gingrich. I sometimes wonder which of these I really want to win, but I'm pretty sure which one is going to. It's B-52's vs. punji sticks. It's machine against nature. Sooner or later, nature takes the game.
No matter how much death they rained from above, the bombers lost Viet Nam. They're going to lose Cyberspace too. For exactly the same reasons.
It was almost ten years later that Phil Moore, then an Executive Director in the UNIX Engineering team at Morgan Stanley, walked to the front of the room during the Q&A that following a panel on "Commercial OSS Business" at OSCon (the O'Reilly Open Source Convention), and delivered a soliloquy on What's Going On. Here's part of what he said:
I work for the 38th largest company in the world, Morgan Stanley. We have a billion dollar IT budget. And we use a little of everything. Unfortunately. Excuse me, a LOT of everything. The trend I've seen in the last ten years...is the exponential growth in the variety and the depth and breadth of installation of open-source software in our infrastructure....What I'm seeing is that in the infrastructure, the core infrastructure, open source is going to take over, leaps and bounds....I'm predicting, right now, that by 2006 or 2007, we're going to be a 90% Linux shop.
He added, "I'm not supposed to talk about that, for fear of being sued by SCO." Then he turned to one of the panelists, Matt Asay of Novell, and said, "which is the reason why I couldn't go to your conference, the OSBC. I wasn't allowed to go." In fact Phil had been scheduled to serve on a panel I organized at OSBC. Another panelist (making two out of the original four) was also told by higher-ups not to speak.
That open source plays a huge role at large IT shops is one of the worst-kept secrets in the world, yet one of the hardest to write about.
Personal computing was a hobby at best and an oxymoron at worst. Minicomputers were at the leading edge of the business. They occupied ventilated rooms with raised floors, and were administrated by a priesthood of professionals. In those days computing consisted of "data processing". Our workstations were called "terminals", which were sold in "smart" and "dumb" breeds. They were as personal as elevators.
Vendors dominated and defined enterprise computing. An MIS department was an IBM, Digital Equipment Corporation, Control Data, Honeywell, Bull or NCR "shop." It used not only central-processing hardware from those vendors, but also disk drives, terminals, networking hardware, cabling and nearly everything else. Even the third-party "compatible" peripherals in hardware and software were generally designed to work only with one vendor's breed of computer system – its “platform.”
In addition, just as each vendor's system was siloed, the systems themselves were isolated from each other – even systems of the same kind. For example, Digital and Wang networks eventually "standardized" on Ethernet, but with incompatible systems called DECnet and WangNet. Meanwhile, BM pushed Token Ring, an incompatible standard that also required IBM's then-new Structured Wiring System, which differed form the company's own existing (and widely differing) networking and cabling systems. Interoperability was a rare achievement, and mostly a distant dream. [I added Token Ring because I'll bring it up later.]
Looking back from today, it’s easy to see that customers have always wanted compatibility and interoperability between platforms. They wanted choices at every level, including the platforms themselves. They wanted freedom from dependencies up and down each vendor's "stack" that limited choices to the vendor's own offerings. In those old days, customers didn't get any of that. These days, They're getting some of it — but far from all that they want. [I don't want to say "starting" when later we say we're "half way." But I also think we might want to say something here about Microsoft's persistent desktop monopoly proving how far we still have to go… and that's a long way.]
So the history of IT has been a slow war of liberation — a struggle toward independence for users as well as for professionals.
Behind each of our buzz-concepts, from object-oriented to client-server to open source to Web services and service-oriented architectures, has been an end-game of independence — one in which modular, transparent and easily manipulated components let people build the systems they want.
In a continually self-reinforcing virtuous cycle, the Internet has played a part. It gave people and systems the ability to communicate, and thereby gave them the impetus to communicate better – with systems that were intelligible to and manipulable by other people and other systems. Rather than design applications, the professionals in the priesthood will eventually build modules and tools that users can assemble and manipulate…not to build applications, but to accomplish tasks and processes. [This paragraph still seems vague to me. How do you "accomplish" a process? What kind of process are we talking about? Since it's your paragraph, Esther, maybe you can unpack that a bit.]
In the construction industry, this kind of independent and self-reliant activity is called DIY, for do-it-yourself. For the computer industry, we'll call it DIY-IT. For now, DIY-IT is about the empowerment of professionals, not of end-users...though at the application level users are demanding and getting applications that are far more customizable -- and forgiving -- than they ever were before.
And, I submit, both free software and open source are parts of much larger software market trends. While both provide the necessary base conditions for those trends, they are insufficient to explaining them. Those trends are: