Understanding Infrastructure

We need a new conversation about infrastructure. That's what Linux, the Net, and half a million FOSS (Free and Open Source Software) applications require. Because they're all part of a new infrastructural system that cannot be explained in terms of the old one — especially since infrastructure itself is not well understood. Or rather that it is too well understood in a general way, though not in the specific.

Look up "infrastructure" on Google and you get over a hundred million results. Relatively few pertain to what infrastructure meant in the first place, which the American Heritage Dictionary places about eight decades ago:

Usage Note: The term infrastructure has been used since 1927 to refer collectively to the roads, bridges, rail lines, and similar public works that are required for an industrial economy, or a portion of it, to function. The term also has had specific application to the permanent military installations necessary for the defense of a country. Perhaps because of the word's technical sound, people now use infrastructure to refer to any substructure or underlying system. Big corporations are said to have their own financial infrastructure of smaller businesses, for example, and political organizations to have their infrastructure of groups, committees, and admirers. The latter sense may have originated during the Vietnam War in the use of the word by military intelligence officers, whose task it was to delineate the structure of the enemy's shadowy organizations. Today we may hear that conservatism has an infrastructure of think tanks and research foundations or that terrorist organizations have an infrastructure of people sympathetic to their cause. The Usage Panel finds this extended use referring to people to be problematic, however. Seventy percent of the Panelists find it unacceptable in the sentence FBI agents fanned out to monitor a small infrastructure of persons involved with established terrorist organizations.

Linux and the Internet are infrastructure, in the sense that they qualify for the American Heritage Dictionary's #1 meaning: "An underlying base or foundation especially for an organization or system". Infra is Latin for under, and you can find Linux under countless applications on devices that range from supercomputers to cell phones to wireless picture frames. Linux is also the platform of choice for many (perhaps most) sturdy and popular Web sites and services, including Google's search and Amazon's S3 and EC2 (to name too few among way too many).

And you'll find the Internet under Linux. If we're stacking up connected people and devices, nothing could be more infra, more fundamental, than the Net itself. In fact, the Net is so infra that two legacy forms of widespread commercial infrastructure — telephony and television — now run on it as well.

Yet neither is widely regarded as infrastructure.

Instead, Linux is seen as a straight-up OS alternative to other operating systems. And the Net is seen as a tertiary "service" offered by phone and cable companies, subject to the physical limitations of those companies' copper, fiber and radio plants.

The first reason is that Linux and the Net aren't physical things. Linux is software, while the Net is a set of protocols.

The second reason is that they are products of neither public nor private enterprise. Linux began with a single programmer and grew into the project of a development community comprised of thousands of individual programmers, with nearly as many different employers. (Significantly, most of those programmers will tell you they are not under anybody's command, at least as far as their kernel hacking work is concerned.) The Internet grew out of academic and defense work, while its most familiar subsystem, the World Wide Web, grew out of work at high energy physics laboratories.

The third reason is that they do not themselves generate much wealth. Instead they support an incalculable sum of it. While Linux and the Net were spread and popularized by commercial activities, they were built to support business than to make money for themselves. That is, while much money has been made with both Linux and the Net, that sum is dwarfed by the amount of money made because of both. Their main job is to support countless purposes other than their own "business models", which are minimal if they exist at all. In this role they are like geology, sunlight and atmosphere. They are free in both the free-as-in-freedom and free-as-in-beer senses of the word. It's as silly to ask Linux and the Net for their business models as it is to ask the same of H2O. You can sell bottled water, but is that business worth more than the oceans support?

The fourth reason, related to the third, is that they are seen as external to the primary economic activities of buying and selling goods. As I wrote in Greater Goods, "Abundant free software production and use might be seen as a network externality, resulting from the network effects caused by cost-free goods that are easily obtained and used—which is fine. But there is a cost to this perspective." That cost is in disregard for the foundational economic importance of these essential goods. For today's economy, the roles of Linux and the Net are better conceived as internal rather than external, much as the role of the Earth is also internal to everything that relies on it.

In economic terms, Linux and the Net are public goods, because they are both non-rivalrous and non-excludable. Of course some Internet Service Providers (such as Comcast) have claimed (without necessarily using this terminology) that the Net has at least some rivalrous qualities. When one person is "hogging bandwidth", they can prevent another person from using it. But the notion of rivalry turns on consumption, and the Net is not something one consumes. The scarcity experienced by ISP customers is a matter of provisioning, not of the Net itself. Use of the Net may be limited by bandwidth, but bandwidth does not define the Net.

The fifth reason is that they are open to improvement by anybody. They are not the property of any company; nor are they limited by law and regulation. They are too generative for that. It is the generativity of Linux and the Net that makes them very different from everything else we call infrastructure — as well as from everything else, period, They support origination, production and reproduction with a degree of fecundity that shames the most reproductive species.

In addition to a host of commercial applications and services, Linux and the Net support nearly the entire FOSS (Free and Open Source Software) portfolio, which now numbers upward of a half million code bases. These are as practical (and in some cases as essential) as any element in the periodic table, and yet their population is growing on pace to outnumber the world's species — while evolving faster than any of them.

In his new book, In The Future of the Internet — and How to Stop It (Yale University Press, 2008) Jonathan Zittrain describes the Net as a chaotic and organic creation, ideally suited for boundless generativity by its inhabitants. Jonathan defines generativity as "a system's capacity to produce unanticipated change through unfiltered contributions from broad and varied audiences." In an earlier research paper, The Generative Internet, he explains,

Generativity denotes a technology’s overall capacity to produce unprompted change driven by large, varied, and uncoordinated audiences. The grid of PCs connected by the Internet has developed in such a way that it is consummately generative. From the beginning, the PC has been designed to run almost any program created by the manufacturer, the user, or a remote third party and to make the creation of such programs a relatively easy task. When these highly adaptable machines are connected to a network with little centralized control, the result is a grid that is nearly completely open to the creation and rapid distribution of the innovations of technology-savvy users to a mass audience that can enjoy those innovations without having to know how they work.

Linux and the rest of the FOSS population fit this description, yet are exceptions to it because they are not the products of "audiences", but rather of composers, working in coordinated ways to create and improve the code itself.

FOSS code is pure building material. The free, abundant and practical nature of this building material gives it some qualities of commodities; yet its generative nature is exceptional to traditional economic constructs. It also inconveniences economic belief systems that anchor their perspective in the work of business, government, or both. FOSS goods grow naturally outside either context.

In The Future of the Internet, Jonathan Zittrain shows how the Net and PC operating systems are generative by locating them at the waists of hourglasses:

Both make possible an endless variety of invention and innovation both above and below them.

The Net (on the left, represented by IP, or the Internet Protocol), supports not only a variety of higher level protocols, but applications in absolute abundance. Below it can run on any variety of media, using any number of data link protocols.

Linux (on the right, along with other OSes), can also support an infinite variety of applications, while running on an infinite variety of hardware.

Yet both are not at the bottom in these illustrations. Their infra roles are in the middle. Prepositions are essential for framing many metaphors. Because Linux and the Net run on media and hardware, they seem to be dependent variables of those. They are higher up the stack, and therefore less infra. Add the physical constraints of structure, and you have a conceptual recipe for subordination.

If we conceive infrastructure as a hierarchy of physical dependencies, then Linux and the Net are both subordinate what they run on, and what carries them.

If we conceive infrastructure as utilities of most basic sort, then Linux and the Net qualify.

If we conceive infrastructure as something one must understand deeply in order to create architecture, and devise strategy, it is essential to understand what we are dealing with here — and to understand it on its own new terms, as well as those that have been around a long time but need to be revisited.

Take for example Craig Burton's point of view.

Back in the 1980s I watched in amazement as Novell utterly changed the Local Area Network (LAN) conversation from one about "pipes and protocols" to one about services. Before Novell, LANs were all silos. Arguments were about which "pipe" topology (ring vs. bus vs. star) was best, and which datalink protocol (Ethernet vs. Token Ring) was best. Magazines like Data Communications were fat as phone books with stories and ads that compared the isolated stacks of Corvus, Digital, IBM, Sytek, Wang and a bunch of other companies. Then Novell came along and said, "We don't care what wire you use, or what protocols you use on them. You need services, and we'll give you that, starting with file and print. Those run on our Network Operating System, called Netware." Almost overnight, the game changed. A whole new industry grew on top of Netware and other Network Operating Systems (NOSes), and later on top of PC operating systems. You might say that Linux took off because it turned UNIX into a NOS. Today Novell is a Linux company that continues to sell Netware. But the generative role played by Linux today is one that was pioneered by Netware.

In fact, one of the reasons I coined the expression "markets are conversations" was that I watched the LAN conversation change utterly, almost overnight, and the whole market along with it. The guy behind that was Craig Burton. Way back then, Craig blew my mind. He still does.

When Craig talks infrastructure today, he uses the Burton Matrix:

Here he locates infrastructure in the upper right corner, a combination of Open and Public Domain:

And the process of moving from scarcity to ubiquity as one of commoditization. The arrow indicates a vector of strategy. If you want to create new markets, or disrupt old ones, you create ubiquitous infrastructure. That's what happened with Linux and the Net, even if the motivations were not strategic, the effects were both creative and disruptive. But, Craig points out, if you know that, you can also be strategic about it.

Another mentor is my old friend Stephen Lewis. We met as fellow philosophy majors at the same small college. Steve was an outstanding student whose intelligence, scholarship and powers of articulation humbled me. In the four decades since then he has accumulated an inventory of wisdom and experience that informs an understanding of infrastructure that encompasses — among much else — his photographic studies of Ottoman architecture and his cultural roots in working class New York. In a recent blog post Steve sourced Joshua B. Freeman’s Working Class New York (New York 2000), "a penetrating examination of the unique ethos, economic history, and social and physical infrastructure of the City..." quoting this passage:

Endlessly frustrated by its difficulties and brutalities, try as I may I find it difficult to imagine living elsewhere. What keeps me in New York is neither the high culture of museums and concert halls nor the unrivaled opportunities for working, eating, and spending that New Yorkers revel in. Rather it is a sensibility that is distinctly working-class — generous; open-minded but skeptical; idealistic but deflating of pretension; bursting with energy and a commitment to doing.

To borrow again from Jonathan Zittrain, what Steve and Joshua (and I) love about New York is its generativity — which Joshua locates neither in physical plant nor within cliché'd architectural and cultural frames, but at the base level of a culture: the working people who get stuff done.

It is wrong to assume, as we have been doing throughout history, that those responsible for the foundations of civilization are primarily its leading figures and institutions. While those entities are certainly involved, full respect must be given to the invention, as well as the hard work, being done by the uncredited many.

Take the matter of device drivers for the Linux kernel. The development of these requires a symbiotic relationship between the manufacturers of devices and the programmers who make those devices useful. Over the last several years, Greg Kroah-Hartman and other kernel hackers have been working to bring these parties together. Here's one excerpt from his latest Driver Project Status Report:

A funny thing happened though, what I figured would be a project flooded with requests for companies to get hardware working turned into anything but that.

Two major things then happened, both of which I could have never expected:

It's this last point that made me worry. Yes, a number of companies did ask for drivers to be written, and we have done so, but not as many as I originally imagined.

Then,

I tried my best with a general announcement of, "Tell me all of the hardware that you know of that is not supported by Linux!" The response by users was overwhelming. My inbox was flooded with hundreds of messages, and the wiki page: http://www.linuxdriverproject.org/twiki/bin/view/Main/DriversNeeded was created.

In other words, the driver project, like the kernel project at the heart of Linux, is run by individuals who are "generous; open-minded but skeptical; idealistic but deflating of pretension; bursting with energy and a commitment to doing". Big name vendors are involved, but they're not at the heart of the project.

This applies to physical infrastructure as well. Several weeks ago, guided by Gordon Cook (a guru of post-telecom architecture and economics, I joined a couple dozen other infrastructure obsessives in Loma Linda, California, where we were schooled by James Hettrick, who led the build-out of Loma Linda's fiber-based connectivity plant. James grew up on a ranch in northern Montana where self-sufficiency and resourcefulness extended to telecoms infrastructure: they rolled their own. Too far from civilization to interest Ma Bell or any other phone company, his family and other ranchers built their own phone system. What he learned there is still being leveraged in the work James is doing today with others working at what we're stating to call "Level Zero". That work is equipping individuals and small contractors to do the work of building out high-capacity "pipes" from the edge in. Costs of fiber and other cabling (along with wireless) are going down, while the variety of tools and materials are going up. (You can see more in this series of captioned photos from James' session.)

What we are going to see, over the next few years, is the growth of a new building industry around the physical infrastructure required to bring the Internet to everybody — one that grows outside the existing connections provided by phone and cable TV carriers, but will also connect to them. The carriers aren't going to make the first move here. They're too anchored to their old telephone and television based business models. The smart ones will work with the James Hettricks of the world to get The Job done. In the long run, the job will require both the big and the small, the innovator and the disruptor. Consider the natural generativity of the construction business and you start to see what is bound to happen here.

And it's not just "bottom-up". You can't make microprocessors and circuit boards in your basement. Without large backbones and routers we'd still be talking to printers and file servers on networks that remain terminally local. Linux and academic papers may grow on the human equivalent of trees, but real infrastructure requires real capital investment, sometimes in the many billions of dollars. In the vast ecology of real-world business, there is much that can only be done by large commercial and government efforts, usually in concert with one another — especially if the nature of those efforts are inherently infrastructural. Even if you fill your neighborhood with fiber or wireless data paths, you're not going to lay trunks along railroads or under oceans. And even if you could, you would still have to do business with other backbone companies that will wish to be paid for carrying your bits over their pipes.

On the Linux side, we need to take up Jonathan Zittrain's challenge to preserve the generativity of the Net and the devices we connect to it. The other route is toward what he calls an "applianced" future. Lots of appliances run on Linux. There is a good chance your TV and set top box are among them. They might be hackable, but why bother? They weren't made for that.

The world, however, was made to support hackability. That support is what gives us our infinite varieties of infrastructure.

What we need now is to start understanding new forms of infrastructure on their own terms, and to understand more deeply what infrastructure has been all along. What is there about the nature of infrastructure that can help us understand what Linux, the Net and their like are about? And what do these new members of our technological and economic ecosystem add to our understanding of infrastructure? How can we align two concepts as rhetorically different yet practically aligned as infrastructure and generativity? Answering these requires examining both at a depth one cannot reach just by following the constant roil of news, or by standing amidst the parochial interests of categories and factions — including such academic disciplines as computer science, history, political science and economics.

Where do we start? One place is the comments here. I'll be interested to see how those go. There are already conversations taking place around this topic. If you're interested in contributing, let me know.