July 2003
A Tale of Two Stories
As Linux continues to win acceptance as standard IT infrastructure in enterprises around the world, it is becoming increasingly clear that there are two very different points of view on the matter, and that they do not agree. One point of view is internal, the other external. Both agree that Linux and open source are Good Things, and are changing enterprises for the better. The main difference is in where they place the credit.
The inside view credits the organizations intelligence, resourcefulness and expertise. The outside view credits vendors with the same qualities. Credit is due to both, of course; but the latter gets far more play in the press than the former. And in many vendor-angle stories, zero credit goes the very qualities that not only account for successful use of Linux and other open source software in enterprises, but also for the creation of those applications in the first place.
For one sample of an inside view, here's Leon Chism, Chief Internet Architect of Orbitz, the online travel site:
Orbitz is a company that leverages open source projects in its production environment and isn't afraid to be public about it. We currently use over around 750 Linux machines in our production environment. Our web servers are Apache running on Linux, our application servers are proprietary servlet engines running on Linux, and the back end "booking engine" is comprised entirely of Java services running on Linux. Lastly, the software that does the low fare searching that is one of the key differentiators between Orbitz and our rivals is also running on Linux... The move to Linux on the application server layer was accomplished earlier this year. We were able to replace 12 Sun E4500 machines each with 12 processors with 50 dual processor Intel machines at a significant cost savings and performance improvement.
When we started building this beast, we had always planned on being a top 20 website in terms of bits pushed, pages viewed, visitors a day, etc. Planning for that means that you very quickly come to the conclusion that horizontal scalability wins the day over vertical scalability without much of a fight. So from day one, all the systems we have designed and developed have been targeted for horizontal scalability and dynamic addition/subtraction of hardware. That decision and that design mentality is the big hurdle to overcome.
It helped that all our developers have been on Linux since day one, which instilled a lot of confidence for the cutover.
For a sample outside view, here's William M. Bulkeley, in "Out of the Shadows: Open-source software is not only becoming acceptable -- It's also becoming a big business" -- a feature that ran in the March 31, 2003 Technology section of The Wall Street Journal:
Free software is going mainstream.
Open-source software, developed outside the corporate framework and available at no cost to anyone who wants to download it, has long been considered by some to be somehow untrustworthy or inadequate. But that perception is giving way to an appreciation of its improving quality and increasing versatility.
And, perhaps most important, companies have found ways to make money by providing services to open-source-software users, or by packaging this free software with products they sell. With the profit motive driving its promotion, free software is cropping up all over corporate and government computers.
When I first saw that Bulkeley piece, I thought Cool! We've finally crossed a threshold when The Wall Street Journal says "Free software is going mainstream". But when I read the piece closely a bunch of questions came up:
I wondered, What about customers' needs -- to save money, time, effort and capital investments? Don't those account for Linux' success too?
Not if you only look at what vendors are doing. And not if you assume nothing significant happens unless a big vendor pushes it.
Ask yourself: Is IBM responsible for Linux' success in the enterprise? Michael Totty thinks so. He wrote the lead article -- "Ten Technologies You Need to Know About Now: From Wi-Fi to Open Source" -- in that same Wall Street Journal section. Says Totty,
It helps to have heavyweight backers. Linux, the open-source computer operating system, might have remained a tool of hobbyists and software rebels had it not been embraced by giant International Business Machines Corp.
In fact, IBM's official enthusiasm for Linux came only after company brass discovered a critical mass of Linux usage and know-how among its own rank and file technologists. This was explained to me by a roomful of freshly-minted IBM Linux advocates at a meeting during the August 1999 LinuxWorld Expo. When I asked why IBM was suddenly so rah-rah about Linux (this was long before IBM launched its famous "$1 billion" Linux campaign), I was told that the company had surveyed for six levels of "Linux awareness" among six hundred workers in one technical division. The bottom level of awareness was "can spell Linux". The top level was "hacks kernel code". All six hundred could spell Linux. One hundred and twenty were hacking the kernel. So, while the ultimate decision to advocate Linux was a smart move by the company's brass, it was clear that the tactical decision had already been made by the company's technologists.
That's why, aside from its tongue-in-cheek "Peace, Love & Linux" campaign," IBM's Linux strategy has never been about mainstreaming hobbyists and rebels. It has always been about leveraging its own resourcefulness and intelligence -- and appealing to the same in its customers.
And let's face it: Linux users are tough customers. Here's Chism again:
On the relationship front, there is a vendor that we have and are still considering replacing with an open source solution. For the most part, the application works, but when we find bugs in it, the resolution process leaves a lot to be desired. Escalation procedures, arguments about root cause, long discussions about "supported configurations" all lead to muddling the process rather than focusing on solution. When we have issues or questions with Linux, or Apache, these things aren't at issue. We simply use the customer service app to end all customer service apps -- http://www.google.com/ -- and start researching. Or we start perusing source code. Orbitz is a customer that typically pushes the limits of the products we use. Some vendors are up for the challenge and are willing to make efforts to support that need. Some aren't. And we don't know who is who until, as they say, push comes to shove. Using open source products removes that issue. We have an engineering and software development staff that are fully capable of paddling their own canoe, and the support we get from authors, list servs, and Google is more than enough in most cases.
Yet the tough vendor story gets played while the tough customer story does not. Why? Because the tough vendor story has DNA that runs back to the dawn of the Industrial Age. It shows in stories like "Larry Ellison's Sober Vision", by Mylene Mangalindan, in the April 8, 2003 edition of The Wall Street Journal. Citing "Industrial Revolution concepts" including specialization of labor" and "economies of scale", Ellison predicts "the end of Silicon Valley as we know it". According to the story, Ellison "paints a dark vision of the computer industry's future: increasingly standardized products with little distinguishing technology and thin profit margins. Sweeping consolidation, prompting the death of 1,000 tech companies. Fewer start-ups. And a handful of category-dominating winners, which will control innovation...Maturation of the industry will lead to bigger companies, which will offer a wider range of products and take more market share from everyone else." All because "...cheaper computers that use the free Linux software and other technical efficiencies will drive down prices."
The Intelligence Imperative
Leo Tolstoy's War and Peace, written in the midst of the Industrial Revolution, was one long argument against the great man theory of history. While that theory may have fallen by the wayside, the great vendor theory is still current in computer industry journalism. By this theory, nothing significant happens unless a big vendor makes it happen, and makes a lot of money in the process.
One of the most remarkable prejudices of the great vendor theory is the one against commodities. Hardware has long since moved into low-margin territory (much to the delight of those who buy it in volume), but software is still considered a big margin business. Commodification is therefore a threat to that business. This prejudice no doubt owes in large measure to the massive successes of the biggest vendors. Microsoft has a bad quarter when gross profit margins slip below 80%. Oracle is no less exemplary: Its worst quarter in the last five (at this writing) was still over 75%.
Meanwhile, customers love commodities. In fact, they love software commodities so much that they make their own, put them to use and improve them continuously. This do-it-yourself (DIY) process deserves enormous credit for the success of open source, as well as for the success of all the open standards and protocols on which the Net and the Web were built. Had it been up to the high-income and high-margin software vendors of the world, none of that good stuff would have ever been contemplated, much less invented. That's not a knock on vendors (some of which have contributed a great deal to open code, standards and protocols), just an effort to place some credit where it's long overdue.
Simply put, Linux succeeds in IT because it helps smart companies get smarter. Since smart companies care deeply about saving money, getting smarter will naturally involve efforts to pay even less money than before.
One good example is the University Corporation for Atmospheric Research (UCAR) in Boulder, Colorado. In spite of its public sector name, UCAR is a business with substantial customers that include the national air traffic control system. Greg Thompson, a scientist with UCAR, explains how Linux appeals to the company's smarts:
We're very anti-pay-for-anything around here. Cheap hardware is what got Linux going in the first place for us. That was back around '95. Before then we were a Sun and SGI shop. Now we're all-Dell, except for occasional iBooks, which we use because they run a flavor of UNIX. All the rest of our laptops are Linux -- mostly Debian with a little Red Hat. Some of them are dual-boot with Windows, so we can relate to customers. That's why we run a lot of VMware too.
He explains how the company makes itself smarter this way:
Everybody here has a shelf full of O'Reilly books. We like to figure things out for ourselves. For example, I'm a scientist, not a programmer. But awhile back I saw a need to get past manipulating data with Perl and textfiles. So I started reading up on databases and learning MySQL. I wouldn't have bothered if MySQL hadn't been free. We don't have five digit figures around here to run Oracle. We don't need hard-core database stuff. We don't need transaction support. But we do need results that are Web-accessible. So MySQL made sense. Building internal expertise around something like MySQL is straightforward and easy. I do a little research on my own, see if something I need is already installed -- and if it's not I send a help desk email to a sysadmin-- and then the next day I just download a Debian package, run apt-get and I'm in business. Without spending a dime."
The Savings Market
It's amazing how small a software approval bureaucracy can get when you're dealing with freeware rather than costware. When friction is low and costs are zero, free enterprise lives up to its name. Literally.
"Our IT budget is zero," says Elliot Noss, President & CEO of Tucows in Toronto. Tucows' business is hosting one of the largest software download sites in the world, plus running one of the three top-level domain name registries (OpenSRS), which manages around 3.5 million domain names. "We've built everything from the ground up with Linux, Apache, MySQL, PostgreSQL and other LAMP stuff that costs us nothing," Noss says. "I can't even imagine what it would cost to work with a big Sun system, or to run a big Oracle database. If it weren't for Linux and LAMP, we wouldn't be in business."
This is not the kind of stuff that shows up in the vendor vs. vendor sports coverage that still passes for in-depth coverage of the information technology (IT) marketplace, even by respectable publications like The Wall Street Journal. Still dragging one foot in the Industrial Age, the mainstream business press misses a fundamental economic fact of life in the Information Age: The demand side is supplying itself.
With Linux and open source, the biggest enterprise suppliers are internal, not external. They are the smart and resourceful employees making maximum use of low-cost or zero-cost hardware and software.
Resistance is Futile
Of course, there is still plenty of politics involved. Especially with larger companies. But even we see a kind of reverse-Borg effect. Resistance is futile. Take, for example, this email:
I work for a Fortune 50 company whose IT has a new open-source policy, as well as a fully-approved open-source toolkit (with an internal "brand name") available for employee download from an internal web site. Already up to version 3.0, it comes in both Unix and Windows flavors. It consists of GNU tools and others on the Unix side, and the Cygwin toolkit on the Windows side. Linux (Red Hat for now) is one of the "tools."
The guy who heads up the toolkit project is a take-no-prisoners free software advocate, too. The stated purpose of the policy is to permit the company to save money by eliminating license fees and streamlining the software acquisition process, but the clear subtext is to make it possible for the company to protect its intellectual property from third-party (i.e. Microsoft) manipulation.
Part of the new policy is something of "don't ask, don't tell". IT agrees not to complain if anyone decides to install these tools on whatever computer they use. All any non-management employee now needs to do is get approval from his or her local manager to download and install the tools. Managers do not need anyone's permission to install the tools for themselves. Per the policy, IT does not get involved in any way! So spread of the tools -- of Linux and the rest of the free software suite -- happens naturally and organically.
Before the policy was implemented, even GNU Emacs required an official IT review/approval process. Even if your boss wanted it, IT could fight it if they wanted to. But with this new policy, that's now history.
So policy adapts to new facts of development life inside the organization. And those facts are changing fast, in Linux's favor. In a recent survey of Linux developers, Evans Data Corp. noted a rapid shift in development away from proprietary UNIXes and Windows, in the direction of Linux:
Linux is the primary choice of host platform at 40%. Windows 2000 makes a strong showing at 29%, with Windows XP right behind at 12%. The landscape is about to change, however... Next year respondents plan to increase their use of Linux as the primary development platform by 15%, from 40% to 55%. They also plan to increase their use of Windows XP as a development platform by 8%, going from 12% this year to 20% next year. Both Linux and Windows XP are taking at least some of their share from Windows 2000, since it drops 15% from 30% this year to 15% for the next. Other flavors of UNIX comprise an insignificant share of the host development platform this and next year, so Linux is obviously taking more share from Windows than any other OS.
Charlie Garry of META Group points to a similar trend, in respect to data processing:
With distributed n-tier (DBMS, application, Web) server architectures standardizing on Intel, proprietary Unix (Solaris, HP-UX, AIX) will recede to high-end, low-unit-volume, legacy-platform status by 2005/06, displaced by OSs designed for Intel economics: Windows and Linux. Linux will rapidly mature and gain momentum as an ISV reference platform, moving beyond high-volume Web, technical computing, and appliance server environments into mainstream application and DBMS server roles by 2004/05. Linux server growth will initially be at the expense of Unix (2003/04), but will eventually vie for dominance with Windows (2005/06).
What then becomes of the relationship between customers and vendors? Where they engage, both get smarter.
In 1999, The Cluetrain Manifesto said "markets are getting smarter -- and getting smarter faster than most companies." Four years later, we see something similar happening inside companies with respect to their technology vendors. The result is an ecosystem that subordinates vendors to work. This is good for everybody, including vendors. It means the industry is finally growing up.
The Construction Model
To see what a grown-up software industry looks like, we can just follow software's vocabulary to its primary source: Construction. It's no coincidence that the software world has "architects", "designers", "builders", "tools" and "platforms". Nor that the Web has "sites" called "homes" with "locations" and "addresses". Code is construction material. Programmers are builders. And the software business is turning into something very much like the construction business.
Construction is a good model. It's the oldest and largest industry in the world. Sales estimates run into trillions of dollars, worldwide. The industry is filled with commodities, and with companies making a good living selling, installing and servicing commodities. It has many huge vendors, including some as large as Microsoft; yet it has no equivalent monopoly. No one company dominates or defines the industry. Profits are lower than Microsoft's and Oracle's, but they're nothing shameful. Fluor Corporation, for example, has a gross profit margin of 5.5% on revenues of about $10 billion. Georgia-Pacific has a 22.2% on $23.3 billion. Jacobs Engineering has 13.1% on $4.9 billion.
Open source is standard in construction. Its arts and sciences have few secrets. If somebody thinks of a better way to hang a door, pour concrete or put up sheetrock, that know-how is passed along to everybody else. There are still plenty of patents, and a thriving business in pre-fabricated materials that are full of secret stuff. But the nuts & bolts of the trade are exactly that: nuts & bolts.
The same kind of mature regard for openness-where-it-counts has been coming for a long time to the software business. Avery Lyford, President and CEO of Linuxcare, puts it this way:
Think about the world before Bill Gates' open letter to the hobbyists. That's where we're going back to. We're coming to the end of the Gatesian Era in computing history. Yes, the guy is an incredible businessman. Credit where due. But his time is passing. It's just not passing all at once.
Linux is a lock. It's done. The only remaining resistance is around open source, and that's only because a lot of IT people still don't understand it. But that will change soon, too.
The real issue with open source isn't low cost. It's high value. That's what vendors like ours try to bring into the enterprise today....All we have to do is figure out how much of your problem we can help you solve and how we can help you solve it.
In other words, give customers what they want, and don't bother trying to lock them into no-choice scenarios. They've had enough of that, thank you.
Mårten Mickos, CEO of MySQL, believes vendors like his can raise IT consciousness about the value of open source. He says MySQL successfully sells free software (MySQL is GPL'd code) because "there is more value to code you can see than to code you can't see." He adds,
We are part of a huge community of customers and other developers who are all passionate about improving the code base. Every day we are proving it is possible to have a commercial relationship that benefits free software.
That relationship is still not one that happens between the tops of the Vendor and customer IT pyramids. It happens down among the middle tiers of big customers. For MySQL that list is impressive. It includes Nokia, Yahoo!, NASA, Silicon Graphics and Cicso. When Miklos was on a panel with Doron Gerstel of Zend (vendor of PHP) at PC Forum this past March, I asked them if their customers were at the top of the IT pyramids -- at the CTO or CIO level -- or down in the middle and lower tiers, where the work gets done. Both told me their sales and were in the latter category.
Jeremy Zawody, a self-described "technical yahoo" with Yahoo Finance, says ""MySQL will penetrate the enterprise similarly to the way SQL Server did, but with much greater speed. MySQL is to Oracle as Linux is to Windows. It will slowly but steadily creep up the food chain, just like Linux has."
David Sifry, co-founder and CTO of Sputnik (and co-founder of Linuxcare), explains how a breed of free software such as MySQL represents an adaptive response to the autonomous and growing intelligence inside companies:
MySQL owns its code. They hold the copyright. But the code is also GPL'd. That means they can give it away for free and get the benefits of constant improvement to the code by a vast worldwide development community. But since they own the code they're also free to sell it, along with commercial support. If you want help from MySQL the company, you need a commercial relationship with them. Companies like Google, Yahoo and Tucows are big and smart and well-staffed enough to go to their own people for support -- even if they already have a paid relationship with MySQL. But smaller companies, or smaller parts of big companies, may need to outsource some of that support. MySQL is glad to sell it to them.
This approach also works well for, say, a big bank that may be worried about the GPL issue. Fine: they pay MySQL for the code. Then they can build their own extensions into it and embed it in their products. Meanwhile, people who choose to use MySQL code under the GPL get no downside. All the latest features are there. Near as I can tell, there are no downsides, either way. More choice for customers, better code for everybody.
The Deal with Data
So what's happening, as LAMP establishes itself, becoming the commodity platform of choice for enterprises? The action starts shifting up the stack: from operating systems and applications to data. We think it's the information age, not the operating system age," says Larry Ellison. "The OS manages hardware; we manage the software."
Not that Oracle doesn't care about operating systems. For many years the company made a big deal of its OS agnosticism. Today it is arguably ranks with IBM as one of the biggest "Linux companies" on Earth, and glad to say so. Wim Coekaerts, head of Oracle's Linux kernel team, says "Linux is really, really important to Oracle. We are very much a 'Linux company.' He proudly credits the work his kernel development team has contributed to making Linux "enterprise class".
Early this year Goldman-Sachs issued a report titled "Fear the Penguin". It called Linux "an enterprise-class operating system that we believe will have a significant and lasting presence on the IT landscape" and said Linux' "continuing emergence will likely cause considerable changes in the enterprise IT vendor ecosystem." It added, "the technical developments and market forces are in place for it also to become the dominant O/S on the higher-end servers of the enterprise data center, where mission-critical functions are run and the lion's share of IT spending occurs. In particular, we believe that enterprise customers will use Linux primarily to take advantage of lower-cost, higher-performance Intel-based servers and avoid technology lock-in situations. As a result, we believe the Linux-on-Intel model will displace the existing paradigm of premium-priced proprietary systems based on Unix operating systems and RISC processors."
Jamie Lewis, President of The Burton Group, explains why data is becoming a key attactor for Linux as foundational IT infrastructure:
First, a large enterprise is going to have everything, platform-wise. They'll have Microsoft, they'll have SUN, they'll have LAMP. All of it. But what they care about most isn't that stuff, but the data that is the life blood of their business.
That's why the conversation is shifting to Web services. I have to be careful here to differentiate real Web services from the kool-aid you hear from big vendors on conference panels. Real Web services are about XML, SOAP, XML-RPC, SAML and other standards that bring services together. With a common encoding data format, and common protocols for communicating that data, we are beginning to create what I call a software communications bus. It's just like the hardware bus for the card in your PC. You plug a card in there and it can communicate with all the other piece parts in the machine. In this model all we need is just enough in common so that we can exchange just enough information, just in time, to do something meaningful -- without having to go through recombinant DNA exercises in order to interoperate. This is fundamental. It's a huge deal.
And none of this would be happening nearly so fast if there weren't all this smart activity taking place around Linux and open source. The Linux community is building out huge portions of this new model. Look at Mono, Jboss... There are lots of efforts going on that have the combined effect of turning the network into a platform. Once that happens, things will really get interesting. We'll build stuff on top of the Net that will transform it in ways we haven't even thought of yet.
The importance of open formats, in addition to open source, is brought home by this note from another correspondent who works for the same Fortune 50 company mentioned above:
At work, I (and others) use most of the commercial *NIXes, plus Linux, WinNT, and Win2K. Thus need for cross-platform installation and training, open protocols, and open data formats drove interest in OSS. The OSS toolkit grew as a grassroots effort to meet these needs.
The New Standard Story
We have to get accustomed to a world in which Linux and its open source companions serve as Fundamental Infrastructure for IT: a system as standard as two-by-fours, ten-penny nails and sheet rock screws. As that standard establishes itself, the market conversation will change. Vendors will always be participants in that conversation; but their frame of reference will shift from abstract to the concrete -- from "playing fields" and "battlegrounds" to IT projects where they have something useful to contribute.
Last year, when RealNetworks, Inc. announced Helix, its own open source initiative, I talked with CEO and President Rob Glaser about how the software industry was turning into something resembling the construction industry. He agreed, and added this observation: "Everything is a project."
What we need are more stories about more projects in which Linux helps smart companies get smarter -- with or without the help of vendors. Telling those stories is our job here at Linux Journal. Getting more of those stories to us is your job. We look forward to hearing from you.
Doc Searls is Senior Editor of Linux Journal.