I never made a big thing out of it, but I always had trouble with IBM's claim, way back in 2000, that they would invest a billion dollars in Linux. Maybe I missed it, but I never saw any detailed accounting about that. It looked and sounded like a PR gimmick to me, even if it was true.
But now, three years later, we're seeing the payoff in hard numbers.
According to Gartner's Dataquest group, IBM's first quarter Linux revenues for 2003 were $197 million. That's 55% more than the same quarter one year earlier, and 32.4% of the $609 million total for the whole Linux category. Hewlett Packard came in second with $153 million and a 25.2% share. Dell was third with $120 million and a 19.7% share. HP's revenues were 6% down from the previous quarter and Dell's were 2.3% up.
As I recall, Dell wasn't especially promoting Linux during that time, but Oracle was -- for the 9i database on Dell servers. HP was promoting Linux, among other things, in its advertising. I suspect the Oracle connection made the difference there. So, I suspect, did the mojo Dell seems to have going in general. Many of the big company IT guys I've talked with lately are buying Dell.
But the big story is still IBM. Gartner says IBM's Linux revenue number is up 55% from the first quarter of 2002.
True, the big get bigger; but there can be little doubt that IBM is reaping the rewards of a commitment to Linux that is both unambiguous and huge. But is that all there is to it? Is there something more strategic going on?
I think there is, and it stands in stark contrast from Microsoft's hostility to Linux, and from both HP's and Dell's less-than-absolute enthusiasms for Linux.
IBM saw that Linux was a natural and integral part of the growth of the Internet - and that, in fact, Linux was becoming the primary Internet-compatible operating system.
Like the Internet, Linux and open source are part of a demand-side revolution. In other words, they are less about what vendors do for customers than about what customers do for themselves.
So, starting in 2000, IBM followed the lead of its own engineers, and made an enormous and highly strategic commitment both to Linux and to open source development. The company boasted about making a billion dollar bet on Linux. And that was just in its first year of serious Linux investment.
As a result, today's IBM production environment increasingly involves both Linux and open source.1 Every day, IBM's Linux Technology Center Web site2 lists contributions to Linux and other open source projects by individual IBM engineers.
Significantly, many of these improvements to Linux and other open source products involve IBM's own hardware and enterprise software, especially as they pertain to big iron infrastructural systems such as the IBM s/390 and zSeries mainframe computers.
Today IBM's Linux technology projects3 involve clustering, device drivers, networking, performance, printing, quality assurance, security, standards, system management and many other topics.
Clustering
And this topic list just includes Linux open source projects. There are internal projects as well that also involve Linux - especially Linux deployment on mainframes and other large systems.
It's important to note that Linux is spreading gradually across IBM's hardware and software product offerings. There is, however, a wide range of emphasis on Linux at IBM.
The big Linux thrust at IBM today is server consolidation on mainframes and other large systems. At the other end of the scale, in personal computers, Linux is available, but not emphasized.
This does not mean, however, that Linux won't be emphasized in future PC offerings. I'm betting it will. But for now IBM appears content to let Microsoft have most of the PC operating system business.
Meanwhile, IBM designs and sells the PowerPC chips that are the brains of Apple Macintosh systems, which are themselves based on a free and open form of UNIX called BSD.
The way IBM sees it, customers should be able to bring Windows, Macintosh and Linux desktop systems into their production environments without worrying about compatibility or interoperability.
This is a new thing for IBM.
Customers have always wanted compatibility and interoperability. For most of its history, IBM's was glad to give customers compatibility and interoperability, as long as it was with IBM's own stuff. To be compatible meant to be IBM compatible.
Now compatibility mostly refers to standards nobody owns, everybody can use and anybody can improve. By supporting those standards, IBM has unburdened itself of the need to control its customers' environments, and allowed far better interoperability between its own offerings, and those of competitors and the open source community.
This is important, because, customers today want free and open source alternatives to expensive software and hardware. By offering and supporting those alternatives, right alongside its own proprietary products, IBM is giving the market what it wants.
This good faith effort on IBM's part also allows the company to introduce Linux and open source gradually to more and more of its hardware and software portfolio.
Other large competing companies, notably Hewlett-Packard and Oracle, have similar strategies. But no other company has IBM's extreme breadth of product categories -microelectronics, storage systems, printers, networking, and computers ranging from hand-helds to mainframes.
IBM's commitment to Linux, and its contributions to Linux and other open source community efforts, carries the promise of compatibility and interoperability across all of those categories.
This strategy also takes Linux own growth and evolution into account. For example, the Linux kernel continues to grow steadily in the types of computing it supports. Kernel version 2.6, for example, will up to 16-way multiprocessing and many other improvements that will make clustering and server consolidation even more attractive.
Let's take a closer look at these two technologies - especially the second, server virtualization, because that's where IBM is putting most of its Linux energy these days. Then we'll look at other Linux developments and applications.
Clustering and Consolidation
Clustering and consolidations are two opposite approaches to computing.
With clustering, many small computers combine to form one big one - a supercomputer.
With consolidation, one big computer - usually a mainframe - hosts many smaller servers.
In both cases, one thing behaves like another. Lots of small computers act like one big one, or one big one acts like lots of small ones.
There are enormous efficiencies and cost savings involved in both approaches. And IBM is playing a leading role in both market trends. Let's look at clustering first.
Clustering
Most of the top 500 supercomputers in the world are Linux clusters. Of the top 40, 22 are from IBM, 5 are from Hewlett-Packard, 2 are from NEC, 2 are from Hitachi and 2 - numbers 39 and 40 - are from Cray, a name that used to be synonymous with supercomputing.
Of the Top 500 supercomputers, the most cost-effective, by far, use Linux. So do almost all the new ones.
The two virtues of clustering are
* High Availability, or HA; and
* High Performance, or HPC
At present there are half a dozen open source high availability clustering products, plus a couple dozen proprietary commercial alternatives. In the high performance department, there are a huge variety of offerings, each with its own application program interface, or API. This has resulted in many more middleware products but not a lot of interoperability between them.
To address this problem, IBM has come up with a non-proprietary open clustering framework that is royalty-free and vendor agnostic - which it has to be or it won't get adopted.
In sum, the clustering market is now almost entirely about Linux, but it's also still new enough that there are a lot of problems left to be worked out, including common APIs and interoperability between applications and middleware.
Consolidation and virtual machines
Server consolidation involves a process called virtualization. This is where a single operating system - or a virtual one - operates across multiple hardware systems, or in multiple numbers on a single hardware system, each within their own partition. Mainframes can support up to thousands of partitioned virtual servers, though in practice the maximum number tends to be in the hundreds.
Virtualization itself is a hot category, quite aside from IBM. A company called VMware, for example, started with a handful of customers in 1998, and today has more than 5,000 corporate customers and a million registered users.
VMware products allow server virtualization across single and multiple Intel x86 hardware systems. Among other things, VMware's desktop product makes it possible for a single desktop - or even a laptop - to run Windows, Linux and Netware... and even to network the bunch of them together. VMware also has partnerships with Dell, HP and NEC in addition to IBM.
Chasing VMware's heels are companies like SWsoft, which offers Virtuozzo VPS, or Virtual Private Server, which allows mainframe-like partitioning across multiple servers.
It's easy to lose track of these other developments when looking at what's happening with IBM's Linux server consolidation offerings, which grow and change almost daily.
The important fact, for the purposes of this talk, is that IBM has no serious competition at the high end of the Linux server consolidation business.
Let's take a closer look at where IBM is coming from with server virtualization and how it plays across the company's hardware product lines.
Linux server consolidation on IBM systems
The concept of virtual machines has been around since the IBM developed the first mainframe with virtual memory, the S/360, in 1967 - along with a virtual memory operating system called VM.
With VM, the mainframe becomes a computational whole that can be split up or partitioned into any number of independent parts, each a fully capable computer.
Today's IBM eServer mainframe platforms include both 32-bit and 64-bit versions.
The 32 bit version is called Linux on S/390, and the 64 bit version is called Linux on zSeries.
Linux on S/390 has been available since 2000 from IBM developerworks, as well as from Linux distribution partners such as SuSE and Red Hat.
It runs currently on the S390 Parallel Enterprise Server G5 and G6 and Multiprise 3000 processors. It will also run on zSeries in 31 bit mode.
A new release of Linux on zSeries has been available since May of 2003, and supports the latest versions WebSphere and DB2, while delivering a variety of other improvements.
Linux on zSeries also takes advantage of the latest version of z/VM, the VM operating system for IBM's 64-bit z/Architecture. The zSeries can support up to hundreds of virtual Linux servers on a single mainframe.
While the zSeries is at the core of IBM's server consolidation offering, consolidation is also available with IBM's pSeries and iSeries as well.
The pSeries is built around the company's 64-bit POWER4+ micropocessor, which allows up to 8-way symmetrical multiprocessing,or SMP.
Like IBM's mainframe families, the pSeries supports logical partitioning, which in this system allows up to eight independent virtual servers, each with its own operating system.
The iSeries is the latest continuation of the AS400 family, and runs on PowerPC-family microprocessors. iSeries machines can support up to 31 Linux partitions on a single system, and can be used to complement the zSeries in environments where both are in use.
The iSeries is also able to simultaneously support other operating systems, including various flavors of Windows, OS/400 and AIX, on the same chassis.
The iSeries also has advantages in scalability and support for distributed and interactive applications.
In consolidated systems, virtual Linux servers live in LPARs, or logical partitions.
A high degree of control over LPARs is one of the big advantages of server virtualization. You can selectively assign resources to each partition, which can have one or more profiles. You activate a partition by selecting one of its profiles.
You can use alternative profiles to start partitions with different combinations of resources, or to start partitions indifferent boot modes.
For instance, you might create a profile for running a partition with 8 processors during the week, and create an alternative profile for running it with only 4 processors on the weekend. This kind of resource management is unique to environments with consolidated and virtual servers.
You can have any or all LPARs active at any time, assuming there are sufficient resources for them all to run simultaneously.
Each partition has its own reserved section of the system's non-volatile RAM, where it stores its customized boot preferences.
These capabilities are constantly being improved. For example, recent LPAR improvements eliminate system restarts and allow dynamic movement of resources, such as processors, memory and interactive performance parameters. This means that, during workload peaks, you can move entire processors or even partial processors from one partition to another.
The ability to make these moves differs from one hardware series to the next, but steady improvements keep coming to all of them.
Now lets look at server consolidation benefits.
Server Consolidation Benefits
There are four main benefits of server consolidation:
* Lower Total Cost of Ownership (TCO)
* Improved service levels
* Faster time to market
* More manageable IT environment
Of those four, TCO is the most important. Here are some of the ways your TCO can be reduced by server consolidation:
* Rapid deployment and optimization of fresh servers, on demand
* Using excess capacity on existing mainframes
* Virtualizing test, production and backup servers
* Leveraging existing investments in hardware
* Easy migration from other hardware
* Reduced complexity through simplification of server, network and storage deployment and management, all resulting in increased availability
* Streamlining the server farm environment
* Increasing utilization of new and old hardware, software, and networks.
* Lowering costs in employee labor, floor space, and energy consumption.Lower total costs including employee cost, floor space, and energy.
* Increasing server uptime and availability
* Lowering downtime costs
* Lowering consolidation costs of test and development servers
* Faster deployment of new servers and software
* Easier management of servers, applications and other resources
Some of these factors alone might justify server consolidation. Let's look at two consolidation strategies and the benefits involved.
The first is Centralization.
Here servers currently distributed in different floors, buildings or cities can be brought together into one location with a relatively small footprint. The number of servers might also be raised, allowing higher server/employee ratios, and allocating more resources for applications.
The second is Resource Utilization.
Here different servers running different applications are each given their own virtual Linux server in its own partition, on one hardware system. To put this in context, many hardware systems today are running at only 5 to 15 percent utilization.
Consolidation allows the system to make more complete use of available compute power and other local resources such as disk arrays.
Again, administration and maintenance are also made much more convenient. Depending on the software license models, consolidation can also save on license fees.
Now, both of these strategies are do-able on a well-endowed Linux server running on sturdy Intel hardware. But they also violate the old UNIX dictum about one server per application. Even though Linux and other forms of UNIX are fully capable of running many concurrent processes and keeping them from interfering with one another, there is still the risk that interference will happen. And the risk is to all the applications and all the users involved.
Virtual consolidation using z/VM allows each Linux server to share hardware resources in a secure, transparent and fully dynamic way. To the user or the sysadmin, each Linux server appears to be a physically complete server. But it's actually virtual - and, more important, logical.
See, each new server is bound by logical definitions, not physical ones. Whole servers can be cloned - copied and pasted, as it were - within one hardware framework. Server farming becomes a logical rather than a physical exercise. Rather than adding more and more boxes, or racks, or blades, you simply do it all in software.
Choices and trade-offs
Here are some of the cautions and trade-offs you should keep in mind with Linux consolidation - and, in fact, with any large new deployment in your production enviornment.
* Third party software. Some ISV, or Independent Software Vendor, packages may not be available for Linux, or on Linux servers consolidated on zSeries or other IBM systems. Make sure your applications will work, or can be migrated to new systems.
* Network bandwidth. More servers - even if they're all virtual -require more network bandwidth. This should be taken into account.
* Recruitment and training. Linux or UNIX expertise is very different than mainframe expertise. Recruitment and training in one form of expertise or another may be required, and can get expensive.
* Vendor dependencies. Working with IBM - or any large vendor - as a major supplier involves a serious commitment, not only to dependency on a highly proprietary hardware environment, but to a large company that is, in many ways, the sole source of expertise, support and training. All these potential costs need to be taken into account.
* Personnel cost savings. Major IT projects such as server consolidation invites positive changes in the processes and organization of IT staff and of other corporate departments as well. If the company wants to maintain its current way of doing things, the savings may be minimal, and the costs high.
* Depreciation. The net depreciated value of existing hardware and software needs to be taken into account. Replacing either hardware or software may not be justified. Server consolidation makes the most economic sense when leveraging existing investments, or when making entirely new investments.
* Leasing and service contracts. Some server leases must still be paid, even if they are moved to a virtual machine state.
In examining Total Cost of Ownership, you need to take six factors into account -
1. Hardware
2. Software
3. Personnel
4. Real estate
5. Migration and
6. Downtime
Hardware is the big variable. You need to account for -
* Servers
* Disk drives
* Networks
* System management
* Racks, cable and other infrastructural items
In any IT project, the idea is either to lower the total costs while maintaining or achieving specific target performance and service levels, or to raise those levels at the same costs. All the variables involved need to be kept in mind.
Applications
You can run a nearly infinite variety of applications in a Linux environment. Here's a quick list -
* Database
* Collaboration
* System management
* Requirements management
* Web services
* File and print
* Scripting
* Software development
* Network infrastructure
* Applications
* Security, directory and management
The first five on that list are where IBM has offers large, enterprise-scale applications of its own. For database, there's DB2. For collaboration, there's Lotus. For network and system management, there's Tivoli. For requirements management, there's Rational, and for Web services, there's WebSphere.
These are large proprietary products that are each an environment of their own. All work with Linux to some degree - a degree which in each case is bound to increase.
At the same time, free and open source applications are also available for most of the same product categories. In database there are MySQL and PostgreSQL. In collaboration, there's Jabber instant messaging. In Web services, there are PHP and Apache. In scripting, there's Python and Perl. The list is long.
Commercial software products are also available from ISVs such as Oracle, SAS and SAP.
IBM actually plays three roles here. It develops commercial software of its own. It contributes to open source product development, and it provides both platforms and partnerships for third parties and ISVs.
Naturally, much of the company's open source work is around compatibility issues - for example where Linux, or another open source product, needs to work with IBM hardware or software. But this is a good thing, if you happen to have a predominantly IBM production environment.
Finally, IBM is also involved in with other development efforts in other environments that overlap with Linux. Java is one example. Microsoft's .Net is another.
IBM's high degree of commitment to Linux, however, helps assure maximum compatibility and interoperability with these other environments.
Examples
On the vendor side, companies are finding it easy to port their Linux applications to the consolidated environment.
Bill Carico, President and CEO of ACTS Corporation, says his company didn't need to change a single line of code on its TestManager software when they ported the product to Linux for the S/390.
Alex Liu of BMC Software reports that porting the company's Internet monitoring servers to Linux on zSeries was a very smooth process.
Tony Strayer of Serena Sofware says the enablement effort to make the company's ChangeMan DS support Linux on zSeries was surprisingly modest, considering the complexity of the application and the volume of code involved.
On the customer side, Lawson, a large industrial equipment supplier, worked with IBM to migrate the company's Elevon Web application to zSeries hardware running virtual SuSE Linux servers, and then comlpemented Elevon with IBM WebSphere middleware, accessing parts catalog information stored in an IBM DB2 database. The result is a more robust and much less costly ecommerce system that also complements additional the company's IBM xSeries, pSeries and iSeries servers as well.
Winnebago Industries, the recreational vehicle maker, avoided a $100,000 price tag on an email system upgrade by replacing its Windows NT environment with an eServer zSeries mainframe running SuSE Linux and software from Bynari.
Tamkang University in Taiwan consolidated its data storage by deploying a comprehensive Storage Area Network (SAN) and data center, all running on virtualized Linux on a zSeries mainframe, which replaces a decade-old IBM 3090 -15J mainframe.
Server performance multiplied by eightfold while reducing power consumption by 95 percent. Server floor space was reduced by nearly 97 percent, from 540 square feet to only 18 square feet.
And then there are countless instances where SAMBA servers are implemented on virtualized Linux servers.
SAMBA allows a Linux machine to function as a Windows file or print server.
A Windows NT 4-server environment, for example, consists of 3 kinds of servers: 1) primary domain controller, 2) backup domain controller for authentication, logon and access control, and 3) file and print servers.
SAMBA can replace all 3 kinds of servers. You can also mix Windows and Samba servers.
For example, you can have Samba file and print servers connect to a Windows primary and backup domain controllers, and have a SAMBA primary domain controller doing authentication for a Windows file and print server - with Linux running underneath all of them.
Migration is easy because servers can be moved one at a time. You don't have to switch the whole environment in one swoop. Samba can also be combined with an LDAP server as a back end for authentication. This lets you merge authentication for Java, UNIX, Linux and Windows. Java application servers such as IBM's WebSphere have complete LDAP integration, so Web applications, UNIX and Linux systems, and even Windows clients and file and print servers can use the same LDAP server as a central authentication authority.
Future developments
Linux adoption is growing at a colossal rate of speed, and with it the variety of implementation strategies and products only gets larger and harder to follow. Just while I was writing this presentation, IBM made two significant announcements regarding Linux server consolidation, both of which I mentioned here. There are bound to be many more announcements as well.
Meanwhile, the Linux kernel slowly evolves. The long-proven 2.4 kernel now in use on most enterprise systems covers a lot of bases, but not all of them. For example, it has a number of limitations for heavy-duty enterprise applications, such as database.
The 2.6 kernel should appear sometime before the end of 2003, and will find its way into commercial distributions between three and six months after that.
This new kernel will likely offer improvements in multiprocessing, volume management and other areas. It should also have the ability to set file access permissions using access control lists, or ACLs. Task scheduling and thread handling will be also be improved.
Wim Coekaerts, who leads Oracle's Linux development team, says Linux also needs to have a Logical Volume Manager at some point, rather than the device manager currently slated for version 2.6.
What matters is that the improvements in Linux are answering demand. They are what both customers and vendors are asking for - and, in some cases are actually contributing to the kernel.
The same thing goes for both computational clustering and server consolidation. In time, there may be more economical ways the make supercomputers than by clustering Linux servers, and there may be a hardware vendor offering a wider variety of Linux server consolidation options than IBM. But I doubt it in either case.
Summary
In the old days of computing, people used to say nobody got fired for buying IBM. What IBM has done recently with Linux has been to help create a world where nobody gets fired for specifying Linux and open source, and where nobody in a large enterprise can look at either subject and not consider IBM.
Today IBM is simply the biggest corporate name in both Linux and open source development, and for good reason. Nobody comes even close to doing as much with either one - and therefore also with its customers.
The challenge in your IBM production environment is finding the best mix of goods and services from IBM, from other vendors, and from the open source community, which may very well include your own organization and your own personnel. Again, Linux and open source are a demand-side revolution. They are a product of what customers do for themselves, not just what a company does for them.
By adopting a Linux strategy, IBM has also adopted a customer self-reliance strategy. Which is why, in the IBM Linux production environment, the choices are all yours.
Sources:
Clustering on Linux, a presentation by Alan Robertson, IBM Linux Technbology Center
http://opencf.org/documents/talks/LWCE-NYC-2002/ LWCE-NYC-2002.pdf
Top 500 Supercomputers < http://www.top500.org/>
http://opencf.org/
http://linux-ha.org/framework/
http://linux-ha.org/
http://www.vmware.com/
http://www.vm.ibm.com/
http://oss.software.ibm.com/linux390/index.shtml
http://www-1.ibm.com/servers/esdd/articles/openmosix.html
http://www-1.ibm.com/servers/eserver/zseries/os/linux/
http://www-1.ibm.com/servers/eserver/zseries/os/linux/vif/
http://www-132.ibm.com/content/home/store_IBMPublicUSA/en_US/eServer/iSeries/
http://techupdate.zdnet.com/techupdate/stories/main/0,14179,2870157,00.html
http://www.esj.com/news/article.asp?EditorialsID=265
http://www-1.ibm.com/servers/eserver/zseries/os/linux/facts.html
http://www-1.ibm.com/servers/s390/pes/
http://www-1.ibm.com/servers/eserver/pseries/news/features/2002/annc_1112.html
http://www-1.ibm.com/servers/eserver/zseries/solutions/s390da/linuxenable.html
Case studies:
http://www-1.ibm.com/servers/solutions/serverconsolidation/case_studies/success.html
http://www-1.ibm.com/servers/eserver/zseries/solutions/s390da/linuxenable.html
The eCluster 1350, for combining blades and clusters:
http://news.com.com/2100-1010_3-1008235.html
1 IBM developerWorks
2 IBM Linux Technology Center
3 IBM Linux Technology Center projects < http://www-124.ibm.com/linux/projects/>