A conversation with Roland Smith, Director of Global Operations for LSI Logic, and a leading Linux advocate at the company.
I met Roland Smith last October on Linux Lunacy II, our second annual Geek Cruise. Of all the cruisers on the boat, Roland stood out as a guy who was putting his team's know-how to work in a big way for his company. We caught up last week on the phone when I was doing research for a piece on how Linux helps make smart companies smarter. It seemed to me that what Roland described was a good case history.
What Roland describes is not only how his company makes itself smarter with Linux and open source software, but the uncredited way leadership in the industry often works, with the customers leading the vendors, rather than vice versa.
- Doc Searls
DS: What's your title, and how is the department structured?
RS: I'm Director of Global Operations. I report to the CIO, who reports to the CFO. The department is Information Technologies, or IT. It has a Programming Development Group, an Internet and Business Technology Group, a Manufacturing Systems Group, an Engineering Group... and then I have the Network Group, with corporate data centers, heartbeat services like email and DNS and all that sort of thing -- plus all the e-commerce sort of stuff. There are four of us who report into the boss.
DS: I'm looking for ways that companies are making themselves smarter with our without the help of vendors. Mostly what we hear is about how vendors help solve company problems. I'm looking for how companies solve their own problems -- using Linux to help make themselves smarter.
RS: In respect to vendors, help actually runs kind of the other way around. Let me give you a current example. In our SAP environment we recently decided we wanted to begin inserting Linux application servers. The issue on our side is that Linux servers perform better: they're faster by a factor of 5 or 6 times over HP/UX. So we went out and talked to both Dell and HP and told them we wanted to start doing this. And what we wanted from each of them was an eval server -- one that they think is best suited for running Linux. We wanted four processors and a gig of memory.
A couple of interesting things happened. One was that HP fumbled around and took two weeks to come back and say "What do you really need?" The other was that Dell took three days and shipped us a server. They said "Here. You've got it for three months. Go tell us what you can do with it." That was kind of wonderful.
So then we went to SAP and said we're going to load this server up and make it an SAP application server, and what do you recommend? And SAP said, "Well, we think SuSE 8 runs okay. And we think that Red Hat 7.1 runs okay. But we don't really know." Then we went and talked to Red Hat and they said "We think it'll run nicely on our Advanced Server 2.1, but we don't really know. So how about you go try it out and tell us?"
So we built it. We bought the Red Hat application server, their lowest end of the thing, and we built it and loaded SAP on it, and we've got it in our test environment and it's running incredibly well. We're very happy with it. But it's interesting to me that, of all the vendors out there, only Dell said "We'll happily give you hardware to try it out on, but we don't know anything more than that."
DS: In other words the intelligence is located with the customer.
RS: It is in this case, mostly. That is a true story, happening right now.
DS: May I quote you on that, and name names?
RS: Yes. I would like for HP and Dell to be appropriately noted. We do a lot of business with both of them. But we had never bought a server from Dell before. It's always been desktops from them. They're moving fairly strongly into the server space, and so I'm sure they are anxious to get a server over here, no matter what we do with it, as long as it's... something. In hopes that we'll buy more. Meanwhile HP just couldn't figure out how to make it happen.
DS: Is it perhaps just the part of HP you're dealing with?
RS: It's our HP rep and our account team, which is fairly large. And the funny part about it is, when I initially approached them, I was up at one of their executive breakfasts in Denver for a few selected folks, where they had their main guy in from out East somewhere, who is their Linux guru or something, and giving a presentation on HP and Linux. He was talking about how HP was going to have its feet in both places: in Linux and HP/UX, available in both 32-bit and 64-bit architectures. And they really, really stressed how your only choice, your best choice, was HP.
DS: They're very strong on selling the 64-bit thing. Seems like their main focus sometimes.
RS: Oh, they're very big on 64-bit.
DS: Seems like a narrow market.
RS: They helped develop it and have a lot of money in there, so I think they're trying to recoup some of that investment. But shoot, I think IBM has a better 64-bit solution than Itanium.
DS: Didn't Linus say that last fall on the Linux Geek Cruise?
RS: Yeah. And I kinda agree with him. There's an awful lot of stuff that doesn't run on Itanium. Anyway, it was interesting to me that with all this touting of their Linux stuff, not only could they not get me a server; they didn't know how to make it happen even if I had one. Even more interestingly, Red Hat had no knowledge. They had no idea how to help set it up.
DS: It seems the commercial open source model, at least according to Marten Miklos of MySQL, is to locate most of the R&D outside the company, where most of the expertise is as well.
RS: Yes.
DS: What's the larger context for all this at LSI Logic?
RS: This is our first foray into putting Linux into the company mainstream backbone financial environment. Hopefully in about a month it will be in production. We've got five or six other HP/UX application servers that we intend to take out of production and replace with Linux servers, which just run so much better, and are so much easier to administer, and so much cheaper. And cheaper is the last thing.
We're happy to share how it happened and what we did with whomever is interested.
DS: Is SAP interested?
RS: We spend several million dollars a year with SAP, and they're kind of interested, but... maybe not. Red Hat would be interested to know, but they don't really have time to spend here in this arena. The SuSE folks ... all their response was "Well, we think it runs... " and gave us these versions that they've worked with SAP on.
DS: I know Oracle is very aggressive about certifying its products for Linux.
RS: We have three Oracle databases running on Linux servers. Oracle has been a delight to work with. Of all the vendors, Oracle is probably the most Linux-savvy. And I think that's because it's coming from the top down.
DS: I recently talked with Wim Coekaerts, who heads Linux development there. His nose was a little out of joint that IBM gets the most credit for being THE big Linux company. He thought Oracle was at least as deserving.
RS: I would say that Oracle is probably the best vendor in the marketplace, in respect to Linux. When we told our Oracle account team that we wanted to put up an Oracle database on Linux, they were all over it. They really knew what they were doing. Within a week they had us in touch with their development folks back in Redwood Shores. They had good documentation: it was easy to put up. They were patient. It was easy to run, easy to connect to. Folks who were running on these Oracle databases didn't have a clue that they were running on anything other than just standard stuff. We're very happy about it.
DS: On your SAP project, I know those things tend to be long-term, too.
RS: A year-long project would be a short project. They've been moving very stongly into the customer management relationship arena, because they want to start taking over some of Siebel's marketplace.Siebel has to be the least Linux-savvy company I've found. You can't talk with anybody there about Linux. They simply don't understand it. Which is really kind of unfortunate, because that's expensive software. I'm of the opinion that when companies like Siebel figure out that they can maintain their own margins and reduce costs to customers by getting off these expensive platforms, they'll do it.
I was talking a couple of weeks ago with somebody in Utah who works with library systems, and they were lamenting the fact that nobody's got any money. I asked him why he's still running everything on Sun, which is dead-end anyway. We talked at some length about what it would take to get their product running on Linux. They'd be able to take sixty or seventy thousand dollars out of the initial installation price by doing that. And still maintain their own margins. But like a lot of companies in this business they don't have a lot of money in R&D. One of these days, though, they'll figure it out.
DS: I'm interested in hearing how you guys got into Linux in the first place. I'm especially interested in the DIY -- the Do It Yourself -- side of the enterprise. Some of that has to do with vendor relationships and some of it with simple resourcefulness inside the company itself.
RS: I can tell you a little bit about our story. Our first foray had to do with the fact that we needed to do something substantial in upgrading our DNS. This was about two years ago. My networking team came to me with a proposal on the thing, and it was pretty expensive. They wanted to do it on Sun servers, and it was pretty elaborate. I'd been running Linux personally for quite awhile, so I told them to go away and come back to me with a Linux solution that took advantage of that kind of pricing. They came back with a proposal to use Penguin computers as the Linux platform. They kinda liked the way they run headless and all that sort of thing, so they could remotely administer them. The price was about a third of the other. The were incredulous at how much less expensive it was.
So we rolled that out. Fifty Penguin servers.
DS: How did that go down with others in the company?
RS: It got enormous push-back within the UNIX community inside the company. Our UNIX sysadmins and such. I half expected that, but I had no idea how much vehemence would come out of it. "Why are you not running on a real operating system and on real hardware?" If it was Intel then it smelled like Microsoft to them.
DS: What did they prefer?
RS: HP/UX, Suns on SPARC. That kind of thing. Again, this was two years ago. That kind of attitude has changed. Anyway, we persisted. We pushed it out and put it in place. And it ran well. Within a few weeks all of the furor went away.
Then I went over to the guy who is responsible for building our compute ranches. We were rolling out a new product called RapidChip, which is a way to be able to design an ASIC.
DS: This is where customer engineers can come in and work on design workstations, customizing ASICs using your back ends.
RS: Right. This is a large collection of computers with load balancing software. When a job comes in it runs around looking for a processor in that collection that's not busy and loads the job there. That's literally what it does. It's primarily running tools from Synopsis. Plus a couple of tools from Cadence and others. My proposal to them was, I had read that Synopsis had a customer who had ported their tool to run on Red Hat Linux 7.1. So I went over to him and said "Hey, let's put a couple of these up. I've got about thirty PCs, regular PCs, laying around from the last RIF, and so let's just build a compute ranch with them, using Linux.
DS: What kind of PCs were they?
RS: They were reasonably good. Mostly Pentium IIIs. Dells. So I told him we could do this and it wouldn't take much time. There were a couple of folks who were actually pretty interested in doing that. So we built it, up there in Fort Collins. And it was incredible how well it worked. That was the catalyst. Today everything that goes into that engineering environment is all going into Linux.
DS: And that made believers out of these other guys too.
RS: Exactly. They don't call it a toy operating system running on toy hardware anymore. Now the first question anybody asks over there in the engineering department is, "Does it run on Linux?"
The other interesting thing that happened as a result of that was, after we put up the new database servers running on Linux, SQL Slammer, the SQL worm -- probably the fastest-spreading worm ever on the Net -- came through. We were well protected and didn't have any issues with that. So we used that as an opportunity to shoot seven or eight Microsoft SQL databases and put them all up on Oracle running on Linux. Now my boss, the CIO, reporting to the CFO, is in the same mode. Whenever somebody comes in and wants to talk to him about a new application, or a project or anything new, his question is, "Does it run on Linux?"
DS: So Linux is now defaulted as infrastructure in the company.
RS: Yeah. And that's by some design. We're trying to get folks not to specify, to me, the hardware or the operating system. Tell me what you want to do. If we can deliver it on Linux, that's how it's going to get delivered. And we're very strong with our vendors in that same strain. We've got folks who come in from Siebel, SAP... with their portal strategy, and we've told them deliberately that we're not going to proceed down that road until they can deliver a product us that runs on Linux. We're not going to run it on Microsoft.
DS: Would that still be their preference?
RS: Right now it is, yeah. They would like to get more divorced from Microsoft, though. The problem with Microsoft is that once you belly up to them and make your products run on their systems, they're pretty quick they move into your space. SAP about three years ago, in order to reduce some of their costs, made made very major moves into the Microsoft arena. So their first portal products were all Microsoft. Version 2 now is slightly less Microsoft-centric, but still the primary servers are all Microsoft.
DS: Is there a deliberate intention in LSI to move away from Microsoft?
RS: In the back office arena, yes. We've retired Exchange and all of our mail services now run on Linux. There are standard Linux, UNIX-type product there. But it's not there on the desktop. We had a project going through February of this year to put together a Linux desktop that an admin and a clerk would be able to use in the company. We have not been terribly successful there. We have three applications that we can't run on the Linux desktop, so we have to put up a terminal server for it to work. That's not the answer we want.
DS: Curious: What kind of laptop do you have?
RS: I'm working from home today, because I just ordered a 17" Mac OS X PowerBook, and it's going to be delivered this afternoon. One of the new big ones. It's ironic because when I came to work for LSI Logic back in 1995 one of my first tasks was for the the IT director in Japan, driving all the Apples from the company.
There are some bright spots for Linux on the desktop, though. I bought my 78-year old mother a computer from Dell running Windows 2000, and she always had problems on it, even though all she did was email and browsing. But my sister re-arranged her desktop, and my brother upgraded her Netscape, which she uses for email, and screwed that up. So when Red Hat 8.0 came out, I went over and re-imaged her machine and put Red Hat 8 on it because I knew my sister and brother couldn't do anything to screw up her machine. And she loves it, because everything works the same way all the time. And I like it because every time she has an issue, I can just SSH over there and fix it over her always-on Internet connection.
DS: Are you running any virtual Linux servers at the company?
RS: We run VMWare in the data center. I'm trying to get away from running a single application on a single server. We want to get more than one thing on a server. Back in the mainframe days that's what VM was all about. Multiple virtual machines all on one piece of hardware. VMware does that. So we've got probably half our Web servers are running on one piece of hardware under VMware. It's much easier from a cost standpoint. If somebody wants a new Web server, we can add it onto hardware that already exists.