In this interview, Scott Swigart, Sean Campbell, and Richard Bowler interview Stormy Peters of Open Logic. Stormy is a frequent keynote speaker on business aspects of open source software at major conferences such as the Open Source Business Conference and the O’Reilly conferences. She has addressed the United Nations, European Union and various U.S. state governments on open source software. Stormy is a co-founder of the non-profit GNOME Foundation, which is based on open source principles to encourage the development of a computing platform, comprised of free software, for use by the general public.
Stormy has impressive experience in the field, and this interview covers a wide range of topics, including:
- When does it makes sense to open-source a given project
- Considerations for open-sourcing a product
- How open source software is supported
- How open source is and isn’t good at fixing bugs
- Choosing between using open or closed source
- How open source software is actually built
Why open source projects don’t really fragment (fork)
- The pain of incompatible packages
- And, what OpenLogic does to take pain points away
Sean Campbell: Stormy, tell us a little bit about yourself, what you do, and your relationship with open source.
Stormy Peters: I actually started out in open source software when I was managing a desktop group at HP. We had HP-UX, which has been around for a long time, and it had CDE on it, which was a proprietary desktop. I had a very large team in
At that time, Linux was becoming really big, and they had two open source desktops. I thought, "Well, what if we just take one of those desktops and put it on HP-UX and then people will get this great desktop with all the features that they want. You will be able to have OpenOffice on it, and we’ll have all these other people out in the community helping us fix bugs, as well as my team, and so I looked at porting the desktops, and that was really easy, technically.
So my job soon included explaining open source internally, and pretty soon I had created my own job of helping people understand open source software and figuring out how it could benefit them or how they could use it successfully, or when they might not want to use it. And so I created the Open Source Program Office at HP, and the Review Board and the policy, and it kind of grew into a new job that was really a lot of fun.
Later, I moved to OpenLogic, and what we do is help companies — enterprises — use open source software.
Scott Swigart: When you are dealing with a company, or a company is looking at producing a new product, and they’re thinking about whether they should produce a product as open source or closed source, it seems like there’s a natural tendency to choose closed source by default as "the safe way" to do it. But what do you think the real decision points are about whether to go open source or not with a product?
Stormy: There’s a whole list to consider, but the first consideration should be whether the piece of software really provides a competitive advantage, or whether it’s something that pretty much everybody in the industry has to write. For example, if you need an application to manage your customers, are you really writing a new and better one, are there a whole bunch of other ones out there, or are you writing something that everybody is going to need and so everybody has written in house, even if no one has actually published the software?
I think that if you look at banks, for instance, there is probably software that they all have inside, that no software vendor has published yet, and so they all had to develop it in house. If you have one of those, then open source would definitely be the way to go, since there’s no reason to be writing and maintaining it yourself.
The other thing I think you need to look at is the model of starting closed source and then open sourcing it later. I don’t think that works well very often, because people that work on open source software work on it because it’s fun, and so if you wait to release something that already has all the cool features in it and just needs bug fixes or some fixing up, it’s going to be a lot harder to get a community started around it.
Scott: Two recent examples that spring to mind are Adobe moving the Flex stuff to open source, and Second Life open sourcing the client. Do you have any thoughts about those, based on what you just said about it not always working out quite the way the company thought it would?
Stormy: I think both of those, especially Second Life, have a huge advantage in that they have a very large, interested user base that’s very technical and can help code. I think Second Life will work out just fine as far as when they open sourced the product. They have a very large user base that can also be a developer base that’s very engaged and very interested in working on it.
Sean: In a recent blog entry, you said you saw the future of development as people building their own tools when they need them — kind of a "just in time" methodology. How far do you see that extending, or what do you see as the vision of that if open source was to really continue on a march against closed source?
At some point, you have a group of people who can’t code. Do you see the toolsets just becoming richer and more available and so people can build things anyway?
Stormy: Exactly. Someday the toolsets will be so rich that you won’t even know that you’re coding. In fact, I think this already happens today — look at databases. You have administrators that use databases, and they’re in a sense coding every time they write a new report. The user interface and the tools they have are sophisticated enough that they don’t actually have to write lines of code to generate that report. I think someday that will be true of a lot more applications, and a lot more things in our life.
Richard Bowler: Working with open source, are you working with ongoing projects as well, or are you more just an evangelist and an open source consumer?
Stormy: At OpenLogic, we have a unique way of doing support, in that we support hundreds of open source software products, 24/7, for enterprise customers. In order to support that, we could have tried to hire a person per product, but that obviously would have an enormous amount of overhead, even if we could get the experts on the OpenLogic payroll.
Instead, we set up what we call our "Expert Community," which I manage, where we have written agreements with members of the open source community who help us out with support calls for pay. They fix the problem and they check it into the source code tree. I work directly with a lot of the committers and contributors. I don’t actually write any code at the moment though.
Scott: That feeds into one of the apparent myths — that closed source companies give you support, but if you do open source, you’re on your own. Let’s say that I have a problem with an open source product. What are the guarantees that you can ultimately solve my problem one way or the other?
Stormy: I think the problem with open source and support is that there are too many choices. Say you buy a proprietary product from Oracle. It’s very clear how you’re going to get support from Oracle for that product once you buy it, and you only have that one option. You can’t do it yourself, and you can’t go to any other company, so it’s very clear cut.
For open source software, though, if you use Apache or MySQL or Postgres, it’s not so clear what to do for support. Maybe you go to OpenLogic for support, maybe IBM for Apache support, maybe you can do it yourself, or maybe you have two guys in your IT team who have been working with it a long time and can help you out.
In the open source world, there are many more choices than people are used to, and so it’s harder to evaluate. There’s no really easy "sign on the dotted line and we’ll take care of you" type of option for most projects.
Scott: We wrote a blog post a while back that basically said, "Whether you choose open source or closed source, there are three ways to get support: you can pay for it, you can pay for it, or you can pay for it."
You can buy software from Microsoft, or Oracle, or Adobe, or whoever, and you can call them for support. In that case you paid for support by paying for the product. In the case of open source, you can pay a company like OpenLogic or IBM for support. Option two is to hire smart people who can figure things out. There are lots of Windows and Linux experts out there, so just hire one and pay the big bucks. Option three is to bring in a consultant when you’re stuck. Again, there are consultants on every closed and open source product. The support options don’t really seem all that different.
Stormy: I think open source is a mixed bag. People seem to think that the mailing list fixes everything unique and you can always get your answer there. I think sometimes you can. You have people out there who are willing to go through tremendous fits to solve your problem and they don’t even know who you are.
It’s not really unique to open source. I had a friend who had a problem with his TiVo or his DVD player and he posted his problem. Someone actually went and got their video camera and showed him on a video how to fix it and posted it to the web for him.
Sean: Do you think there are things we should have added or subtracted from that theory — "you pay for it, or you pay for it, or you pay for it"? And as a follow up to that, what type of support questions, broadly speaking, do you think the community at large is more active to help with? What categories of things do you think are not really well documented or tackled?
Stormy: I think people are really bad at evaluating what different resources cost, so I think it’s a hard trade off for them to decide whether it’s better to pay a company $100,000 or better to hire somebody. Even when you can try to boil it down to numbers, I think people have a hard time trading off the different types of support and trading off the different types of resources.
Second, it is true that with open source and the mailing lists, you do have to pay for support too, but there’s a lot more sharing so the costs are shared. If I had a hard time installing PHP with Apache on a host drive, there are people out there who get so frustrated they actually blog about how they did it just so that somebody else can find it and benefit from their experience. You see that less in the proprietary world because people usually have a contract that says they are not allowed to do that. So there is sharing of the costs to some extent.
As far what open source is good for supporting, and what it’s not, we find that open source is really good for fixing bugs — the bugs get fixed. One of the problems for enterprise customers is that they get fixed only in the latest version, and there’s not a lot of back-porting. The community just tells you to move to the latest version, and often it’s really hard for enterprises just to move whatever they are doing and change out a production environment to the latest version of whatever package it happens to be.
But the community isn’t always so easy to find information about how to integrate things or how to install things, especially if you’re talking about large clusters or large production environments. That’s because the users and developers don’t typically have a hundred systems they are trying to install a MySQL cluster on. Information about how to get that up and going isn’t always on the web, and there isn’t always someone who can answer your question when you post it.
Scott: Would you say that there is some validity to the stereotype that in the open source world, people like to write code, but they don’t really like to write documentation?
Stormy: I have met very few people in this world that like to write documentation, and they’re in high demand — the really skilled technical writers. I do think that a lot of people out there in the community are more than willing to type up an answer in an email, but the information that is out there is just not always in a nice quick start guide.
Scott: If you’re building something that’s just common infrastructure, why would you build it proprietary? Why wouldn’t you do it as open source? Consider SugarCRM, for example.
If somebody decides that they are going to release a new product, and they are going to do it as an open source product, what do you think are some of the things that would be advantages and disadvantages of going that route?
Stormy: The first thing if you want to go the open source route, is always to find the community you think is going to work with you. Maybe you don’t have to find everybody, but go out there and talk to a few people and figure out whether other people think this is a cool idea. Open sourcing something is not cost free. It costs money to put it out there; to have somebody answer the emails; if it’s an older product, to clean it up, to modulize it, and to add some comments or let people know how to set it up. The first thing is to go out and talk to some people, or present at a conference and see if there’s much interest around the project you’re starting.
Maybe you’re using an infrastructure product and the other banks are also using the product, but none of them are interested in working on it with you. This might end up being a dead open source product. If you don’t have a community, you don’t really have an open source project. It might be under an open source license, but it’s not really an open source project.
Scott: Assuming that the project does gain critical mass and take off, what are some of the big advantages of going that route?
Stormy: You get a lot of end user feedback. You might get additional development help. MySQL gets a huge amount of testing done by a huge community of users of open source software project versions.
Scott: For organizations that are looking at buying a proprietary system or going with an open source system that seems to do the same thing, what do you think are things that an enterprise should look at when they are deciding between closed source and open source? What are some of the advantages and disadvantages that they can expect, in general, with an open source product?
Stormy: I think they need to go through the same procurement process they go through when they evaluate two proprietary or closed source pieces of software. They need to understand how they are going to get support, how often versions come out, and how big the community is around it, just like they would evaluate how big a company is that supports proprietary software.
Scott: So you don’t necessarily see one as having an automatic advantage over the other?
Stormy: I think the procurement or evaluation process might have a few additional steps or a few additional options to some of the answers so that open source software can be weighed equally with proprietary software. But I think they need to evaluate it right along with the proprietary versions.
Scott: One trend is that it seems like a lot of government entities are going pretty strongly in an open source direction. Are you seeing that trend, and if so, what do you think is some of the motivation behind it?
Stormy: I think you see it in the States. It started with the Department of Energy and the Department of Defense. But I think it’s easier to look at what happened in Europe and Africa, because they are really big promoters of open source software and moving very strongly in that direction.
And I think they did it for three reasons. One, most governments, ours included, have a clause that says if it’s created by the government, for the people, then it should be available to the people; like if the US government writes software, that software is freeware.
Not if they paid another company to develop it, but if government employees wrote it, it’s freeware. So I think there’s a little bit of that behind it. If the government’s spending money on something, it should be available to the people.
I think there’s also a very strong wish not to be dependent on a single foreign entity. And then you see the cost savings, especially in places like
Scott: How much of it do you think is a cost issue? That seems likely for governments, but for a lot of enterprises, the number one reason that they cite for choosing open source is often not cost. On the other end of the spectrum, for the three person startup that’s trying to bootstrap itself from nothing, not paying a lot of server licensing costs and things like that seems like it might be really important to them getting their company off the ground.
Stormy: I think you do save costs by moving to open source software. I don’t think it’s always as much as people seem to think because there are costs associated with using software, period, no matter how much you paid for it up front.
But I think one of the cost savings that you see is that, because it’s open source, more than one vendor can pick it up and support it. Microsoft pretty much controls the price of Microsoft Windows. There’s not a whole lot of people you can buy Microsoft Windows from that didn’t get it from Microsoft, whereas if you want to get Linux, you can get Linux from Novell or from Red Hat or from Ubuntu, and you pay different prices because they can all source it for free.
Richard: I’d like to talk about differences in development models. It seems that closed source development is a lot more process bound, as in a lot more formal processes for moving from step to step in the SDLC and whatnot. And that is pretty much lacking in the open source world.
For instance, we talked to a guy that works on the Linux kernel. We were talking about security for the code and he said, "Well, we don’t really have any process to make the code secure. We just get the many eyeballs looking at it, and that fixes it." What do you think are the advantages of closed source development models and the advantages of open source?
Stormy: I think it’s really interesting that you are looking at how the development models are different and how that changes what people do, because there have been a lot of studies on open source software, but I don’t think anybody’s really looked at the development models and truly understood how they both work when they’re so different.
Open source has created a different culture and so it’s created a whole different development process, and that’s what makes open source as a whole different. In open source software, they don’t hire managers who are really good at managing; technical people just pick up various responsibilities. There aren’t regular weekly phone calls — a lot of that process has gotten left behind and has been replaced by other tools.
So for example, when decisions get made in an open source software project, they get made over a mailing list that anybody in the world can subscribe to. So if you want to understand how things are working on Apache Derby, you can just sign up for the Apache Derby project mailing list, and you can see how they’re making decisions and what goes into the next version. You can see how they argue out how to do something, and you can see the process for resolving that.
I think they’ve replaced meetings and different documentation, design reviews and stuff, with things like mailing lists and chat rooms. It’s really kind of interesting to see the different ways of getting the same thing done.
Scott: Talk about that a little bit, in terms of educating us about how that works — there are certain people who are sort of gatekeepers, who have to approve code that gets checked in, right? How does it work, where there are people who decide what features a release should have, who’s actually going to work on it, and whether the thing that somebody did is good enough to get checked in? How does that play out?
Stormy: It really depends on the project. There are probably three main models, and it depends on the project, but the key to all of them is that there are some people that can check code into the source tree, and those are the people that are the gatekeepers for what goes into the project and what doesn’t. I think the key to the model is that everything is done on the mailing list.
Sean: If somebody’s failing, once they’ve been given the keys, how do you take the keys away? Not just the mechanics of it, but how do go through the process of demoting somebody, so to speak?
Stormy: I don’t have any specific examples in mind, but it would really depend on the project’s structure. There are projects like the Linux kernel, where Linus Torvalds runs it, and he could pretty much remove somebody if he wanted to, just like a manager could, or a company could fire somebody.
Then there are projects like the Apache Foundation, where everyone that has commit access is a member of the Apache Foundation. Last time I counted, it was 600-something people, and they elect their board of directors every year. In that case, it would be much more of a democracy, and the Apache Foundation would probably actually vote on whether this person should be removed. I don’t know that for sure, but it would be that model.
Scott: How often does it happen that somebody just goes off and writes a feature and tries to get it in because they think it’s cool, versus the case where discussions in the mailing list have decided that a feature would be cool, if only somebody would go and write it? In other words, how does it go from an idea of a cool feature, to somebody actually coding it?
Stormy: The idea doesn’t usually get proposed, until that person’s willing to write it. It’s usually kind of assumed that if you brought up the idea, you’re willing to write it.
Scott: It also seems that there’s a certain amount of risk that you might spend a lot of time coding something, and then have it get turned down.
Stormy: That’s why we would have the discussion in the mailing list. If I had a really cool idea, and I was willing to write it, I would probably get on the mailing list for whatever project it was and say, "Hey, have you guys thought about this? I think this would be really cool because XYZ." And a bunch of people would jump in one way or the other, and that would kind of be the design process.
So, at the time that was done, I would know if I had people’s buy in or not. We would probably have argued a few things through, and so when I went to write the code, I would pretty much know I had everyone’s sign-off on the design.
Scott: It sounds like there’s a certain amount of politicking that happens, right? You just can’t be good at writing code, you also have to be able to be persuasive too?
Stormy: Agreed — and when people want to get started in open source, I suggest they watch mailing lists or read some of the archives, and see if that’s a project where they could jump in. Some of the cultures, like the Apache Derby project, are very nice. If you want to get started, they’re going to be very nice and helpful to you, and the person that runs those projects and moderates the mailing lists jumps on people that are rude, and tells them to take it off.
Then there are other projects where the way you get heard, and the way your ideas get listened to, is by screaming the loudest. So not only is politics involved, but there are different cultures associated with different projects. When you join the project, you might want to see if the culture is something you can handle and that you can be effective in.
Scott: One other issue is what happens when some person or group really wants or needs a feature that the majority of the community isn’t particularly interested in, or the smaller group doesn’t like the direction the community is going with it. For example, someone might want some kind of real time feature that is only going to apply to a small percentage of people that use the product.
It seems that sometimes what people do is just to fork it — grab the source and take it in another direction. Is that accurate — does that happen?
Stormy: I think there are very few cases of a real fork where like someone took the project, and started another open source project based on it. I think you might find a few more forks if you looked at cases where companies are involved, but I think one of the ways open source handles that is that open source tends to be much more modular than proprietary projects. That’s a good thing when it comes to cases where you need a feature and nobody else wants it. You may be able to make it a separate project that plugs in.
On the flip side, one of the reasons that OpenLogic exists is that, when you have all these little modular projects that may work together, but nobody bothered to write the "how-to" guide or document all the integration points, that’s a weakness.
Scott: It seems like for an open source project to survive in the wild, it has to be modular by design. The ones that get built that aren’t extensible often don’t seem to make it very far. That seems to cause some problems, though — you may find a module that does exactly what you want, but it works on the version of the product that’s two releases ago. Maybe it wasn’t updated when the main product updated, and the creator isn’t working on that module anymore.
For example, you see that with WordPress all the time — there are a million WordPress blog plugins, but who knows whether or not they work with your version of WordPress?
Scott: How big of an issue is that in the open source world? I think there’s a somewhat widespread perception that incompatibility between versions is a significant issue. Because in open source, everything’s built a little bit independently, but one of the advantages is that nobody just runs off and builds their own database, right?
How much are you seeing compatibility as being an issue, and is that getting worse? Are people like the Linux Foundation addressing that?
Stormy: It’s definitely an issue that we solve through our knowledge base. When someone selects something, we can tell them what will work together, and we find that to be good enough right now.
The problem was just not knowing which version you needed, or not knowing that this version wouldn’t work with that, or getting it installed and then it wouldn’t work. So we manage the dependencies, and manage the versions for customers.
As to whether it’s getting worse or better, I haven’t really been tracking that.
Sean: I think it’s important just for context too, for us to understand how you guys monetize your efforts around open source. You’ve touched on it a couple different places, but I’d like to get that a bit more explicitly.
I’d also like to circle back to the differences in the software development methodologies. You brought up transparency and a variety of things, but if you had to stand back and identify the most important differences, what would they be?
Stormy: To me, I think the main one would be that, in open source, the users and developers are so close together. And they’ve managed to stay so close together, even though they’ve moved away from products that the developers weren’t typically interested in.
I think that the hardest part about software — open source or closed source — is how to make sure the requirements of the users get into the product. That’s always been one of the product manager’s hardest roles. I think with open source software that they’re really close together, because the users have direct access, not only to the developers and the mailing list, but to see the plans, and to see where they’re going, and to put in their two cents.
I think that’s why, and it started out even much closer. Because the people who were writing open source software were writing it for themselves, and to a large degree, they still do. So, the developer is the user.
So the software gets developed with just the features that the users need. It doesn’t have a whole bunch of other fancy things that the company thought could sell. It doesn’t have a bunch of things that were misinterpreted, where the developer thought the user said they wanted one thing, so they developed it, and it turns out what they really wanted was something different.
I also think the transparency of the whole communication model is just really exciting. I don’t think Web 2.0 is the right term, but it’s so much the way of the future.
When you look at our Expert Community at OpenLogic, half the people don’t live in the
That communication model that’s very open, very transparent, is one of the most exciting things to me. It’s actually frustrating to go back to the business world, and that document got printed and handed out in the meeting, but nobody emailed it around, and Fred wasn’t at the meeting and so he didn’t see it. You can’t help but notice that if they just did it a little more the open source way, that would never have happened.
Richard: Great. To follow up on that question about how you monetize your efforts around open source, how does that work, in a nutshell?
Stormy: OpenLogic is about enabling enterprises to effectively use open source software, making it safe, easy and cost effective for them to use open source software where it makes sense. And so we have a library of hundreds of open source software products that we’ve certified, tested, and selected as being good for the enterprise. We deliver those to the enterprise with a knowledgebase on what each one does, what its dependencies are, and how it works with everything else.
Our customers can select the ones they want and install, and it just happens, when all the dependencies get selected. Then we wrap a bunch of other services around that. We have a whole set of services around governance, so you can set policies, and you can say, "I don’t want anybody in my company installing any GPL software because I’m really afraid we’ll copyleft the product we’re shipping." You could say, "Only such and such team can install this piece of software."
We also have auditing and discovery tools, so you can actually make sure what’s happening. You can run the discovery tool and find out all the open source software that’s on this developer’s box or your whole IT environment, all the systems.
We have audit and tracking of whatever gets installed through our tool. You can run reports on all of the open source software licenses that you’re using, all the products you’re using, or how many systems it’s being used on. The attorneys can review the licenses to really help them with all that and managing it within your organization.
And then we have help with the legal side; we offer full indemnification for all the open source products that we ship, and we offer support for all of them. In short, we’re trying to resolve barriers to entry for companies that want to use open source software.
Sean: Let’s go jump way out there.
Imagine a world where OpenOffice is on 90 percent of the desktops and Office 2007 isn’t — or Office 2010, or whatever it would be at the time. How would a mainstream, non-technical user who can’t forcibly make their case on a mailing list say, "I really need this feature," particularly since they’re not the person who’s going to write it?
Where do you think these projects go when they get a little farther out of the domain where they’re coupled to a user base that’s pretty technical and really wants to extend it, when maybe they’ve taken another step and the product is still open source, but 90 percent of the people interacting with it can’t code, can’t really articulate a technical viewpoint as to why they want something in a product?
And maybe 80 percent of that 90 really want it. How would the process of pulling in that type of requirement work?
Stormy: I think there will be a lot of ways that it will happen — a lot of ways that it does happen. Some 10 percent of the users are going to be those technical folks, and so they’ll be the evangelists for the whole group. Then all those people are part of this community. I’ve actually seen several projects like that. There was a project that reads the text on the screen aloud, where someone wrote an open source project to do it for his daughter who couldn’t see.
All these people that are using this open source software have friends or husbands or daughters that are in the technical community too. It’s not like all the technical people are sitting to one side and not involved with the rest of the world.
Especially with the open source software field, it’s a meritocracy. Everyone knows who wrote what, and you can say, "I wrote that piece of software."
And so I think there’s a very strong sense of pride with that piece of software, so if people are complaining that when they print their report, it’s always green instead of red, there’s a strong motivation to help fix that and make people want to use it. I think that’s lost just a little bit with companies, especially these days, when you see people moving around so much.
Scott: That’s an interesting point, because one question is how does my mom get a feature into Ubuntu or OpenOffice? And the answer is that she doesn’t. She doesn’t even necessarily know the features she wants. I get it in there because I’m tired of answering the question or fixing the problem for her, so I go off and write it.
It seems like the place where open source has really knocked it out of the park in the past is things that have been written by developers, for developers. It sounds like you’re definitely seeing open source moving more and more toward products by developers, for mere mortals.
Stormy: Yeah. And the developers are mere mortals, too, right? So their paths cross.