How Software is Built

A blog forum to provide deep dive analysis and community conversations about software development models. For more details click here.
Tags:

Interviewers: Scott Swigart and Sean Campbell

Interviewee: Ross Gardler

In this interview we talk with Ross Gardler - Service Manager for OSS Watch. OSS Watch is an organization that helps institutions in the UK make decisions in and around open source topics. In specific, we talk about:


Scott Swigart: Thanks for taking the time to chat. If you don’t mind, could you give us a brief introduction?

Ross Gardler: Sure. I’m a computer scientist by background, but I’m in a management role now in an organization called OSS Watch, which is a publicly funded organization in the UK that provides an advisory service to the UK higher and further education institutions. We talk about things like “How would you engage with open source in procurement?” and “How do you develop open source in research projects?” Basically, any questions to do with open source come through to us.

We’re not an advocacy service–we don’t go out and say that open source is the best or only way. We try to enable people to make a fully informed choice.

My most active involvement with open source is within the Apache Software Foundation, where I’ve been a member since, I think, 2003.

I’ve done some dabbling in other things on the outside, but in my earlier life as a contractor, I found myself getting drawn more and more into the Apache way of doing things, and I’m fairly ingrained. These days I’m lucky enough to just do it for fun. I get paid for doing the consultancy and advisory stuff now, which is great. I can hack purely for fun again.

Scott: You’re probably in a good position, then, to talk about one of the first really eye-opening things we found in doing this project–that open source isn’t just one thing. Apache HTTD does things one way, the Linux kernel has its own culture, and MySQL is different still. What are some of the different common models you see for how the software is actually built or the philosophies that exist in different projects within the open source cosmos?

Ross: I think from the outside that they often look massively different. Take the two extremes, of say the benevolent dictator of the Linux kernel, where Linus theoretically has all the control over what happens in there, and compare that with something like the Apache Software Foundation, where it’s what they term a meritocracy and there are potentially a large number of people involved. I think there are currently 60 people involved in the Project Management Committee within the HTTPD project.

But once you get involved, they’re pretty much the same thing. Linus is there because the community wants him there, and they want him there because they respect him, he listens to their opinions, he works with them on patches that need to go in, etc.

At the end of the day, because the kernel could be forked at any point, if Linus didn’t represent the community properly, the kernel would be forked and the community would go off somewhere else. It might not be a meritocracy, but the community feels that they’re being listened to. So there are more similarities than perhaps there might first seem.

Sean Campbell: When we talked to the Apache Foundation, they talked about how they went for a commercially friendly license. But My SQL is known for unlocking the GPL and letting you embed My SQL with a commercial product. There is a lot of nuance around open-source licensing.

Ross: The real difference, at least for our target audience, is the type of community that builds up around the license. If you decide that you’re going to go with a GPL because that will enable you to dual license–if that’s going to be your exploitation route–it has a significant impact on the types of people that are going to contribute to your project, because it means you have to have full control over all copyrights. So any contributor has to sign over their copyright.

Some people simply won’t do that, especially for those of us in the academic community. Institutions like to hold onto their copyrights. The tech transfer groups within universities, who are traditionally responsible for exploiting advances from within their research groups, are used to dealing with patents.

That’s particularly true in the engineering sector and the sciences. When you go to them and you say that if they want to engage with an open-source project, they’re going to have to give away their copyright, that’s a no-go in many cases, straightaway. And that can have considerable impacts on collaboration between universities partners.

Sean: I saw that you have put up a couple of blog posts about the idea that just because someone puts a GPL bumper sticker on a project does not necessarily mean the project will just be embraced by the world.

Where do you think open source communities go right or wrong when they start to build a project?

Ross: I’m a great believer that the strength of open source is not in the license. The license really has come about to protect the passion and development methodology that has sprung up around the free and open source development methodology.

If you look at the history of free and open source software, you’ll find that we did collaborative development before the free and open source licenses existed. The licenses emerged to protect that development model as it became threatened by closed source approaches.

I think we have begun to lose sight of that over recent years. I shouldn’t really single MySQL out, because they are not the only ones doing dual licensing, but of course they are very much in the news at the moment.

They have used a GPL license and made the software available, and they have done an absolutely fantastic job of producing a really fantastic product and company. Nobody can question that.

But what they haven’t done is create an open development community. Their business model has been very much about closed development. It is true that people could get at the source. It’s not totally closed, and I don’t mean to imply that it is. But it is certainly less open than in something like Apache.

Again, I wouldn’t like to really single Sun out. I mean Sun is doing some fantastic stuff in the open source arena, but they also do some questionable stuff in some areas. Sun seem to have a certain need to control the software. Again, this is not necessarily wrong, if they have a business model that requires they control the software, I can understand that.

But let’s not pretend that’s an open development model. Open development is about saying, “I know this software can do something really useful for me. And it can almost certainly do something useful for other people as well. Let’s free it up and see where it goes.” Often you will end up with something that’s far better than you could possibly have come up with if you had retained full control.

However, this doesn’t mean that you have to give up all control. You know, you only need to give away partial control to people you have grown to respect and understand and see that they actually have good, genuine ideas that are complementary to your own.

Some companies seem to think that they are the only people who can be technology leads. In some projects, they are the only people who can innovate around the software. They slap an open-source license on it and they make the source code available, but if you try to engage with the development process of some projects, it’s next to impossible. In my view that is not a true open source project. Legally it is–there is no question about that. But the development model is different from what I have grown to love about Free and Open Source software.

Scott: On the Apache projects, you have to have maintainers who are from three different companies, so that no one company can ultimately have control over the project.

Sun, on the other hand, has taken heat around OpenLDAP, and some other projects, where their governance is written so that Sun employees have to be at the top.

And so if people leave Sun, they lose that position on the project, whereas in a lot of other projects it doesn’t really matter what company you happen to be working for. It doesn’t really affect your stature on the open source project and in the community.

It seems to me that community driven open source is really difficult for companies that produce proprietary software, because the mindset is so completely opposite whet they’re used to. Companies like Adobe are experimenting with taking certain products and moving them to an open-source license, and Microsoft is dabbling with putting some things under an open-source license, but it seems like the really the hard thing for a company to get past is this notion of control.

A lot of times it’s also hard for companies to recognize that if they spend money and are good stewards of the project, they don’t really have to worry a lot about somebody forking the code and taking it off in a completely different direction.

That’s really only going to be an issue if they go off the rails like Netscape did, and end up with Firefox emerging out of the ashes. But that only happened because Netscape was messing it up so badly, and it was really obvious to some developers what would make a much better browser.

Anybody could fork My SQL, but who’s going to build a company of 200 people to do the level of engineering that MySQL AB – now Sub – is are doing around it?

It seems like the sticking point for a lot of companies as they play around with open source is this fear of losing control.

Ross: I think you hit the nail on the head. It’s interesting that you brought up Firefox emerging out of Netscape (although I wouldn’t say it emerged out of the ashes, it was a business decision to try and turn the tide in the “browser wars”). I recently re-read the first essay in Open Source 2.0 (a book published by O’Reilly), which talks about the transition from Netscape through to Firefox. It was written at the time all that was happening, and the essay ends at the point that Firefox 1.0 is released.

The article talks about the difficulty that the Netscape management had in releasing control to the newly formed “.org”, it is an enlightening read. I recommend that anybody who is trying to go through any of these processes should read that chapter.

I think you’re also right that well managed projects don’t really need to worry about forking. There is a huge barrier to making a strong fork. I mean, you can fork the code, and it is definitely possible that a competitor could come along with deep pockets and throw money at it. But they don’t have the knowledge and expertise of the staff that you have working on a project. That’s where the real value lies, in the project team.

This is one thing that worries me about many Sun projects. There was an open letter to the OpenDS community and Sun Microsystems in Nov 2007 from Neil Wilson that described how Sun had essentially kicked all the developers off of the OpenDS project as a result of corporate restructuring (http://directorymanager.wordpress.com/2007/11/28/an-open-letter-to-the-opends-community-and-to-sun-microsystems/). This got me thinking, “Well how can they do that? This is supposed to be an open project, it should be the community who decide when it is necessary to change the project leadership. Sure, Sun have every right to decide where their employees spend their paid time, but should they also have the right to dictate what they do in their own time on voluntary open source projects [Neil reports that he and the other developers continued to work on the project as volunteers]? Furthermore, the people being kicked off the team are the ones that have got the true knowledge about that software and, more importantly, the community around the software.”

Again, I should stress that Sun are not unique in this kind of behavior and perhaps more importantly Sun have some really fantastic community led projects out there. Like all large companies it must be difficult for Sun to have a consistent approach to everything you do. They have room to improve, in my opinion, but then other companies have much more room for improvement than Sun.

The important lesson, I believe, is that people are the valuable thing. It’s not the software–it’s the people who create the software that are valuable. That’s why you see people buying open-source companies. They’re not buying the software–it’s available through an open source license. They are buying the expertise.

Scott: Right–Sun didn’t buy the MySQL code base, because I could run off and start doing my own thing with that code base tomorrow, and if they aren’t good stewards of the project, they may have ended up spending $1 billion on nothing. But they would have to mess it up for a very long time before someone would sink the resources into a strong fork, a fork that the user base would have confidence in and follow. The barrier to doing a fork like that is so high.

I think you’re right about what company acquires when they buy an open source company. When you see JBOSS get acquired. You see XenSource get acquired. People will say Citrix bought Xen. No. They bought the company that had a business model and was able to staff people on the Xen code base, and were able to move it forward. They bought the mindshare.

Ross: That’s it, exactly.

Scott: It seems like if a Fortune 500 company is deciding what it wants to acquire, Linux and Apache aren’t really for sale. They don’t have one main corporate sponsor that you can buy.

On the other hand, it seems like it’s really hard to get to the point where you’ve got a project as big as, say, the Apache Web Server or the Linux Kernel. And so the half step, which is really getting to be the norm, is these open source projects that have a primary corporate sponsor.

The Fortune 500 company is then making the buying decision on pretty much the same factors as they would if they were acquiring a closed source software company. They want to know the annual revenue, profitability, growth, market share, etc. of the software company. The fact that the company being acquired is wrapped around an open source code base is almost incidental.

If you look at something like Canonical and Ubuntu, it’s a fantastic distribution. They have done a lot of things to take Debian and make it more usable. And they have done a lot of great work.

But I don’t think it’s a profitable company. In a 2007 interview with Forbes, Mark Shuttleworth basically said, “Look, I put $15 million into it. I don’t know if it will ever turn a profit. I’m just doing it because it’s the right thing.”

And I picture a CIO for a Fortune 500 looking at that and saying, “Well, do I want to bet a piece of my infrastructure on some guy’s philanthropy?” So, what are your thoughts around that type of issue?

Ross: Again, I think your observations are absolutely right. One of the things we do is to advise institutions who procure software about how they can actually evaluate open-source software. And the kinds of issues that you have raised there are things that get raised within our community.

Really, the way we advise people on that is to ask questions like, “How much am I going to invest in this? How much is it worth to my organization?” And then say, “OK. So how much am I going to invest in the ongoing development? How much am I going to contribute to the ongoing sustainability of that project? And am I willing to give that money to the project?”

Normally they don’t contribute cash. They tend to contribute through involvement by their IT departments. But it may be cash. It might be paying contractors to work on it. It doesn’t matter how you get that resource to the project. There are loads of different ways of doing it.

The key thing is you have a hand in making sure that that software becomes sustainable. And the question then becomes, “Is the balance sheet right at the end of the day or do I trust a closed source company more?”

Scott: That’s a great point, because I think that, in the same way that a closed source company has a lot of trouble getting over the hump of saying, “We can’t give up control of this,” the same thing happens with a lot of corporations that adopt open-source. They have trouble getting over the hurdle of, “This is something that I really have to participate in the care and feeding of.”

I think a lot of companies take the attitude toward open source that it’s free, and they will just take what’s there. And they hope they will be able to continue to take the new versions as they come out, without supporting the overall effort in any way.

Do you find that perception in the corporate world of, “Well we can sort of take what we want from open source, but we don’t really have an obligation to give back,” or are you finding more companies becoming aware of the need to participate in it they want to be able to count on it?

Ross: I think people are beginning to understand it more, but there is still a huge number of people who think that open source software means it’s free as in beer and that they approach it in order to save money.

It can often cost you less to engage with open-source software, but we always try to encourage people to look at their budget and decide how they’re going to spend it. If after getting your open source software and your training, ongoing support and maintenance and all that kind of thing, you’ve got a few thousand pounds or a few tens of thousands or a few hundreds of thousands pounds left in your budget, then invest it in that sustainability. The sustainability of the product you are consuming is also part of your own sustainability. With open source you can choose to take the responsibility (with staff effort or cash), or you can sit back and hope for the best.

I think that message is being heard more and more now. In the academic sector, it’s slightly different, because budgets are very tight, and they are always tight. There is no sort of thing like increasing your profits. You just want to pay less money, and the government is giving you less money, and so on.

So there is more of a tendency within the academic community to think, “Well, we can save money here.”

Sean: For full disclosure here, I taught at Purdue for a couple of years–a pretty university in the Midwest–and I know you have to save a buck everywhere you can.

I imagine it has got to be an interesting dynamic to make the discussion point resonate when you say, “I know it’s free, but for this you really should go for the full-on support package or you should partner with Red Hat. Is that a pretty big uphill climb?

Ross: It is hard. I mean people have got to save money where they can, and if open source is going to save them money, that’s great.

But institutions in the UK, at least, have been rolling out open-source quite a lot recently. Over the last four to five years, our national survey has been showing an increase in open source adoption.

What we are finding now is that some people have been running open-source for a reasonable amount of time. Because they have got IT departments, they have been customizing that open source in-house. So they have actually been giving back to the software in some way, they’ve been developing it. It’s just that, in many cases, they have not been giving it back to the community; they’ve kept it in house.

We then go to them and say, “Since you have expended resources on customizing software to your needs, why don’t you just spend a little bit extra effort and resources to improve the project? So you are giving back within the confines of the budget that you have available. And the payback for you is that your upgrade path is easier and therefore cheaper.”

I think that message is beginning to become accepted a bit more now as people struggle with upgrading. We’ve got over 60% deployments of Noodle, the virtual learning environment, here in the UK. And people are finally beginning to realize that if they are going to customize locally and they want to upgrade, they have to engage with the community.

A lovely convincing argument is that staff–particularly techie staff–feel that they are doing something rewarding. “I’m getting peer review, I’m getting recognized for my contribution, et cetera.” They are happier staff, and we all know happier staff stay around longer (so there’s another saving).

There are other ways that you can contribute back. It doesn’t have to be something that appears in your budget line. It could just be a side effect of the way you engage with the software. For example, simply making people aware of a successful roll out helps the project remain sustainable as it attracts more users and potential contributors.

Sean: Broadly speaking, what do you think is more unique about the open source software environment in the UK as opposed to somewhere else? We are in Oregon, which is a hotbed of open-source. The state and local government has a really passionate interest in it compared to some other areas in the country.

Ross: I get the impression that companies engaging with open source in the UK do engage with the software because it’s part of their core product and they recognize the fact that they have to engage with the software and make it sustainable. So that’s great.

But what’s not happening here, that you can see happening in other places, particularly in organizations like the Open Solutions Alliance in the States, is they are not cooperating across company borders. So they are not working together to make the software solutions that they provide to their customers play happily with solutions that are being provided by other companies. They failing to form consortia to provide end to end solutions that the big players can do in-house.

We do have, here in the UK, the Open Source Consortium, which has over 80 members, and if they read this and they hear me say that companies aren’t collaborating, they’ll probably argue this point next time I see them. But, that’s the way it looks to me. It might not be true–it might be that they are collaborating behind the scenes, but they’re not letting people know about it. They’re not doing it in an open and collaborative way like the Open Solutions Alliance in the States do.

It would be great to see that kind of thing happening more, and that’s perhaps a little bit of the British reserve. I think there is a cultural thing that might have some impact on the way we engage with one another commercially.

Having said that, there are similarities in the types of people who get involved with open source, and I think they transcend the national boundaries a bit. You’ve got to be a bit of an egotist. You’ve got to have a thick skin and all these other things.

So I think to a certain extent, we are kind of a nation unto ourselves, those who work with open source.

Sean: That’s a great observation. Is there anything you feel like adding in closing?

Ross: Well, I’d like to reiterate the point about open source not being just a license. Just slapping a license on code will not give you the full benefits of open source. And if you are going to engage with it, then you have really got to understand the open development model, and understand that open source governance requires you to let go of ownership (but not necessarily full control). Otherwise, you are just doing a slightly different version of closed source development.

I’d love to see something like the open source initiative for open development. I’d like to see it somehow define what open development is. Of course, it’s a lot more difficult than defining what a legal thing is, because it’s much more fluid, but that’s something we need to keep an eye on.

We’ve got to stop open source being diluted with all these other sorts of things that are just loosely grounded in the open source ethos. We need to really focus on the benefits of open development, particularly in the academic community, where we don’t have the same exploitation routes in mind.

Sean: Ross, thanks for chatting.

Ross: Thank you, it’s been great.

Posted by campsean on Tuesday, March 25th, 2008


You can follow any responses to this entry through the magic of "RSS 2.0" and leave a trackback from your own site.

Post A Comment