How Software is Built » quality

Interview with Mark Osborne, Developer Division Architect, Microsoft

scottswigart — Tue, 04 Mar 2008 20:05:47 +0000

Interviewers: Scott Swigart and Sean Campbell

In this interview, Scott Swigart and Sean Campbell talk with Mark Osborne. Mark is an architect in Microsoft’s developer division, and focuses on the productivity of developers building Microsoft’s products.

Scott: Mark, thanks for taking the time to chat. Tell us a little about your role at Microsoft.

Mark Osborne: Sure. As an architect in Microsoft’s developer division, I’m responsible for the productivity of our developers. Basically, I think about what we can do at a systems and process level to make sure that the division runs smoothly and can get quality releases out on time.

Scott: Let me drill into that a little bit. When I talk to people at Microsoft, I get the impression that there’s an incredible amount of effort that goes into pretty much every line of code. Code gets written by a developer, it gets reviewed, and then there are all sorts of static analysis tools that get run over it, you have to make sure it doesn’t break the build, bugs are posted against it, and the developers have to be thinking about all kind of things simultaneously.

They have to consider performance. They have to make sure there aren’t going to be localization issues with code. They have to follow security best practices, coding standards, and all sorts of things. Being a developer, and knowing the kind of code I write, I think, “Man, it must be really daunting to come on board and write your first line of code at Microsoft.”

Mark: I think there is a lot of stuff a developer has to think about. You mentioned pretty much everything that you have to think about when you’re actually writing code. We have lots of training courses and lots of best practices available within the company. The developers use those as resources so they know how to write great code. And of course we have lots of people with years of experience that know what to do and what not to do.

But at the end of the day, there’s this kind of tension between needing to get product out the door, and all the stuff that you have to do in order to produce a quality product. So at a divisional level, we’re really looking at what can we do to put a process in place that basically helps developers do the right thing.

This way when schedule pressure is applied to them, for example, they have a mechanism to push back and say, “No, actually I need to get the performance right. I need to get the security right. We’re going to have to cut features or we’re going to have to slip the release, or whatever, in order to be able to release with quality.”

Scott: One thing that seems unique to Microsoft, and unique to the developer division within Microsoft, is that you’ve got people building the technology that they then use to build the technologies, right?

Mark: Yep.

Scott: People are working on the .Net Framework, they’re using Visual Studio, but they’re building the next version of the .Net Framework and the next Visual Studio, which they can’t necessarily run as their development tool early in the process, but probably want to switch over to later in the process. How does that work if you’re building the foundation you’re standing on, to some degree?

Mark: [laughs] Yeah, we have to build a compiler to build the compiler. We have that kind of problem. We have a complicated build process because of that, but as much as possible we try and dog food the environment that we’re building. So we try and build the tools with the most current version of the tools.

We have a process called LKG, which basically stands for Last Known Good. Which means that the stuff that we use to compile everything with, and all of the tools that we use, we recompile those about once a month. This keeps us reasonably up to date, but also keeps the division from being shut down by a problem in the absolute latest build.

But generally, we’re pretty proactive about dog fooding our own stuff.

It depends which part of the product you’re working on. Some teams may need a second version of something. For example, the debugger team might have a second version of the debugger. If you break the debugger because you’re working on it, you need another version you can actually debug with. So there are a few teams that fall into that situation and need a safe version. But generally, people just use close to the latest stuff.

Scott: How do you guys know that you have a good version? My understanding is there’s a build verification test that runs nightly on the stuff that got checked in, but then there are much more in-depth tests that the product goes through, and it either gets labeled a broken product as a result of those tests, or considered to be a good build.

Talk a little bit about how long it is between the time that a piece of code gets checked in, and knowing whether you’ve got a good build that you can roll people forward to.

Mark: Yeah. We do have this concept of being able to get through the build clean without any build breaks, and then on top of that being able to get through a set of build verification tests. And those are intended to be a representative set of tests that cover most of the core functionality on the product, so that that build should be usable by anyone in any one of the product units in order to move forward.

Scott: Right.

Mark: So stuff at the periphery could be broken for a certain team, but you’re not going to take down a large amount of the division. We do a lot of work at the beginning of the product cycle to make sure the build verification tests are solid.

One of the big changes that we made at the beginning of the 2008 cycle was that we changed the way that we check in code. We used to have everyone check into a set of branches, and then those branches were regularly integrated into the main branch. What that essentially meant is you have in-flight code from a couple of thousand developers across the division all being squished together in main.

Even with build verification tests, trying to keep that stable was very, very difficult. It was just too much stuff moving at once. For Visual Studio 2008, we adopted a concept of feature complete and feature crews. Features are now developed in isolated feature branches. A team works to develop that feature, to test that feature, and there’s a set of about 15 quality gates that they have to go through before they can say, “This feature is complete.”

And only when that feature is complete can they actually check it into the product for integration into main. And in that way the mainline branches that we have for the product actually stay very stable, because what we have going in is only completed, well-tested pieces of functionality.

Scott: As a user on the outside, how would I perceive that change? When Microsoft was building Visual Studio 2005, you guys might have popped out a community tech preview, a CTP, every six weeks or so. And sometimes the CTP felt like it was very broken.

Mark: Right.

Scott: I think that’s because of what you said, right? It was a snapshot of what had been checked in. I remember people in Microsoft’s developer division messaging to the rest of the world, “OK. You’re going to notice a change with the 2008 product where the CTPs are going to be a lot more stable, but a feature that you maybe heard about, or are interested in, is just simply not in the CTP because it just hasn’t gone through all those quality gates yet.”

Mark: Yeah.

Scott: So do I have it? Is that an accurate way to describe how it works?

Mark: Yeah. If you work back from the processes that we put in place, and the reasons that we put those processes in place, one of our goals was to keep the main branch customer-ready. The idea was that the main branch would be customer-ready at all times. That way we could just snap a CTP, and release it to customers, and the customers could have a good experience.

Scott: Yeah.

Mark: What was there in a previous CTP should now not be broken, and the new features should come together in pieces.

Scott: Right.

Mark: You may not have an entire end-to-end scenario early on, but the functionality for a scenario would slowly come together in stable chunks over time.

Scott: I know that after 2005 shipped, Soma blogged about something called MQ, or Milestone Quality. It seemed like that’s where a lot of the reworking of the process happened.

Talk about how the process for building code in developer division changes. Because obviously you can’t just be changing the process on 2000 developers willy-nilly, or nobody could track with the current, correct way to do their job. So how do you go about changing the process itself, in a way that everybody can keep up with?

Mark: Yeah, you’re right. We went into MQ and we looked at what we’d done before, what had gone well, what had not gone so well, and we came out with three main goals. One is we wanted to be more predictable. Basically, being able to say what we’re going to do, and then actually go do it.

We wanted to be more efficient. We wanted to be able to push more work upstream and not rework stuff over and over again because we worked on quality too late.

Finally, we wanted to be more agile. We wanted to be able to be ready to ship when the time was right and not be held hostage by having hundreds of thousands of lines of code that needed stabilizing before we could ship.

So those were the main things that we were trying to achieve by the end of MQ. We went to the division and worked with the division to say, “Here’s what we’re trying to achieve. What changes would we have to put in place in order to be able to achieve that?”

Three things came out of that. The first is, don’t defer work. A lot of our problems with agility and predictability were because work was being deferred. We had an incredibly long bug tail. A lot of work was happening after we were supposedly code complete.

Scott: Right.

Mark: And then there might be a year of bug fixing and DCRs and that kind of thing. And another thing that we want to do, as I said already, is keep the main-branch customer ready. And we wanted to increase our investment in automation.

What you don’t do is dream all this up in an ivory tower and then try and dump it on everyone.

Scott: [laughs]

Mark: So we had a very inclusive process where we worked with the managers throughout the entire division. We came with some ideas and we worked with the management to refine those ideas. We also did benchmarking around Microsoft; we looked at what was happening elsewhere in Microsoft and brought some of those ideas in.

And we came up with three things. One was the way that we planned the products. We wanted less of a peanut butter approach, which is basically adding a little bit of functionality across the whole surface area. We wanted an intentional planning process where we decided what the value propositions for the customers would be, what those experiences would be, and we then broke those down into features.

I’ve talked about being feature complete before you check in, so we defined a set of quality gates, which is really a way of describing what it means to be complete. The feature crew concepts, we actually took from the Office product group, and we adapted it. The quality gate concept was something that Windows had started with and we adapted that. We merged those two together and added in some ideas of our own and then worked with the division to roll that out. We did a lot of pilots during MQ to refine that as well.

Scott: So there are two pieces of terminology to drill into. Define for me what a feature crew is, and also define or give some examples of actual quality gates that stuff has to go through.

Mark: Yeah, sure. A feature crew is essentially a small interdisciplinary team that is assembled to write a feature and to push that feature to a completed stage. It would normally be a program manager, a set of testers—maybe five testers—and a set of developers—maybe five developers. They work together from somewhere between three weeks and ten weeks to design, implement, test, and move the feature through all the quality gates.

The next question is, what is a feature? And for us a feature is…well, ideally it’s something that you can present to the customer; it’s a customer experience. But in reality, it’s an independently testable piece of work, because in some cases, when you start breaking things down, you may have a piece of infrastructure that is required by customer-facing features. That piece of infrastructure work itself is large enough that you might develop that as an independent feature.

A lot of work goes in to the planning stage, and breaking down what you want to present to the customer, and how to arrange that as a set of features, and what the dependencies between those features are.

And then in terms of quality gates, we’ve basically got simple things like having the functional specification complete, which is basically defining what the feature is going to do; having a design so developers know what the architecture is going to look like—how they are going to design that feature, what the components are going to be; having a test plan and defining how that feature is going to be tested; and having a threat model—basically understanding what the security implications of that feature are and ensuring that it is secure.

And then we have things like being test-complete, which is basically that all your tests—all the ones that you have designed—have actually passed against the feature. Test-complete also says that your automated tests are hitting at least 70 percent of the code. And then we have ones around static analysis, pre-fast, FXCop, and a number of other ones as well.

Scott: So now you’ve gone through the new process from end to end. When 2005 shipped, you reflected on the process you had up to that point, kept things that worked, changed things in processes where you felt like things could be improved, and have now spent years building a new product using this new process.

Where are you at now? Are you back to that period of reflection and figuring out how to evolve the process again or…?

Mark: Actually, this time, we’re actually very happy with the results. With 2008, we achieved the goals of predictability, efficiency, and agility that we wanted. There’s still room for improvement, but as far as the general process is concerned, we’re happy with the results. Our delta between when we wanted to ship and when we actually shipped is probably an order of magnitude better than it was for Visual Studio 2005.

This time we’re looking at keeping the same processes in place and just tweaking them and refining them and basically baking that into the culture of the division so they become the normal way of doing things.

Scott: It sounds to me like it’s a lot easier to—not to oversimplify it—to basically run a report and figure out how far through your development process you actually are. If you’re saying you know how many features you’re slated to build—and features can get added or dropped as a product moves along—and each feature has these quality gates, and you’re not deferring work, you can look at it and know where you’re at against the plan.

It isn’t a case where you’re saying, “Well, our feature’s done, but we’ve got all this testing to do, and all our unit tests are actually broken, so we don’t have the code coverage. But we think we’re done.” Right?

Mark: Yeah.

Scott: You’re not in that situation. It sounds like you really know better exactly how “done” everything really is, across the board.

Mark: Yeah, definitely. We had 600-plus features. We had, almost to a tenth of a percent, a very good indicator of how far through the project we were. We would have these planning milestones, where we would look at the features we intended to implement during that period, and then what we actually managed to implement, and then we’d look at what our forecast was, going forward.

We could use that data in order to re-plan, to reset expectations, to say, “Actually, we were being a little overly ambitious.” And based on what we’ve learned, we need to cut some stuff that we were planning to put in the release if we still want to ship by this date. Or maybe we need to add a little bit more focus in these areas, because things aren’t going as well as we might want to.

So yeah, breaking it down into features, and actually knowing that a feature was done when it was checked in, gave us high-fidelity data around which to do re-planning, which definitely helped us to be more predictable and actually hit our RTM goal on time.

Sean: Based on the primary research we’ve done over the last nine months, if you talk to somebody in the open-source community, they would say, “Well, we have rigorous processes for checking in code too.”

But they differ, right? If you’re talking about security, people will talk about “many eyeballs” versus the Security Development Lifecycle. Around usability, if you talk to someone from OpenOffice, they’ll say, “It’s the voice of our users on the mailing lists and the forums.” And there’s a certain legitimacy to everybody’s argument.

But I’m curious, having been through the whole process, end to end, how much of your ability to push through a level of engineering excellence do you think was predicated on having a closed-source organization, where you can really kind of set down some pretty clear guidelines, and have people follow them? You want them to play ball, and you want them to feel like their voice is heard. But do you think the same kind of rigor could have been applied in an open source model, where you can scrutinize the code, but you can’t control the process that was used to create the code?

Mark: It’s difficult to say, because when you’re running a large division, command and control is not the best way of getting stuff done. You’re really depending on the energy and excellence of the engineers. What we’re doing is trying to put into place a system that encourages people to do the right thing. They already knew what the right thing was, but like I said, they’re under conflicting pressures.

The business pressures to get a product out at a particular time can offset an engineer’s natural, innate desire to do excellent work. A lot of what we did with the feature complete concepts, and the quality gates, was really around putting a system in place, at a divisional level, that allows people to live up to that expectation. I’m not sure how much being a corporate entity versus an open-source project plays into that.

Sean: Can you give me an example of a time you felt the processes you put in place was able to assist a developer, or a team, to say, “I know I’d love to have this feature in, but we’re going to have to put that off, or we’re going to have to delay it to a later CTP, because I just don’t really feel we’re hitting the quality mark.”

Mark: Code coverage is a good example. It’s very easy to skip on the testing side of things. Testing tends to get squeezed during the product cycle. Being able to write all the automated tests and get the code coverage that you need can be very difficult. You’ve got all of these other things that you’re trying to balance. The normal thing that happens is development rushes ahead, and test gets left behind, right?

Scott: Right.

Mark: This happens all of the time. What happens in the structure that we have now is that there’s this feature crew, with some developers and some testers in it; if test gets left behind, the feature crew can’t complete. You essentially have five devs there who are stuck, and they can’t get out of this feature crew, because they haven’t reached their code coverage goals.

So what happens in that case is that rather than leaning on the test guys to do more automation at the end of the cycle, you start having the discussions at the beginning of the cycle about how everything’s going to get done, and how the feature’s going to be built to support automated testing.

You get load leveling. You actually get developers helping out to produce automated tests, and helping to get code coverage up. It’s a much smaller set of people. It’s not the entire division saying “We need to hit 70 percent”; it’s a group of ten people who are working closely together, saying “What could we do to get to 70 percent?”

When you’ve got a small number of people trying to solve a problem like that, it’s much easier to turn that around, find solutions, and be creative. You get this accountability for the quality of the feature that runs across the whole feature crew. Everyone pulls together in order to achieve that result.

Sean: Where did the Engineering Excellence team get its inspiration from? Was this driven by an internal feeling that things were not going right? When you had those initial meetings and you said, “We really need to fix this,” was there anything you were trying to model from?

Mark: Yeah. I think the real start of it was going through the Visual Studio 2005 development process and not feeling particularly excited about how it went at the end. The product that we produced was great, but we didn’t enjoy the process.

Sean: Right. The testers have knee bruises from being dragged across the finish line, and being told to eat pizza at midnight. You know what I mean? To get stuff done.

Mark: Yeah. So it’s really coming out of that and saying, “How can we do better? What can we do to really improve the game here?”

And I’ve got to give a lot of credit to Soma for sponsoring this effort, for actually stopping and saying, “We’re going to take three months out and think about this problem. We’re going to do an MQ. We’re going to invest in our engineering. We’re going to try and do better next time.” And it really is difficult, when you’re running a business, to say, “Let’s take three months out and work on improving our engineering processes.”

Mark: Yeah.

Scott: And you probably maybe only know the answer to this anecdotally, but having been through the 2005 development cycle and having been through the 2008 cycle, do you feel like you guys made up that three months, just in having a smoother, more predictable process?

Mark: Yeah, we were always aiming for kind of a zero-sum game, which was to improve the process enough to at least get that time back. We built a schedule for VS 2008 that had a far more aggressive endgame. We closed down much more quickly than we did for 2005, and we were able to do that because of the investments that we’d put in up front.

We got a lot of stuff into VS 2008, and we managed to get that stuff in on time and not have this sort of unpredictable bug tail at the end.

Scott: Are the quality gates all mandatory across all features, or are some of them kind of optional?

One that you mentioned was a threat model. And I think, OK, well, if you’re right-clicking in the IDE and that’s supposed to make a pop-up menu appear, is that the kind of thing that somebody actually has to do a threat model for? Or are there certain things that don’t really warrant certain steps?

Mark: The reality is, there are a thousand things you have to think about. And when we created the quality gates, it was really about questions like “Where do we really want to focus people’s attention? What are the mandatory things that we need to be successful, that are true for everyone in the division?”

So, really, the quality gates are mandatory for every feature, but the amount of effort that may go into satisfying a quality gate for each feature may be different, right? If you do the threat analysis on something that is deep in the user interface and is building on a bunch of framework code, then it may be quite easy to say, “Look, we’re not doing anything new or risky.”

Scott: Right.

Mark: We’re not producing any new threats. Threat model done, right? Whereas if you’re introducing a new wire protocol, then you’ve got a lot of work to do. But the important thing is, by having it on a list, by actually having a person have to go and check the box and say “Yes, I’ve done due diligence on this line item,” it doesn’t get missed for the features where it’s critically important.

Scott: Do you think there are cases where people were working on a feature and they thought, “There’s zero risk on this. This thing doesn’t listen on the network, it doesn’t parse a file. It doesn’t take user input, no threats.” But then when they actually got to that stage in the process, somebody said “Well, but what about this?” And people went “Oh, yeah, yeah, I guess there is an attack vector here that we maybe wouldn’t have caught if we hadn’t stopped to think about it.”

Mark: Yeah, and I think that’s really the point—to make sure that people at least sit down and think about it once. They don’t just sort of push it to the bottom of their priority list and never quite get to it. And also, especially for the threat models, we had a separate process to go and identity all of the features that had an elevated risk.

The ones that had bytes on the wire, and that kind of thing. We did an extra deep dive into the threat models for those features, to make sure that the due diligence had been done, and things were going to be secure. So we had that second level. Again, because we had this inventory of features, it was possible to do that process.

Scott: Like you said, there are a thousand things that people need to think of. When a developer comes in who’s never worked for Microsoft before, how do they get up to speed? I know that you guys have formalized training, but there’s a lot of on-the-job education that happens too. How do you get new people up to speed on the Microsoft way of building products?

Mark: I think it’s a combination of training, and the groups that they’re working with providing that peer feedback, code review, design review, and helping to provide them with the experience to do a good job. At a divisional level, we try and address the problems that are going to help the division to be more effective. But what happens at the team level we leave very much to the product units and the individual teams.

So we have a bar, we have a set of expectations that we expect them to meet, but how they achieve those expectations, how they run the teams, is really up to the teams. Just to give an example of what I’m saying here, a lot of teams use agile methods—Scrum and Test Driven Development—in order to run their feature crews, but at a divisional level we don’t mandate that they follow any particular approach. It’s up to that team to decide what the appropriate approach for the feature that they’re working on is.

Scott: Which to me is kind of fascinating, right? I mean, on one hand, it’s cool that you’ve got a lot of autonomy, and teams can figure out what they want. On the other hand, people get so religious about agile methodology or think test driven development is the only way possible to build code. You have this product with 600 new features, thousands of developers, thousands of people working on it, and one team might have said, “I don’t know, waterfall’s always worked fine for us.” And another team says, “Test-driven development is the only way to go.”

At the end of the day, regardless of which methodology they’re using, they’ve all gone through the same gates to get those particular lines of code checked into the source tree and building on the mainline build.

Mark: We need a process that enables the division to be effective and meet its goals, but we also need to allow the individual developers and the individual teams, and the business leaders in those areas, to be passionate and creative, and not be held down by process. So at a division level, we don’t want to micromanage what’s happening.

We want to provide just enough structure that it all comes together nicely at the end, while allowing the teams to be creative. This is the developer division, and development methodology is something that people are excited about in this division, so we want to allow teams to experiment, to try out different techniques, different development methodologies, different tools. We don’t want to mandate a one-size-fits-all approach.

But at the end of the day, we need all of those features to come together and fit together and be able to ship at the same time. So that’s really what our goal was in building this process.

Scott: I know that Microsoft takes pride in eating its own dog food, right? For the 2005 time frame, you had some brand new types of products. You guys were rolling out a new source control system, for example.

You were rolling out kind of a new command line build technology, MSBuild. You guys were rolling out work items and defect tracking.

Out of that stuff, work items, source control, and build system, what are you guys using? I realize that you have to have working tools to have a working build process, but at the same time, you want to be using your own stuff that’s in development, but you can’t really expect a source repository that’s in development to be bug free… So talk about that a little bit.

Mark: We try and dog food that stuff as aggressively as possible. It’s my job to make sure that we strike an appropriate balance here. The division has to continue to be productive and yet we want to be as aggressive as possible.

So in 2005 we had a good chunk, maybe a third, maybe a half of the division using the Team Foundation System source control system, and the other half was using Source Depot. In the next product cycle, we’re going to have the whole division using the Team Foundation System source control. The way we used both was through mirroring, where we have certain branches mirrored between the two source control systems, which allowed us to exist in both worlds at the same time.

That allowed us to be aggressive about our dog fooding, but also hedge our bets a bit. As far as the work item tracking was concerned, we used a similar approach: we mirrored everything between TFS work item tracking and Product Studio, which is our internal bug-tracking tool. And that enabled people to exist in both worlds.

For the feature planning, we entered all of the features into TFS. We could enter that it had completed quality gates, all of that was done inside TFS work item tracking. That system provided us with the ability to define new work item types, which we didn’t have with Product Studio. So we try and use as much of our own stuff as possible.

Scott: It seems like one challenge with feature crews is dependencies. If my feature is dependent your feature, but you’re not 100 percent done, then your feature doesn’t show up in the mainline build for me to build on top of. That seems like that would be a big challenge.

Mark: Yeah, the best approach to that is to identify those problems during the planning stage so that you can architect your features in such a way that it takes those dependencies into account. So either the thing that everyone’s dependent on is produced first, or the codependent components are being developed at the same time in the same feature crew.

If you can architect the order and the dependencies between the features correctly, then you can deal with the fact that the code has a lot of cross-dependencies between different components. If you fail to do that, because you didn’t anticipate it up front, then you end up using sort of backdoor mechanisms to share code, and then you have to synchronize when you check in the features.

We had a couple of instances where things were very complicated, and we had teams mixing code together outside of the checked-in product so that they could use each other’s interfaces, but then when it came to check-in, they had to stage the order so that one team went first and then the other team went afterwards.

I refer to those as coping mechanisms. Ideally, everyone plans this perfect feature crew model, where everything has nice dependencies and is done in serial and there’s no overlapping. And then there are these coping mechanisms which allow things to happen in parallel in real life when things don’t quite go perfectly, so there’s some built-in slack there that allows people to still get the job done even if they didn’t do a perfect job of planning.

Actually, this was a learning experience for us, so my hope is next time, when we’re doing the planning, we’ll think about these things more carefully and we’ll have fewer instances where we have to rely on coping mechanisms in order to get things done.

Scott: It seems like one of the natural outputs of this is that you guys would have to do more thought up front about the dependencies, because you wouldn’t be able to depend on partially complete code checked in.

Mark: Yeah, and that’s where it’s important to understand that a feature doesn’t have to be a complete chunk of user-facing functionality, right? You may want to slice it into little pieces so that you can start to satisfy some of those dependencies earlier.

Scott: Oh, sure. I imagine a lot of stuff in the .NET Framework itself is a feature, but it’s only a code, it’s only an API that other stuff is going to use.

Mark: Yeah. If it’s necessary, you might even create a shim or a subbed-out API so the other people can start coding against it, even though it’s not hooked up at the back end, and maybe another feature will then hook that up and complete the scenario.

Scott: This has been a great conversation. Thanks for taking the time to chat.

Mark: Thank you.

Bookmark this:

Interview with Josh Berkus – PostgreSQL Core Team Lead – Sun Microsystems

campsean — Wed, 22 Aug 2007 16:35:17 +0000

Interviewers:
Scott Swigart and Sean Campbell

Interviewee: Josh Berkus

In this interview with Josh, PostgreSQL Core Team Lead at Sun Microsystems Inc., we asked him about:

Sean Campell: Josh, tell us a bit about your role with PostgreSQL.

Josh Berkus: OK. Well, what I’m best known for is that I’m a member of the seven member core steering committee for PostgreSQL Open Source Database Project. I also work at Sun on PostgreSQL and I’m the PostgreSQL lead, which is sort of a strategic and evangelism position at Sun Microsystems in our database technology group.

I’ve been a database applications developer since about 1994. I started out actually with desktop databases and moved to Microsoft SQL Server, and from there to PostgreSQL in 1998. For the last couple of years, I worked at Greenplum, a data warehousing company. I did a lot of consulting on database performance. Now I seem to spend most of my time going to conferences.

Sean: Talk a little bit, if you would, about PostgreSQL and where it kind of fits in the ecosystem with things like MySQL, open source on one side and the proprietary Microsoft SQL Server and Oracle Database.

Josh: Well, the way I like to describe PostgreSQL is we’re sort of the high end of open source databases: very SMP scalable; capable of running large, complex queries involving multiple sub selects; and, all kinds of SQL tricks. A lot of functionality is built into the database, including triggers and views and schema, and the ability to use 11 or 12 procedural languages to write procedures in.

Mostly, people who look at PostgreSQL are also considering databases like Oracle or DB2 or SQL Server 2005 for their needs – large database installations involving possible terabytes of data and usually a dedicated database administrator.

That’s probably the majority of our usage. We also get a fairly substantial amount of sort of embedded usage. Not because PostgreSQL is really an embedded database, but because of our extremely liberal BSD open source licensing terms.

Something I would like us to be better known for is the extensibility of the database model. The origin of PostgreSQL goes back to 1986, which was the POSTGRES Project at the University of California – Berkeley. It was the second UCB database project. The reason it was called Postgres was it actually stands for post Ingres, Ingres having been the first UCB database project.

One of the principles that PostgreSQL was founded on was the idea of an object relational database. That is, the database administrator or designer should be able to modify the behavior of database objects with code, or add their own new types of database objects. On a useful basis, that’s given us a whole bunch of exotic data types that are extremely useful for keeping certain specific and unusual types of data genetic sequences, geographic data, or cryptographic data that might be hard to store in the standard SQL data types, let alone to index or do anything useful with.

Sean: Well, tell me a little bit about out that, just to dive bomb for a second. Coming from a database background, there’s been a lot of discussion about this on the SQL Server side, progressively integrating additional data types, working in an XML data type, and so on.

Josh: Right.

Sean: But obviously, one of the challenges with something like this is making sure that it’s valid and useful for the community at large. I think the deeper you get into a particular data type, you really have to have it vetted by the community, to make sure it’s actually useful, and people don’t just go back and throw it into a more generic field type.

So, tell me a little bit about the process of getting something like that included in PostgresSQ, and maybe take the most interesting example you can think of. From soup to nuts, how did it get originally proposed, and then how did it get eventually integrated into the product?

Josh:So an example of one, actually, that’s going to be fully integrated into the core code is the UUID data type. It’s coming out in version 8.3, which is the next release version, and will come out somewhere around October.

Now, I know that SQL Server has had UUIDs for a while. We’ve been actually putting off incorporating UUIDs and GUIDs, simply because there’s five or six competing ways to make up such a data type.

So, when UUIDs and GUIDs were proposed for inclusion in Postgres, one of the goals with that was to actually create a data type that would uphold the various different ways of forming a UUID, and yet have a compact representation as well as its own index types and operators.

That is, for example, equality for a UUID is the same sort of concept as it would be for an integer. Differential operators, like greater than, less than, and that sort of thing, actually don’t function the same.

Sean: Sure.

Josh: Because, among other things, UUIDs will contain information on which machine created that particular ID.

Sean: Right.

Josh: Yeah. So, an example of a more exotic one that’s not yet in the core code, but is out there in our add ins and that I find much more interesting, is genetics data types. There’s a couple of projects; one of them is called Unison DB, which was sponsored and open sourced by Genentech, and another one is a community project called BLASTgres.

These are all projects of genomics scientists.

One of the things that they needed to be able to do is they needed to be able to store sequences of base pairs in the database. Now, that’s what we call a multi value data type, as in it has meaning as a whole, in sequential order, but also, each component has meaning in small pairs. Initially they tried to store this, actually, sort of vertically – you have each pair as a row. But when you’re talking about millions of protein sequences, each of which can have a couple thousand base pairs, that’s really not a realistic method of storage.

So instead, they actually modified our already existing array data type which is quite fully featured, in terms of having its own special comparison operators and special ways of indexing it and modified it to hold base pairs.So, that allowed them to do meaningful comparisons of things like equality, and particularly to compare individual base pairs. That is, if the base pair in this third place equals both sequences, what percent of the base pairs are equal to those sequences?

Because in protein analysis, you’re not expecting an exact match; you’re expecting a match, basically, according to percentages. You want to be able to look up and say, “OK, I want to find all the genetic sequences that have this particular sequence of six base pairs anywhere in the proteinî and that requires a special index type which is based on our generalized index search tree. The tree allows you to create your own index types to support some of these exotic data types.

Sean: One of the questions out of that, I guess, because that lays out pretty clearly what the feature set is, is about how you go about making the decision to incorporate something like that into the mainline product and/or to just kind of leave it as a community add on, for lack of a better phrase?

Josh: There’s actually a variety of features, and we’re going through this right now with our full text search type. So there’s a whole variety of decisions that goes into that. One is how broad the usage is. That is, is this something which is used by a large percentage of our users, or is it relatively obscure?

A second question is how good the code quality is. That’s a big deal, because we’re an open source project that’s been open source for 11 years and part of a project that’s 21 years old. Maintainability in our code base is actually, possibly, our number one priority because every one of our major contributors now was not here at the beginning of the project.

Every single one of them inherited the code from someone else, and they know how important it is to be able to pass it on. So that’s actually been a big holdup in incorporating our full text search type into the core, because it was written by some Russian developers and they had a lot of issues with doing internal documentation of the code in English.

So, in that case, for other data types and the like, there’s going to be the same sort of standards in terms of internal documentation of the code; good quality public user documentation; the code being formatted correctly and easy to read; and, having consistent sub functions and references that are easy to follow and match our other coding standards that are part of our documentation.

Another part of it is going to be, again, for the maintenance issue, whether or not we feel that the contributors are going to be with the project for the medium to long term, because if it’s something that somebody just dumps on us however good it is right now and then leaves, then what we’ve done is we’ve added to the burden of the core maintainers to keep up that extra 10,000 or 15,000 lines of code.

For an individual data type, that’s not very much. If you look at it like we add 10 new data types, then that requires us to have two more code maintainers, just to maintain that extra code base. So, if somebody is not going to be making a long term commitment to maintain the code, then the bar to add it to the core code becomes much higher.

Then, a final consideration would be external dependencies. That is, one of the things that we do to make PostgreSQL easy to install is that the dependencies to install the very core code of PostgreSQL with no additional options are extremely light. You basically need a handful of GNU utilities, certain C code building utilities, and that’s it. That’s made it very easy to install PostgreSQL on 30 different platforms; so has building it from source, so there’s a variety of means. For the people out on Windows, you can build it with either MinGW or with Visual Studio. That wouldn’t be possible if we had a whole slew of external dependencies.

So, thereís add ons that require heavy external dependencies. For example, there’s a procedural plug-in to allow you to use PHP inside the database. The reason it hasn’t been included in the PostgreSQL core code, even though it’s fairly feature complete and is reasonably popular, is specifically because in order to build it you have to have PHP and Apache installed and even configured in certain ways. So that makes it a real dependency issue in order to build that component.

That’s very tricky, and we don’t necessarily want things that are that hard to build in the core code. We’ll put them in add ons, where people realize that they have to take extra steps.

Sean: One follow on to that, too, back onto the core maintainers. I’m just curious, how many core maintainers do you have, considering that came up a couple times in the vetting process? And, how does someone kind of move from just a general community member to a core maintainer?

Because different projects handle that promotion process a little bit differently.

Josh: Yeah. We don’t have a formal process. We’re actually at the sort of extreme end of open source projects in that all of our policies are negotiated and unwritten, pretty much. It’s because of the age of the project and it’s because we’ve always had a consensus process for making decisions, which has yet to break down. So there hasn’t been a need for some of the more elaborate, formal structures you would find in, for example, Apache.

Sean: Right, so you guys haven’t had to have like a voting process, per se, on things. It’s kind of been a communal decision process.

Josh: Yeah. There are some things that are conventions. You’re not going to be considered, for example, for getting commit ability on the CVS tree if you haven’t been around the project for a couple of years, contributing. Again, it’s a 10 year old project. We feel that we can wait two or three years for somebody…

Sean: [laughs] Right, right.

Josh: If they go away in that amount of time, then we didn’t want them as a committer anyway.

Sean: Right, right. Somebody that flashes onto the list and is like, “I want to be helpful! I want to be helpful!” then six months later, you’ve never heard from them. You have kind of a base vetting process just from that alone.

Josh: Yeah, yeah. So we’ve got people who’ve been around and contributing code for a couple of years. Volume is also a consideration, because somebody who’s only contributing one or two small patches per version, then there’s no particular need for them to have any greater level of access.

The big issue is having lots of free time or having time paid by your employer to work on these things, because the main thing that we need from major contributors now is actually time to review other people’s code.

Sean: Well, one question on that, because we’ve found this to be interesting as we’ve talked to different projects, is how much of the project is funded by proxy, in the sense that somebody’s got a day job?

Is there a scenario where someone is funded predominantly by their employer to write code for PostgreSQL?

Josh: Yes.

Sean: And what percentage of the base of people writing the mainline code probably falls into that type of category?

Josh: We haven’t tried to do a count for about three years, but I would estimate, now, 80% to 90% of the code changes that go into any given version are written by people who were either paid directly to work on PostgreSQL development, or for whom working on PostgreSQL development is an approved use of their work time.

The second class would be large PostgreSQL users, like the staff of Afilias, for example. They’re not required by Afilias to contribute to Postgres, but if they want to spend Tuesday afternoon working on a Postgres patch, it’s completely acceptable to Afilias if they do so.

There’s a number of those. So, yeah. That’s actually one of the big myths of open source is people imagine a bunch of hobbyists. And I’ll say, in the early days of Postgres, we were hobbyists, because you couldn’t use it for much. I was actually earning my living as a SQL Server performance consultant, and working on Postgres for sort of my own stuff. But once a project gets big and commercially adopted, you’re going to find that at least three quarters of the code contribution comes from people who are paid to work on the project.

Sean: I want to give Scott a chance to chime in here, too. But one of the things that came out of one of the conversations we had I think it was with Michael Tiemann – was the concept of: alright, so you’ve got a set of developers, they work for a large company, and that company would like to get something into the product. So, they go off and squirrel away and work on some feature.

Letís say it’s the genetic discussion we were having earlier, right? That’s probably not the best example, but let’s imagine that was the case. How do you deal with the challenge of someone going and building ìxî, and they feel they’ve invested real company time and stake and equity in it, but yet, maybe it doesn’t make it in because the community writ large just doesn’t feel it’s maintainable or it doesn’t meet the standards you guys are looking for?

Josh: Yeah. Well, that’s something that needs to be handled with a fair amount of diplomacy. And there have certainly been failures on that in the past, but I think itís because all of our interactions with our contributors tend to be highly personal. That is, if something gets rejected, then it’s going to be after Bruce or one of the other reviewers had 16 or 17 different email interactions with a contributor. But no, that’s not until after we’ve given the contributor multiple chances, made it clear to them what needed to be changed, and given them multiple chances to modify their stuff, and hopefully made them understand why it was being postponed.

We’re actually struggling with that right now, because we’re having a bit of a fire hose problem with version 8.3; as in, when we hit feature freeze at the beginning of April, we had something over 100 different patches pending, some of which involved up to 50 60,000 lines of code. So, the result is we’ve actually set a very high bar for things making it into 8.3. Patches that we might have accepted in an earlier version and spent more time getting up to the acceptable standards of code are instead being held back for 8.4.

Sean: Well, one last question, and then I want to give Scott a turn.

So, from a database side in the open source world, what do you think the open source development methodology brings to a database product that either gives it more credibility, a better feature set, etc. when compared to a closed source model for developing a database product?

Because we’ve asked everybody this, in some form or other, and the responses have been really interesting. I don’t mean that in a pejorative way, I mean they’ve been very interesting to listen to.

But, we haven’t talked to anybody from the database side.

Josh: Well, actually, one of the biggest benefits is for security and reliability. We actually had Coverity run a code check on PostgreSQL a couple of years ago, something that they’re apparently going to do again for us, and one of the first things that I noticed is that the PostgreSQL core actually has possibly the lowest code count, in terms of lines of code, for any major SQL database.

What that’s indicative of is that we’ve spent a lot of time; that every time we release a new version, there is significant refactoring involved in it, and a real effort to keep the code clean and eliminate anything that’s Byzantine or hard to read.

The payoff for that is that it makes it very easy to keep the code reliable and secure. That is, if somebody reports a security issue, we can generally come up with a fix in 24 to 48 hours, because it’s very easy to zero in on exactly where the problem is happening.

It also prevents such issues from occurring in the first place, because there aren’t mystery functions that nobody understands and can’t touch. Having worked on some proprietary software, I know how that kind of stuff creeps into your proprietary software, because you’re more concerned with meeting the ship date, and the idea is that you’ll clean it up in the first update version. Only after you meet the ship date, cleaning it up becomes a low priority.

So for us, because all of the code is out there and that it’s all visible, maintainability is a primary goal. There is no postponing cleaning it up. The cleaning it up has to happen before we release. The result has been very highly secure and very highly reliable code.

Scott: So, you mention that there’s a lot of patches queued up, and a lot of them aren’t necessarily going to make it into this upcoming version. How is that a decision that’s made? Open source projects kind of have a different hierarchy and a different culture, so I’m trying to understand with PostgreSQL, is it kind of a representative democracy where the steering committee sort of votes on those or…?

Josh: Think of it as almost a pure democracy. Yeah, because most of those decisions are made by rough consensus on what we call the “hackers mailing list” which has something on the order of 7,000 subscribers. Although probably only 75 -100 of those people are really active.

Scott: Sure.

Josh: The rest of them are just monitoring what goes on. Basically what happens is, if there’s a patch, there’s a couple of other lists attached to that – the actual patches mailing list or the actual committer’s mailing list. But most of the discussion happens on “hackers.”

If somebody submits a patch, or preferably a specification before they submit the patch, then there’s going to be lots of discussion and we’ll form a rough consensus on whether it’s a good idea or not, whether it belongs in the core code, and whether it belongs in an add in, and other issues like that. Then, when the patch actually gets submitted, it becomes up to the code reviewer who will generally be one of our handful of committers, people who actually have direct access to the CVS tree who decide whether the code is of sufficient quality to make it in or whether it needs work.

If that’s an extended process, they will generally take that back onto the hackers mailing list and say, “This is a really cool feature, but the code is a mess and it needs X, Y, and Z. If the original contributor didn’t clean it up, is there someone else who cares enough about it to clean it up, or are we going to hold it back?” That will get sort of worked out there. And it’s sort of peer democracy. It’s not so much pure democracy as actually what I call “volunteerocracy.”

[laughter]

Josh: It’s that somebody can force a decision by saying, “This feature is really, really important to me and I’m going to do whatever it takes to clean it up so it can go in.”

Scott: Right.

Josh: When somebody doesn’t step forward and do that, often stuff gets held back.

Scott: OK. So there isn’t like a formalized vote, but it is pretty obvious kind of what the consensus is.

Josh: Yeah. I mean, the core team is a steering committee, but our goal is to actually do as little as possible. The main thing that we do is we set the date for feature freeze, beta and release. We handle security issues, because those need to be dealt with in a confidential forum, and that’s it.

There’s been months where we’ve gotten maybe a dozen messages total on the closed core list. The vast majority of any decision making, any reviewing, any discussion, happens on the public mailing list, particularly hackers, but also to a lesser degree, patches and committers.

There’s a few segments of PostgreSQL, things like the JDBC driver, which have their own development mailing list, so they make their own decisions within their development mailing list to submit their decisions to hackers. We take their word for it because the rest of us don’t know anything about Java.

Scott: Now, one of the things that happens with something like, let’s say, the Linux kernelÖ You’ve got Linus Torvalds, who essentially says, “Here’s the new kernel” and it goes out to the different distros.

And the different distros are all basically kind of a fork of that kernel.

The kernel that ships in Ubuntu isn’t exactly the same as the one that’ll ship in RedHat or something like that. And part of the reason for that is there may be certain features which are very important to RedHat customers, but it’s not something that they’re able to get necessarily into the core kernel, right? I’m curious if the same kind of thing happens with PostgreSQL; if Sun, for example, had customers who really needed certain features but the consensus was that those features shouldn’t end up in the core products – at least at this time – do you end up with companies kind of making their own weak forks for their specific customer or does that not really happen on this project?

Josh: Yeah, I’ll speak for PostgreSQL in general and then I’ll tell you specifically what we’re doing at Sun. In the case of PostgreSQL in general, it wasn’t something that used to happen, even though we supported the idea. We always have had sort of a kernel model, like Linux.

If you have the PostgreSQL core code, that 13 megabytes of code, and then you have probably 100 megabytes of add ins on places like (TP)Foundry and SourceForge and our contrib modules and a whole bunch of other places. In the past, it’s sort of been up to the user or the developer to add these things together themselves. What’s been happening more recently is that the packager for the Linux distribution or the BSD distribution has made certain decisions about what packages they want to include.

But those decisions have been fairly lightweight. People have not been taking the strategy of putting together an actual distribution until recently. And what’s changed recently is that we’ve gotten a lot of startups like Greenplum and EnterpriseDB that have sort of their own special version of Postgres that are deliberately maintained forms. In the case of Greenplum, it’s a data warehouse enhanced form of Postgres, and in the case of EnterpriseDB, it’s an Oracle compatible version of Postgres. There’s been a couple of others that are older, like the old Windows version, and the multi threaded Windows version called PowerGres in Japan, which is actually about four years old. Fujitsu actually has their own version, which is called Fujitsu Supported Postgres.

So, what’s been happening with this is that a lot of those companies do develop their own features for Postgres, which they submit, but don’t necessarily get accepted immediately, and possibly don’t get accepted in their original form.

That’s going to cause problems for those companies down the road, because… Well, I’ll give you an example. Greenplum actually developed bitmap on disk indexes. We currently have bitmap in memory indexes released with Postgres. But Greenplum developed bitmap on disk indexes a couple of years ago and put them into the Bizgres open source project to be submitted into the PostgreSQL core code.

The thing is that there have been some issues with index maintenance and with code style, and particularly with conflicts with other patches that we’ve incorporated to improve the performances of indexes, and that bitmap index patch.

Because those have not been resolved, the bitmap index patch is still not in the core code of Postgres. The problem that that is going to lead to is that when that patch does make it in in 8.4 or whatever it’s quite possibly going to have a slightly different API from the version that Greenplum has been distributing with their own proprietary product.

That will put Greenplum in the position where they actually need to have support for both versions: their original version and the version that made it into the core code. What that results in for these companies that are actually making core changes and distributing them in advance of getting at least vocal approval from the community, is that they develop an increasing maintenance burden. Now, companies like Greenplum and EnterpriseDB and Fujitsu in general recognize this and try to avoid that situation. They try to wait until their patch is queued and accepted before they start distributing it. But, like with the bitmap indexes, it doesn’t always work. Since the process of actually distributing modified versions of Postgres and marketing them heavily is relatively new, except for PowerGres, then it’s a little hard to see what could happen.

I mean, what could happen is what happened with PowerGres. XRA in Japan developed a multi threaded, high performance version of PostgreSQL 7.3 for Windows. But they modified PostgreSQL heavily to make it work in that context, to the point where it no longer worked on Linux. When the PostgreSQL project decided that we were going to adopt Windows as a platform, which we finally released in version 8.0, one of the decisions was that we wanted to have the same core code with no substantial differences regardless of operating system. That is, we weren’t going to have a separate code tree for Windows because that was going to be impossible to maintain.

So as a result, when we released the official Windows version, it was substantially different from PowerGres. So now a lot of the users in Japan are in the sort of weird position where they’re stuck with PowerGres, which is no longer advancing; it’s stuck at the version 7.3 feature set.

Or, they adopt the new official version, which is already five versions and four years later, and have a different performance profile and a lot of changes that will be requiring them to change their applications. So, that’s the sort of thing you want to avoid. That’s why at Sun, with our distribution of PostgreSQL, the PostgreSQL for Solaris, one of our policies is that we actually don’t distribute anything until it is accepted into the patch queue with a very strong assurance of acceptance versus revision by the PostgreSQL community.

If it is completely separable as an add in, if it’s something that can be added at build time and no later and doesn’t modify other APIs, then we might accept something. The particular example I’m thinking there is probes, which is that we can add in additional probes non invasively. It doesn’t matter if we add in a few extra probes before those probes appear in a community version, because they don’t affect other functionalities.

Scott: If you had to kind of ballpark it, how much of the effort do you feel like goes into adding new features to the product versus how much effort it is to support such a variety of platforms?

There’s different Linux distros, obviously, and supporting Windows on the same core code base obviously presents challenges also. How much time goes into bugs and testing and things like that just to ensure really good compatibility on all these different platforms versus building out new functionality?

Josh: Well, see there’s another way… You asked earlier about what the benefits were of the open source development process, and that’s another area where having really clean code is the big benefit, in that it’s allowed us to actually minimize the amount of platform specific engineering that we do.

We’ve also made some sacrifices in order to minimize that maintenance version, that maintenance issue. For example, on Linux and Unix platforms, we only use POSIX standard interfaces. This means that we’re not making use of some other operating system interfaces specific to particular operating systems that might give us additional operating system features and performance. For example, one of the big discussions I have here at Sun is that Sun’s new file system is ZFS and has some of its how APIs that are non POSIX. The ZFS people keep telling me it’s going to give us some tremendous benefit using databases on ZFS.

But we really don’t want to do that as an open source community, because the moment that you do that, you’re dedicating some amount of hours of somebody’s time just to maintain that interface for the code. So, we’ve completely avoided doing that, and that’s allowed us to minimize the maintenance version. That sort of platform specific bug and compatibility issue then becomes less than 20% of our overall development effort.

Now, Windows in particular, because there are more maintenance issues associated with Windows is very different from the POSIX platforms. We do have a couple of people for that. For example, one of our major contributors is Magnus… I’m going to mispronounce his last name, so I won’t say it. Magnus H. from Sweden. He spends most of his contribution time to Postgres, and he probably spends somewhere around half of his work time contributing to Postgres. He spends the majority of that, actually, maintaining Windows compatibility issues. So think about that as sort of one quarter of a developer for a year. Plus, a bunch of our other contributors and maintainers, like Bruce and Tom and Dave Page particularly, spend a minority of their time dealing with Windows build specific issues.

Again though, we try to stick to standard interfaces and to minimizing any particular code paths for Windows. Now, unfortunately that does limit our level of performance on Windows and our ability to integrate with some of the Windows utilities. But in terms of preventing us from having to have a completely separate version of Windows, it works.

Scott: If you talk to somebody who’s shipping a compiler, they would expect that… Well, Intel would sort of put people on the project to make compiler optimizations for Intel architectures, because Intel would really know how to do that.

AMD would put people on the project who make the compiler optimization for AMD architecture, because they would know how to do that. So you end up with kind of this collaborative effort where companies are kind of coming together and they’re putting their expertise in on the stuff they’ve developed. Microsoft isn’t staffing anybody on making sure that Postgres runs as well as it can on Microsoft’s operating system, is that correct? I mean, it sounds to me like you’re saying that work is being done by other people who…

Josh: Yeah. The Microsoft folks have been friendly to us, particularly Microsoft Labs have been consistently friendly to us, but Microsoft doesn’t contribute any efforts or help with the project. And I’ll actually say, except for Sun, who is directly involved.

For some of the other things, for example, Intel did actually have some ICC optimization efforts, but that was through EnterpriseDB, and I don’t think it would have happened without EnterpriseDB’s involvement. So, a lot of this has been by proxy, which would be the case with Microsoft as well, if there was any huge interest in it. Microsoft has its own database product though, one that they’re pretty dedicated to promoting. Well, a lot of individual Microsoft engineers have been extremely friendly to us, but nobody actually contributing to the Postgres project works at Microsoft that I know of.

That work is done entirely by community people. And even those that are primarily Windows maintainers, like Magnus, spend as much or more of their time using, say, Linux than they do Windows. So, there hasn’t been a big push to Windows specific optimization through them.

Sean: This is good. I guess the same thing we’ve offered to everyone, Josh, are there any closing comments you would like to add to the discussion as it’s extrapolated so far? Is there something you feel we haven’t’ touched that you’d like to get on the record, I guess?

Josh: I’d like to throw in a fun little factoid. This is aimed at Windows developers, yes?

Sean: Well, both. We’re getting heavy traffic from both sides. But still, there’s definitely some Windows folks involved, so go ahead.

Josh: Well, actually one of the other things that made it possible for us to do Microsoft for it has been that in general not for our product specifically but Microsoft has actually made tools to build and run programs made for Linux and Unix a lot easier on the Windows platform than it used to be.

One thing in particular is that the PostgreSQL project was actually the first user of the open source WiX installer, something we’re actually extremely happy with. It has allowed us to make PostgreSQL vastly more accessible to users on Windows because it provides them with a really nice installer. So nice, in fact, that after we released the first Windows version, a bunch of the Linux folks began saying, “Hey, why can’t we have an installer like that in Linux?”

[laughter]

Scott: Well, that’s one of the other things too that you run into, is that in the Linux world, and even in the Unix world, people are more comfortable kind of running, making and building it for their particular configuration.

But in the Windows world, it’s really sort of mandatory to ship a binary and an installer, not so much the source. But that’s interesting that you found the WiX project to be really useful for your needs.

Josh: Yeah. It showed up at exactly the right time, because it got released like, I don’t know, three or four months before our targeted first Windows release. We were able to go out of the gate with an installer, as I recall. I wasn’t actually involved in the Windows build at the time, so I’m not sure that’s 100% accurate. But certainly very close after having the Windows release, we had a nice graphical installer that was not only a nice graphical installer, but it also has like little check boxes for all the most popular add ins.

Scott: Oh, nice.

Josh: And again, if you’re a Linux user or whatever, somebody says, “Oh, that’s in the contrib module, you just need to type in these three commands and it will be installed.” That’s not really available to Windows users. They have to have a whole tool chain that doesn’t ship with Windows. So, having that nice binary installer has made it vastly more accessible for Windows users.

Scott: Do you have any sense for how much Postgres runs on Unix versus Linux versus Windows?

Josh: It’s a little hard to tell, because where we really get a full sense is the people that are active on the mailing lists. But we’re keenly aware that that doesn’t actually represent what’s actually out in the field.

Scott: Sure.

Josh: Because we only have, like, 35,000 – 40,000 people that are active on the various mailing lists and forums. Whereas even just judging by a certain manufacturer’s distributions, who are bundling PostgreSQL in their products, there’s several tens of millions of copies out there in the field.

Just given that Windows users tend to be more used to web forums and IM than they are to mailing lists and IRC, we’re guessing that we probably have a lower level of participation from Windows users. So, on the one hand, we have this vast majority of downloads from the Windows users, but on the other hand in terms of people who actually come back to the project and ask questions, the people that we know are on Windows is a minority.

But that doesn’t tell us who’s using it. You follow me?

Scott: Yeah, sure.

Josh: Because it may be that a lot of people are using it on Windows and they’re just not joining the mailing lists. They’re getting their help in other ways.

The other thing is that having a unified code base where we have the same version for Windows and Linux, etc., means that actually for a lot of the newbies asking questions about how to do this and how to do that, until we have some interaction with them, we don’t actually know what platform they’re on.

Scott: Right.

Josh: I’ve had a number of chats on IRC where it wasn’t until I got to like the third or fourth exchange of questions that I realized that somebody was on Windows. Which is a terrific thing in terms of being able to support our user base, because that was one of the biggest things I was worried about when we released the Windows version, that all of the sudden we would have 50,000 new Windows users hitting the mailing list, and those of us who are not Windows users would not be able to help them.

Scott: Right.

Josh: And it hasn’t turned out that way. So, I would guess that probably the majority of installations are in Windows, just based on the download numbers. But how much those people are using their installations, I couldn’t tell you.

Scott: OK. Great. Well, thank you very, very much for taking the time to chat with us. This has been great. You were really good to talk to because you were able to really dig into a lot of how the process works. If you look at a lot of our interviews, process tends to be some of the stuff that we’re the most interested in. And I learned a lot, just from this conversation.

Josh: Well, I’m happy to have a chance to explain this and some stuff about how the project works. We don’t have a lot of written policy, so it can be very opaque to somebody who’s just joining. I actually recently did a presentation on developing PostgreSQL at OSCON because we realized a few people had problems with how very indecipherable this was to how we were supposed to do things…

Scott: Right.

Josh: I’m really glad to have a chance to explain that a little.

Bookmark this:

Interview with John McCreesh – Marketing Program Lead – OpenOffice

campsean — Sun, 05 Aug 2007 21:03:49 +0000

Interviewers: Scott Swigart, and Sean Campbell

Interviewee: John McCreesh – Open Office

John McCreesh

In this interview with John who is the Marketing Program Lead for Open Office we asked him about:

Scott Swigart: John, thanks for taking the time to chat. Could you take a minute and tell us a little about yourself?

John McCreesh: I have been doing work on a voluntary basis for OpenOffice.org for five or six years now. For the last year I have been leading the worldwide Marketing Project.
My day job is in big commercial IT shops. I got into open source during the .com era – I suddenly found this wonderful world outside! As I’ve got a technical background, I started hacking for projects and then after a time decided I could probably do less damage by marketing than hacking. So that’s where I am now.

Scott: OpenOffice.org is interesting to us because a lot of the open source projects that we look at are things like the Linux kernel or Apache. Those are almost by developers for developers, to some degree. The feature set is driven by the developers who work on it. OpenOffice.org is different in that it doesn’t target IT professionals so much. It targets end users, right? The users are people who are producing documents, essentially.

So I’m curious, with OpenOffice.org, what’s the mechanism for features to get proposed, to be slated to be worked on? What’s the process for people deciding they are going to work on a particular feature? How does that work? Because it seems that it would be different from something like Apache.

John: That’s right. I think the traditional open source model is very much about scratching the itch. If you are a developer and you are not happy with the editor that you are using then you go off and write your own. That model does not apply in OpenOffice.org where as you say, it is much more of an end-user tool.
I suspect a lot more of our developers would sit down and write a document with Emacs than they probably would with OpenOffice.org.
The thing is though, lots of people get a big kick out of developing for a project that their mom and dad use, or the kids use, or they can take around and hand out at the school and say, “Look, you can use it for free and I am one of the people who helped develop it.”
Another thing is that this is a huge piece of code. So, for developers that poses challenges, compared to a lot of traditional open source projects which are comparatively small in terms of code and have a small number of developers.
The development methodology of breaking things down into small modules makes it easy for more people to work in open-source. But, OpenOffice.org has grown up over 15 years, so there is some very old code there and there is some very new code there.
Getting developers in to find their way around and to make contributions can take a bit of time. But equally, if you’re technically inclined, to get some code in there, to build OpenOffice.org, and see your work coming up the other end, again, that’s a big achievement.

Scott: Right.

Sean: When we talked to Stormy Peters, one of the things we talked about is how products take in end user requests, But So with that in mind what is your the overall process with OpenOffice.org for taking in a user’s request likesuch as, “I would like the ability to handle text editing just a little differently here because my boss wants me to.” But they have no ability to write it. In other words: What’s the process for getting those things integrated?

John: Yeah, we sometimes get interesting conversations between developers who say, “Well if people want this, why don’t they just write it, or find someone that can write it?” I think one of the challenges for us is to bring together people who’ve got the ability to respond to the user’s requests, and to get the users to put the request together.

We have the standard open source toolkits. Anyone can raise an issue or raise a request for an enhancement on our website. It will get checked to make sure it is valid. Other people can go in and support it, add notes to it and so on. So there is that traditional open-source process: people vote for things. It is an entirely open process and you can see whether what you’re looking for is commanding support in the community.

But equally we have a fair number of developers who are paid to work on the project. So, Sun Microsystems’ developers take the OpenOffice.org code and release it as StarOffice. Like any commercial software house, they have big customers who say to them, “Hey, you know this particular feature of StarOffice is poor. Can you do something about it?” If they’ve got a support contract with Sun, then Sun will put some developers onto it and they will get into the OpenOffice.org code base that way.
Similarly, when Novell started looking at the code base and decided to take OpenOffice.org and make it a significant part of their SUSE offering, they started using it internally themselves. There were things that they thought, “Yeah, we need to do this. We need to add this feature. This is important to the kind of market we are going to sell in.” So again, they took some of their developers and put them to work on it.
So OpenOffice.org is a combination of the traditional software model where you have a software company who listens to its users, plus input from users around the world.

Sean: What percentage of the new features that go into OpenOffice.org do you think are driven by a developer request mostly from soup to nuts or vs. what is onethose that is are driven predominantly by an end-user request but then just implemented by a developer?

John: That’s a tricky one.

Sean: Do you think it is equal or is the product still driven by developer input more than end-user input?

John: Well, it’s hard to say. One of the things that people said to us at the launch when we released the Version 2 product was, “Yeah, you’ve got all the functionality that we want but can you make it run faster?”

Now, from a developer point of view, a lot of the hacker community was really motivated by this. The thing about making more efficient code and making it smaller and leaner and all the rest of it really rang bells with a lot of people. That was something that had tremendous developer appeal, but it also has real marketplace end-user appeal. So I don’t know whether you would say that was driven by end-users or driven by developers. It was a happy area where the two interests coincided.

Sean: OK.

Scott: So on an open source project there is usually a chief maintainer right? If you take a look at the Linux kernel, it’s Linus Torvald who ultimately gets to decide what’s in or not. Is it Sun Microsystems that’s the final arbitrator of what gets checked into the main source tree and what doesn’t?

John: We have an Engineering Steering Committee, or ESC. It’s the ultimate arbiter if there are disputes about what goes into the code or what doesn’t, and the general technical direction of products is decided there.
On a day-to-day basis, the project is too big for any one person to sit there and say, “That goes in and that goes out.”

Scott: Right.

John: So OpenOffice.org is divided into a number of different projects, where the project leads have the ultimate say as to what goes in or what doesn’t go into a particular build.

Scott: OK. But I’m guessing that because Novell and Sun are so involved in it that some of their people would be owners of these different subsystems?

John: Yes. And I think that’s perhaps going to continue more and more as more commercial companies sponsor developers.

Scott: And that’s not uncommon. The one thing that I have heard across the board with open source is that company involvement is not a bad thing. Open source owes a lot of its success to the fact that IBM, Sun, Novell, Intel, AMV; other corporations are paying developers full time to work on it. So that’s an integral part of the process.

John: Yeah, I saw some analysis years ago about the Linux kernel. At that stage most of the contributions were coming from people who were paid by corporations.

Scott: Yeah, I would guess OpenOffice.org isn’t any different. I mean, when you take a look at a product that’s this large, that is this complex, there is a decent barrier to entry—I would guess—in really being able to get in there and really understand the code base, really develop experience with it, really understand the architecture and just get to the point to where you can make good, clean contributions that are going to add features or fix bugs and do it the right way.
For a lot of these projects, they have grown to the point where there is a high barrier to get to the skill level you need to get that to be able to do that. So it makes sense that the people who are being paid to work on it are naturally going to have an advantage in just really be able to produce the quality of code that is going to ultimately make it in.

John: One of the changes we have tried to make with the architecture in the past couple of releases is to open up the product more for extensions.

Scott: Yeah.

John: People who aren’t as technically skilled, and will find it a real challenge to go through pages and pages of code (some of which might be commented in German) and try and make sense of it, are quite capable of providing functionality in the form of a plug-in.

Scott: That also seems to be a key factor in the success of a given open source product—that it has a good extensibility story. It has a good modularity story. Apache is a great example of that. To work on the Apache core there is a high bar. But to write modules—anybody can do it, essentially.

John: Firefox is another classic example of that. It’s used a lot. Again, the developer gets a real kick out of seeing their piece of code looking as if it is part of the core product. Once you have incorporated the extension into the menus and in the help system, etc. then as far as anyone looking at the product is concerned, you have written a piece of Firefox or a piece of OpenOffice.org code.

Scott: Right, right.

Sean: I have a question about the new features. How is the QA process handled? A lot of our conversations have taken us in interesting directions as we talk to people who submit stuff, not in any pejorative sense for an open source but it just is interesting how who are contributors as the process of checking in code, andgoing through the process of validating it differs from project to project in terms of the rigor or the processes that go about it. So how does that work for OpenOffice?

John: There are two schools of thought. In OpenOffice.org there is the “Community OpenOffice.org”. So if you go onto our website and download OpenOffice.org, that is what you get. There’s also a hacker’s version of OpenOffice.org, which doesn’t go through the Community QA process. This version feeds into some of the Linux distros, who will take this code and will do whatever QA they feel is appropriate around it.

So if you want an absolute leading edge of OpenOffice.org, you go to the hacker’s version. If you want something that has been through quite a structured QA process, you go for the community version, which is what we do.

Scott: The community version, is that…? Who is ultimately responsible for doing that QA? Are there QA teams within some of the corporate sponsors of OpenOffice? Is it more of the responsibility of the distros to do that QA as part of putting their distro together?

John: QA is one of the OpenOffice.org community projects, which runs one of the most widely geographically distributed QA processes I have ever seen, because it’s not just a matter of getting the American English master version correct.

Scott: Right.

John: The native language teams in all the various countries will go ahead and do a QA process on their own build. Ultimately that’s one of the things that controls how quickly we can release. We used to go for long periods of time between releases. A couple of years back we looked at that and decided it was putting developers off. If you have to wait a year before you can see your code emerging into the marketplace, you’re not really very interested. We’re now down to about four releases for the year.
We would like to do more but the QA process is a limiting factor.

Scott: Sure.

John: Similarly for the hacker’s version of the code: if Red Hat or Ubuntu decide to use that as their source, they have to put it through whatever QA process they feel is appropriate for their marketplace. We all have tradeoffs between features and a stable product.

Scott: Yeah, yeah. And things like documentation also—is there a documentation team as part of the project as well? I would guess that would run into the same localization challenges because the product is such a global product.

John: That’s right. We also have a Documentation Project that does user guides and manuals and how-tos and all that good stuff. But there must be about a dozen other independent sites on the web where people have set up help forums or wikis or whatever.

Scott: Gotcha.

Sean: How much of the documentation creation is deflected or is tuned based on gaps you find that users are asking for? How much documentation is written because someone feels a personal need to write that documentation, or because they know people have struggled with it—similar to a developer adding a feature just because they need it?.

And how much of it is driven top-down from OpenOffice, such as, “We see X amount of requests for this, we need to build a doc set out to cover that need?”

John: It’s a mixture of the two. The way that I got into the OpenOffice.org project in the first place was I wrote a piece of documentation – a ‘how-to’. I’d been playing with the database stuff, thinking “Hey, this is really cool.” But it wasn’t not obvious how you got into it or how you found it in the menus.

So, I asked a few questions on the lists, and wrote the how-to. That got a fair amount of circulation, so I realized there was obviously more to OpenOffice.org than meets the eye. And that was how I got into the project.

So, an awful lot of that goes on. You can never have too much documentations and how-tos for a product as big as this, because different people have different learning styles and different needs.

Sean: One last follow-up on that. Are there types of documentation or areas of the product where you find that maybe there’s a pocket missing content-wise because it’s more community-driven versus top-down?

And are there any defined areas where you feel there is a gap and you have to motivate the community to contribute in a particular area?

John: I think the problem we have is the old Google phenomenon, that there is now such a mass of stuff out there on the web, it’s tricky finding the stuff that may be current, knowing what versions it applies to, and then finding the information in the form that you require.

We usually do one big master user guide, which we publish on the web. It’s also available from one of the publish-on-demand organizations, so you can order paper copies of it from there.

But apart from that we don’t try and have a monopoly on documentation because there are just so many other people out there who want to do it.

Sean: But you would say that if there is a pain point of sorts, it is that, “OK, I’ve Googled how to do mail mergex, but I don’t know if it’s the appropriate steps for this version of the product.” You know what I mean?

Or if it is, it might be missing an extra step that could accelerate ithe process of getting the job done from a user perspective. It’s like you said, you could solve a lot of developer problems by Googling them. That’s almost like Step 1. But on the other hand, you’ve got to make sure it maps to your version of the API, your version of the development tools, those kinds of things.

So, that’s a fairly accurate reflection of the main pain point.

John: I think so. We can produce a master reference manual, but it is in reference manual style, and some people find that off-putting and they’d like it more conversational style.

Sean: So, one question that I have is in OpenOffice, one of the key features is its cross-platform nature. It runs on Windows, it runs on Mac, it runs on every Linux distribution. Do you have any sense of how much development effort goes into ensuring that it is supported and runs well on all of those different platforms compared to the effort that is put into building new features?

John: I think it’s back to the QA limitation, that the more that we officially support, then the more testing that you have to go through. It’s also a major technical challenge because, if you’re completely cross-platform, then in a sense you’re always slightly sub-optimum on any given platform.

Scott: Well, that’s another question. Does OpenOffice.org strive to have full fidelity across all platforms? Because I know that some open source products go the route of they’re really optimized for Linux, and they’ll run on Windows or Mac or things like that. But they’re not 100% the same experience. Do you know if the philosophy of OpenOffice.org is that the experience and the features be 100% and the same, even if it means that it’s not really optimized for any platform?

John: That’s effectively where we find ourselves.

Scott: OK.

John: This is why the Mac port has taken such a long time, because the Mac is such a specialized platform and Mac users want all their applications to look just exactly like a Mac application should do. So, it’s quite hard to do that and still maintain the cross-platform thing.

Scott: So, there’s a core that’s really generic, but then for some of the presentation layer stuff, there’s actually Windows-specific code, Mac-specific code, you know, Gnome- and KDE-specific code.

John: We provide for example a common file selector out of the box. Or you can switch it off and say, “Just use the native file selector for my platform.” But the more of this you offer, the more your documentation has to be platform-specific. The common look and feel that makes the documentation piece easy to do.

Scott: Right.

Sean: This is in a different direction, but there’s an argument that goes that certain products in the open source world tend to follow rather than innovate. Sometimes OpenOffice.org is held up by that, right?

It’s trying to get to some base level of parity with either the most recent or the current version of the obvious competitor, which is Microsoft Office. But adding features that leap ahead beyond the current evolution of the office product isn’t typical.

I mean, just this statement, what’s your general response to that, when you bump into that? Because I’m sure you must have, at least at one point or another.

John: The problem is that office productivity software is a mature market. It’s been around a long time and we’ve been through the WordPerfect and the SuperCalcs etc.. By now, people know what they want out of an office software product, and they also have an expectation about how it’s going to look and feel. You know, where they’ll find things on menus.

So, you need quite a strong driver to go out and break out and produce something radically different. Chances are that unless it fundamentally gives people a better way of working, then they’re not going to accept it. Why would they learn this whole new thing if they only want to write a memo? They already know how to do that.

Sean: I guess in a product like OpenOffice, what would be the path to innovation? If you’re saying, let’s imagine a world where both are free, right? And we’re trying to equate it based on our level of productivity benefit to the end user and those kinds of things alone.
Where do you think OpenOffice.org or another product that’s a productivity suite, can have a leverage point there, I guess?.

John: Let me tell you where I think this happened. When we launched Openoffice.Org 2, we closed all the functionality gaps between OpenOffice.org and Microsoft Office. So for the average user, what they do 90% of the time is absolutely no different on either package. We recognized that and Microsoft recognized that.

So, our response was, well, the one thing that people are telling us is that, “OK, maybe we’ll move to your software, but every time we move we’ve got all these data conversion errors, or data conversion problems”. Or “I spend all my time writing these documents, but unless I’ve got your brand of software, I still can’t access my documents.”

So, we were hearing a demand from users that they wanted to own their intellectual property, the documents and spreadsheets that they’d created, independent of any software supplier. This took us off down this road that is ultimately the OpenDocument Format that you’re well aware of.

Sean: Yeah.

John: So users got an ISO standard for how office data should be stored. Once you’ve got that, then no software vendor can ever lock you into a product again. And it makes it easier to get your data and use it in corporate systems and all that good stuff.

So, that was our response to “how do we differentiate ourselves in the marketplace from Microsoft?” We knew that Microsoft would have grave philosophical difficulties going down that route.

At the same time, Microsoft looked at the same problem from their side and went, “What can we do to put a gap between ourselves and OpenOffice.org?” And their response was not actually to add new features or change the core of the product, but to change its look and feel.

This is where we got the Ribbon and all this other stuff. Sure, it looks different. Their response was pretty much a consumer-driven thing. “Let’s make it look sexier in the marketplace. Let’s get it looking so sexy so that people see it at home and then they go back to work and say, ‘Hey, we’ve got to have this Ribbon thing at work.’” It doesn’t actually add anything to their productivity, but it looks cool.

So, Microsoft has gone off in one way, which is a consumer-sexy-look-and-feel thing, and we’ve gone down the more technical route. What people are telling us on the domestic side is they want to know that their grandchildren will be able to pull up Granddad’s diary from 50 years ago and still be able to access it in whatever office software they’ll be using then. On the commercial side, public administrations worry that 20 years from now, someone’s going to come slap on with a Freedom of Information Act requirement to dig out some documents they’ve filed today. Are they going to have the word processor that wrote that document 20 years ago? Absolutely no chance!

So, the ODF development was our response to that demand for freeing the data from the application. So, was that innovative? I think that was hugely innovative. I think it was far more innovative than Microsoft’s response – . they got a few bells and whistles and their menus look a bit different.

Scott: Something like the Ribbon, that’s an interesting point, because how does OpenOffice.org make a decision about whether or not OpenOffice.org will adopt that look and feel or not?

Because I think that’s been a debate. OK, Microsoft significantly changed their look and feel, should OpenOffice.org do the same or not? What’s the thinking on that? And, I guess, how do you make decisions like that?

Sean: And –before you answer that—one of the things that I’m pretty cognizant of is that part of the change for the Ribbon was obviously just change itself. And then the other part of the change was driven by some studies that said, “Well, let’s put the first thing a user wants to use first in the Ribbon, etc.”

There are a lot of arguments about the decisions that were made, right? One person’s choice might not be another’s. But that was the driving influence too. But And to Scott’s point, how would something like that come to fruition in the OpenOffice.org environment? If it was deemed necessary, obviously…

John: OK. The Ribbon thing is one of the best things that happened for us for a long, long time, because over the years, we were sick of hearing arguments that said, “OpenOffice.org menus are slightly different from Microsoft Office’s, therefore, if we’re going to have to migrate people, it’ll cost us billions of dollars to retrain everybody to know that option isn’t here, it’s somewhere else.”

Scott: And now they’ve given you even a bigger change on their own end, right? So, yeah, there you go.

John: It’s a much simpler migration path from an end user perspective to go from current Microsoft Office to OpenOffice.org than it is to go to completely new Ribbon-style interface.
Will it catch on in the marketplace? I don’t know. I think it’s a big gamble on Microsoft’s part. They’ve tried a couple of times in the past to push the market in ways the market has eventually revolted against. They have had marketing failures in the past.

And it also takes an awful long time to get people off legacy versions of Office. There’s never very much more than low double-digit numbers of people using the latest version. So when latest version is such a big step change, it’s a big gamble for them.

Would we ever go the same route? Well, if five years from now, if everybody in the world has decided that Ribbon is the way to go and that’s what they want in a product, I suspect we will have to do something similar or come up with something better and convince the world that’s there’s a better way to do it. But at the moment the jury is very much out.

Scott: So in other words something like that, the prudent thing to do is to take a wait and see approach, in other words.

John: Exactly, just as Microsoft is doing around ODF. It’s doing its usual thing of trying to say, “Well, OK, if you want a standard, we’ll invent the Microsoft standard, and try to sell that to the world.” So that was a fairly predictable response.

Scott: Now, I thought I did read something saying that Microsoft was going to allow something like a “Save As ODF.” I thought recently there’d been a change in their thinking on that. I mean that’s outside of the scope of this conversation. But it seems like I bumped into that, but maybe I, maybe not, I don’t know.

John: What they have done is they’ve offered some support to an open source project to set up converters – ODF plug-ins for Office. Part of their recent agreement with Novell—and I think there’s something with Linspire as well—had some clause in it about working to get more file compatibility.
So, they’re not stupid. They’re keeping in touch with the technology, finding out how it works, so if they do find the market is forcing them down that way, you know it’ll be a comparatively easy thing for them to quietly drop into the next release of their product.

Scott: OK, I get it.

Sean: Well, one question on that too. So would you say that one of the strengths of open source is that you’re not going to push for a change as significant as the ribbon unless the user base asks for it? In other words, you won’t push a change like that top-down and if the users aren’t asking for it then a wait-and-see approach makes the most sense because nobody’s asking.

John: That’s right. It’s hard to think of anything where we’ve gone to change for change’s sake. But, equally, if one of our developers came up with a really cool new feature that no one had ever seen in an office suite before and, and we looked at it and thought, “Hey, that’s, that’s wonderful,” we’d do everything in our power to get it out into the product.

Scott: Well and I guess too you guys have the option of, like you said, there’s the hacker’s build, there’s the experimental build, and it seems to me like a lot of things could be tried out there, and if they caught on and were popular, it might make sense to move forward into the stable tested one.
Is that how it works? Is there a Darwinian effect with features where, somebody codes it up and it makes it into the hacker’s build, and then it either lives or dies there? It either catches on and it makes it into the, the community one or it doesn’t and it doesn’t.

John: Yep, that’s been a route in the past and we think the extensions route will be a much faster way in the future. So if someone brings out an extension, and we see that everybody and his dog is downloading it and blogging about it saying how wonderful it is, and if it makes sense to put that into the core product, then that would be a very good way of moving the product forward.
But, equally well, if it’s working well as an extension then why would you want the overhead of putting it in the core product, unless from a maintainability or efficiency or some other reason?

Sean: Well, what, what, one question I want to ask goes back to the original genesis of what we’re trying to accomplish in our investigation, “if the question was posed to you, “OpenOffice.org is predominantly an open source project. Microsoft Office is obviously a closed source project. From a development methodology standpoint, what would you say are the important characteristics that are advantageous on the OpenOffice.org side compared to Microsoft Office?”

Scott: What Sean is going for is what do you see as some of the biggest advantages of having OpenOffice.org developed as an open source product? What comes out of the open source process that is very advantageous?

John: OK, from a developer’s point of view clearly it means you know anyone can contribute to the project without being a Microsoft employee. From a general marketplace perspective, the appeal of open-source is it’s a transparent process. Anyone can request features, record bugs, whatever. If you’ve ever tried doing that with Microsoft you’ll know it’s not a transparent or efficient process. Anyone can come along to the OpenOffice.org conference and talk to developers and buy them a few beers and say, “Hey! Why don’t you come work on what I’m looking for?”

Then there’s the whole issue around openness of what’s in the code and what isn’t in the code. There’s been umpteen conspiracy theories over the years about Microsoft back doors etc. OK, most of those are just Internet conspiracy theories, but for a lot of governments in the world who are suspicious of big corporations, the fact that they’ve looked inside the code, seen what was there, get their own people looking at it, is very important.

And finally, software companies come and go. Even Microsoft will go someday. If you own the code or if you can get a copy of the code you’re guaranteed that you can run it just as long as you can compile it.

Scott: John, thanks for taking the time to chat.

John: Thank you.

Bookmark this: