Interview with Bob Bray – Geospatial Architect

Interview with Bob Bray – Geospatial Architect – Autodesk

Interviewers: Scott Swigart and Sean Campbell

Interviewee: Robert Bray

In this interview with Bob, Architect for Geospatial Products at Autodesk, we asked him about:

Sean: Bob, why don’t you give us some background on your role with Autodesk.

Robert: I’ve been in software development working for independent software vendors for about 17 years now. My current title is Architect for Geospatial Products. So, I’ve gone through the various ranks, from being a programmer in the trenches, in the cubes, to managing a product development team, to my current role as architect.

So, as I said, I’ve been through various things, mainly in closed source. Until roughly 18 months ago when we open sourced MapGuide. I think it was about six months before that that we really started heavily investigating that idea and started to put the ball in motion to actually make that happen. Basically, I’ve been involved in an open source project very actively for the last 18 months, and I won’t say that I was the key driver that made the decision that we were going to open source this, but I basically took it from that decision point and said, “OK, these are all the things we need to do…” and made that happen.

The product itself is MapGuide.The open source version is called MapGuide Open Source. The closed version is Autodesk MapGuide Enterprise. The codebases are the same. There’s one open source code stream for the whole product. There are a couple of minor functional differences between Enterprise and open source, but they are extremely minor, and we are trying to close as many of those as we can, actually. What we really have with the Enterprise version is basically a commercially tested and supported version. It’s much like the Red Hat model or the MySQL model, in that respect.

Sean: So, just out of curiosity then, if there are some functional differences, what led to those? I mean, they’re minor, like you said, but was some of that just by the nature of having to switch from closed source to open source? Is there some functionality that just couldn’t make it over that transform right away?

Robert: Basically, that is exactly it. We have a commercial desktop mapping product called AutoCAD Map 3D. We used some technology from that in the Enterprise version of MapGuide, and we do that for compatibility between the two products. So, those particular pieces, our coordinate system library, in particular, and our raster image handling library; those are things that we just could not open source at the time.

Sean:So, when Autodesk made the switch, did you model against an existing open source project in terms of how you wanted to handle QA or how you wanted to establish a community feel? You look at different products, like Subversion – I’ve heard they had one vote in the entire history of the project, right?

Robert: [laughs]

Sean: Literally. I watched their presentation at OSCON, and at least that’s the public face of things – that they had one vote. Or, for example, we talked to PostgresSQL yesterday, and they said, “It’s a very communal democracy” in terms of their decision process. They don’t really have a hierarchy. I know it varies across the projects, so did you look at the open source community and say, “Well, now that we’re going to open source this, we’d like to mimic this particular project” at least in some aspect?

Robert: Absolutely. So, when we made this decision, one of the first things I did was take a broad look across all of the projects. I was mainly focused on looking at the geospatial projects, because, if you will, our kin in the community is all of the open source projects – things like MapServer, GDAL, PostGIS… All those open source projects are very close in nature to us, so I certainly did look at those.

I also read a couple of books. The one I liked best was Karl Fogel’s book, “Producing Free Open Source Software.” Of course, he led the Subversion effort, but if you read that book you wouldn’t know there was only one vote in Subversion.

Sean: [laughs]

Robert: [laughs]

Sean: I was a little surprised, too. But, these were two guys from Google, so technically they’re reinforcing each other in their viewpoint, at least. It’s not just one guy saying it. They’re committers on the project…

Robert: I’m shocked.

Sean: Yeah, yeah. I was a little bit, too. But that’s part of the fun part of this investigation is some of the stuff we hear, and then we get to cycle back and say, “Really? So tell us a little more about that.”

Robert: So, we definitely took the approach of forming a community, and the project, and I say not from day one, but within six months, became a community driven effort. MapGuide has always been a product that, it’s almost like there’s a religious following of users out there. Even when it was a commercial product, it had some real hardcore, dedicated users.

So a handful of those users said, “Wow! This is going to be great!” They knew nothing about open source at the time either, but they kind of all jumped on board right away, started testing the open source version, and started getting active in that.

So, when we formed the community, basically, because we were modeling after MapServer and some of the other open source projects on the geospatial side and I really felt that was important we basically created a project steering committee for the product out of that group of people. So that project steering committee, right now, consists of three people from Autodesk and four people from outside in the community.

Sean: Are those people then the chief committers on the project?

Robert: Actually, that’s an interesting thing, because I would say there are a couple of people there that are active committers; the rest of them are users.

Sean: Got you. Because the interesting thing to me about this is that, obviously, a lot of these projects have grown up over time. PostgresSQL has a 20 year history, and to become a committer on their project, they told us, “Well, we wait two to three years to figure out if we can make you a committer, because we’ve been around for 20 years.”

But, if you birth a project out of the ether how do you determine who’s going to be a committer? Because, obviously, there’s probably a stampede of people at the start that say, “I’d like to be a committer, I want to be involved at the highest levels,” that kind of thing. How do you go through that winnowing out process?

Robert: There’s a URL I can send you, (http://mapguide.osgeo.org/psc.html) but basically what I did was draft a set of rules for the project steering committee to follow that include who can be a committer and how they become a committer.

We basically go through a similar process to, I think, Postgres – although we probably wouldn’t be two years in the road doing it. Basically, people start submitting some code from the open source community. They submit it to a developer, we’ll have that developer review it, and then do the actual commit. It’s almost in a patch style, right? Then, once we’re comfortable with them, we’ll vote to make them a committer – it’s a project steering committee vote.

Sean: Got you.

Robert: So, it’s kind of interesting, because, again, the project steering committee is not the core developers; again, it’s a core set of mainly users.

Sean: Once you transitioned over and said, “I want to make this an open source product, ” since you came mainly from a closed source background, what was the thing that most pleased you about being able to transition into an open source development model? What was the thing that you were, I guess, either most afraid of or most reluctant to see transition over? Because, obviously, the models are pretty different.

Robert: I think I was scared to death for quite a while…

Sean: [laughs]

Robert: …because it was completely unfamiliar territory to me. I’ll be honest, right? I’d never worked with an open source project before. I’ve used a few, but I’ve never really participated in an open source project. So, from my perspective, open source projects were these big free for all things, and I’m like, “Oh, my God. What are we getting into here?”

But, when I stepped back and actually looked at it, what I realized is that open source projects…yeah, they run a little differently than commercial projects, but they follow a fairly well-defined process; or at least the large projects do, that are successful. You can see a well established process there. It’s not a free for all, anybody can commit code, and a thing like that. So, that was my biggest fear going in; it was the step into the unknown, right?

Sean: What do you think was the biggest lesson learned from that? Now that you’ve got a couple of years at it, what would be the thing you might’ve done a little bit differently, given an opportunity? It’s apparent you wouldn’t have changed any decision to go open source, and that rings through clear. But, is there anything you would’ve changed during the process of going in that direction?

Robert: If anything, I would’ve been more open with the community right from the get go. I think I had a lot of reservations going into it, and so I didn’t really communicate as much as I should’ve right up front about what we were doing, how we were going about it, and all of that. So, I think I would’ve been much more open around communication right from the get go.

Sean: OK…

Robert: That’s the biggest lesson I think I learned.

Sean: …then what was kind of the biggest positive that you got? I didn’t want to leave that off the table either.

Robert: The biggest positive is that from a community perspective we are getting a lot more up front review; a lot more users using features before they go into, say, the commercial version or whatever, right? We’re getting a lot more early feedback on things that we’re thinking about doing or the ways we’re thinking about implementing new features, lots of early feedback that has a very, I would say, positive influence on the end result of the code that we actually write.

Sean:Well, I’m curious, too. That’s a functional change in the product, but have there been any change to the engineering process on the closed source side of the product? So, you guys open source it, that obviously rattles around and after a couple of years; I’m wondering if that has had any kind of change in the way you guys have developed internally, just having an open source mirror of the project, so to speak.

Robert: Well, it’s not an open source mirror. We only have one copy of the code, and that’s the public one. So, any change that we make to the code, whether it’s for the commercial release or for the open source release, typically gets vetted through the same process. We’ll go to request for comment. That’ll go out to the open source community, because it’s going into the open source version, one way or the other. So, we get early feedback on pretty much everything that goes in.

Of course some features that we may plan are for the enterprise version only, and those of course do not go through public review. Also we reserve the right to decide which features go into which enterprise release and we make no announcements about that because we are a public company right?

Sean: Right, right.

Robert: We don’t comment on that. We don’t do anything like that in the RFC process. But at the same time, the open source community certainly gets to vet all this stuff early. So, any code changes, and basically the whole product’s development process, are morphed into more of an open source development process for both the commercial and open source releases, because, again, we only have one copy of the code and we have one development process.The entire engineering team has basically transitioned to an open process, which is quite interesting.

Sean: Taking it down one level, as we talked to people sometimes as they’ve gone through this change, and the engineering team has a variety of differing reactions to it…various resistance points, or to put it maybe slightly more diplomatically, just things they have to learn as they go through the process. So, anything that would be worth talking about there in terms of lessons learned by the engineering team as they walked through this?

Robert: Yeah, I think there were a lot of lessons learned. Immediately it comes back around to being open about the kind of change we want to make and how you actually communicate with an open source community, because communication within the internal engineering team is quite different in my experience than communication with an open source community.

Sean: Now all of a sudden it’s a much more communal feel and development managers all of the sudden, I assume, had to make the switch from assuming everybody was on the payroll; there had to be a little bit more negotiation that affected the process of getting something developed, vetted, built, etc.

Robert: That’s definitely part of it. But also for our internal developers, I mean, they’re used to this kind of hierarchy of being able to walk down the hall and talk to somebody else. They’re used to being able to call a meeting and go in and start scribbling some stuff on a whiteboard.

With an open source development community that doesn’t really work anymore. You need to use the mailing lists a lot more. You need to use things like IRC for communications. So, it was really a complete change in the way you communicate.

Scott: So, that’s one of the things that I wanted to drill into because you’re obviously steeped in closed source development methodology. You’ve been in open source long enough to where you’re now immersed in that.
What do you see as the differences, kind of taking them side-by-side when you think about the aspects of the SDL, right? …how’re ideas envisioned, how they’re kind of tasked for somebody to code them, how they’re put through QA, how a product is released… If you don’t mind, I wouldn’t mind just kind of walking through how each of those is sort of fundamentally different in a closed source versus open source project.

Robert: Sure.

Scott: So, I guess starting with analysis and design and that sort of thing, how do you see analysis and design and requirements differing in open and closed source?

Robert: Sure. The requirements from a closed source perspective, I mean, we have project managers who basically go out and talk to our users. They’ll do focus groups. They’ll do surveys; all kinds of things like that just to basically pin down to a set of requirements for release of commercial software.

For open source software, you have a community of users who propose some ideas or a developer who proposes an idea. It gets kicked around on a mailing list for awhile. Eventually somebody either just picks it up and implements it, or the other option is they just walk in with something they’ve already written and say, “Hey, how about this? Isn’t this great?” So, the difference is really to me that it’s a little bit more organic, if you will, in the open source community. Whereas analysis and requirements and product definition is a much more rigid process in the closed source world, in the open source world it kind of happens organically.

Scott: So, where do you see those potentially? I’m guessing that both sides have their advantages. What do you see as the advantages of focus groups, meeting with customers, and project planning and what do you see as being the advantages of kind of more of the organic method of sort of showing up with a feature and saying, “What about this?”

Robert: If you show up with a feature and it’s already got a prototype or an implementation done – it’s a lot easier to talk about because you can actually see it and you can play with it. Whereas in the closed source side, you’re talking more in the abstract, if you will. So, the advantage in the open source side is really that you typically get a lot more users providing some feedback on it, at least if they’re really interested. Eventually you get to experience it before it actually goes in, which are both useful things.

On the closed source side, I mean, the advantage is that the release potentially comes out with a set of features that are much more…I won’t say well-defined because I think that’s the wrong term, but a release will typically come out that is much more cohesive as a whole. Instead of having a bunch of new things sprinkled about like a typical open source project has that’re mostly unrelated, a typical commercial product release will have the sort of features that are more or less a cohesive unit, if you know what I mean. Does that make sense?

Scott: Yeah, yeah. When I was a developer one of the problems with me kind of deciding what should be in the product was that I could have a project manager come along and say, “The product should really do X.” I would try to convince them that it shouldn’t, not because I had a better knowledge of the customer than them, but because I knew what a mess that would be to try to code given the existing architecture.

Do you feel that that kind of thing happens on an open source project because a lot of the people working on it kind of know what it would take to code it? Like, they might shy towards or away from certain things?

Robert: Yeah, you don’t see that in the open source world in my experience.

Scott: OK. So, ideas get proposed on the mailing list, things like that. You guys have internal developers who are still working on it. Is it that everybody sort of volunteers to work on different things or do your own internal developers get tasked with certain features? Obviously in the community it’s sort of all volunteers, so somebody external has to decide to pick something up and run with it. I guess, talk a little about how that works in terms of actually getting the work done.

Robert: So, it doesn’t work all that differently for us because we do have an enterprise version of the product and we do have a product manager who still does all those typical closed source things. What typically happens is that the features that we foresee as important to get into the product will basically go through a similar process as a typical closed source world, but then we transition them over to the open source world.

So, we’ll basically write up, you know… if we have an idea we’ll either code it first, or more likely we’ll write what we call a “request for comment” document that basically describes the change we want to make then actually put it out there to the open source community to vet. Again, because we’re sponsoring it, the things that have somebody behind them who’s actually going to write the code will typically get approved by the PSC.

But, the benefit that I’ve seen in that is that we get a lot of really good feedback, positive influence, and changes to those RFCs before they actually get coded because we have a pretty broad user community out there in open source basically providing us with some really solid feedback.

Scott: Then in terms of things like QA and documentation, how much of that kind of gets done by the community? How much of that do you guys still have to take on?

Robert: For the open source project we do no QA. The open source community is responsible for all QA. So, between the software development team itself and the open source community, that’s where all testing happens for the open source releases. We do not have any type of internal QA or formalized QA process on the open source project.
We’ll decide that we want to cut a release, and the project steering committee will release a beta. We’ll get some users testing it, providing us with bugs. We’ll fix those bugs.

Autodesk has traditionally done betas, but they’re fairly late in the game. I would like to see us actually release our betas a lot earlier than we currently do to get more of that community testing done earlier in the cycle. Then, we have more of a chance to fix bugs, and make changes before we do our final release. We’ll go through basically one or two betas and a couple of release candidates. We’ve done up to two. We may do a third on 1.2, and then we’ll release the code.

So, with open source there’s no formal QA process other than the unit test has to run successfully on every build. But basically, it’s users hammering it, whereas for the closed source project we have our rigid, internal QA process. We have our internal QA team that does a fairly formal quality assurance testing that you would expect from commercial software.

Scott: Do you feel that one way or the other is more effective at kind of wringing the bugs out of the product?

Robert: What my gut tells me right now is that a blend of the two is actually best. The nice thing about the rigid QA testing is that when you release the product, you know exactly which code paths have been tested and which ones have not. What you don’t get out of that typically is the kind of real world testing that a user will put it through. So, my current thinking is that a blend of the two approaches would be better.

Sean: Just a piece of context – how late is late for you guys? If the project has a 36-month development cycle, or 24 months, or whatever, from the initial release definition or spec process down to when you actually have people buying the product, how far along on that trajectory are you guys typically releasing betas?

Robert: In commercial software typically our first beta release is about four months before release, and that for us is late. Part of that, though, is because Autodesk has adopted an annual release cycle, so that has dictated shortened cycles for everything. But, if we could find a way to release those new features earlier for beta testers to test, I think that would be a huge improvement and a way to blend the best of the open source world with the closed source commercial QA testing.

Scott: One of the challenges people sometimes run into when they open source an existing project is that on day one when you open source it, all the jobs on the project are already filled. You already have a team of 10 or 100 or 500 or whatever number of people who are engineers employed working on the projects, and they’re already sort of doing everything. So, there isn’t necessarily a feature that’s just waiting for somebody in the community to write.

So, from this perspective that kind of caused a lot of consternation internally about what was that going to mean for the people whose job was to work on the project. It kind of created issues with the community on day one about OK, well great, what do you really want us to do? I mean, what’s really open for us to work on that you’re not already working on?
Did you guys run into anything like that on this? I’m interested in kind of comparing the experiences. You obviously have a different product, and I’m interested if you ran into similar things or not really.

Robert: I would say we ran into the exact same thing. In fact, community contribution I think has been slow. It was really slow in the first year for two reasons: one, it’s a fairly large and complex codebase. It’s not the easiest thing to jump into. Two, for that exact reason. We had developers that were doing everything, so why do I need to do anything?

That has changed, however, I think in the last…I think this last six months or so we’re starting to see much more significant community contribution. Again, MapGuide is more of an application development platform for web mapping than anything else. It’s not really a product you install and start using. You install it. You build an application. You display it to a bunch of users. You refine it. A typical web product, right? So, what we’ve seen is that a number of people have seen limitations with respect to what they can do particularly around the client end of the product. They want more flexibility in the AJAX client. So, they started with some simple submissions – really just adding some little functionality.

But, of late we’ve actually had a solutions organization that got heavily interested in MapGuide early; DM Solutions in Ottawa. Basically, what they did is they actually wrote a new client front end; a new AJAX front end for MapGuide. They are actually now in the process of donating that back to the project. So, that’s a very significant piece of code that grew out of a need. Once they started developing some applications, they thought, “Hey, I’ve got some limitations here. We could probably do something about this with our experience.” So, they started writing some code. Sooner or later they had deployed it a couple of times for a couple of their customers. Eventually they came back to the project and said, “We think this will be useful for everyone, so we’d like to donate this.”

So, I think again it’s a matter of time on those commercial projects that choose to open source. I think eventually the time will come when developers will come onboard. But it takes a little bit of time for that exact reason. Initially, what do we need to do? Don’t know.

Scott: Well, it sounds like you guys are doing it the right way. I mean, the other reason sometimes why companies open source something is just because they want to sunset it to some degree. They’re done with it. They’re tired of it. They don’t want to put people on maintaining it, so they just say, “Here, we open sourced it. Community – go take it from here.” Those ones usually just kind of die pretty much on the spot.

Robert: Well, for us there’s some interesting history here you guys should probably know that I maybe didn’t introduce early enough for you. MapGuide was first released as an Autodesk product in 1996-1997. About three years ago we took on a complete ground-up rewrite. We were about halfway through that effort that we decided to open source.

So, we basically have a legacy product there in MapGuide. We started a complete rewrite project. We actually completed that complete rewrite project. That’s what we open sourced. But, halfway through it we decided the right thing for the future of this product was to actually open source it and we intend to continue maintaining and being a huge part of the development community on that project, but at the same time getting some open source involvement.

Scott: Yeah, that seems to be key. It seems to be key when the company that’s open sourcing it is going to continue to invest in it – that really seems to be a key to success. You know, companies seem to open source for a lot of different reasons. Sometimes it’s because they feel like maybe they’re not in the best position to do a certain piece of work – like if it requires porting to a different platform or architecture – or they feel like the community will have great ideas that they won’t generate internally.

Sometimes companies do it as a market disruptor. They see that the market is saturated with closed source, proprietary solutions, and by being the first ones to open source they can radically change the market. For you guys, what was your motivation in going open source? Why did that make sense for this particular product?

Robert: Well, I’d be lying if I said market disruption wasn’t a huge factor, because it certainly was. What we recognized halfway through this project… I mean, again, it’s basically a piece of web mapping technology. It’s basically a web based GIS platform. So, what we realized about halfway through… We lifted our heads up and looked around. We saw that web-based GIS was growing in popularity by leaps and bounds and ESRI is pretty entrenched. They are a commercial competitor. All of our other competitors have products in this space as well.

Then you’ve got Microsoft. You’ve got Google and Google Maps. You have Yahoo! Maps. There’s a bit of a commoditization around web mapping that we discovered. What we really realized is that the revenue opportunities for us are not in the platform per se, but more in the solutions that we can deliver on the platform. Does that make sense?

Scott: Yeah. So, by open sourcing the platform and being still heavily invested in it, you’re in a great position to provide solutions on top of that platform that might be hard for your competitors to match.

Robert: Right, because we can influence the platform’s direction as well as direction of our solution providers. Again, we’re a company founded in establishing developer partnerships. It’s really partners that develop solutions on our product.

All of our partners now can influence the direction of the open source project by either: a) actively becoming involved in development on the platform; or, b) hiring companies that specialize in open source development to go enhance the platform in some ways that they never could when it was a closed source project.

Scott: So, how does it work? Has open sourcing it disrupted the market and provided a lot of the benefits that you guys were initially hoping it would?

Robert: It’s definitely been disruptive and that’s beneficial. The larger benefit for me, though, is the fact that our solution partners now have an opportunity to influence the new platform in ways they never have before. They get to see things coming early. Typically we release the open source version two or three times a year, so they are getting features faster and they can start to work with them.

On top of that, they can influence things. The idea of this whole AJAX front end is a great example of that. It’s where a solutions development partner for us basically came in and developed a whole new client framework that’s going back into the platform.

Scott: Have you guys at this point had to contend with anybody who has wanted to do a really strong fork of the source? How have you prevented that or dealt with that?

Robert: I don’t see anybody wanting to do a fork. It’s LGPL licensed. That helps a lot in that respect.
My crystal opinion on that, by the way, is that if you have a project that has a good community behind it, that is a cohesive community, I don’t think you’ll see a fork.

Scott: Yeah, it seems like there’s definitely a high bar. Forking is the nuclear option. For somebody to fork and have it get traction, they have to be making at least the same amount of investment in it as you guys are. That would be a pretty high risk thing to take on.

Robert: Right.

Scott: It seems like as long as the company is really meeting the needs of the community and supporting the community in the right way, it doesn’t really happen. But Netscape is an example of where it did and the outcome of that was Firefox. People really felt like the main sponsor was really not taking it in the right direction and they had a better idea.

Robert: Exactly. Let’s be honest – that’s one of the reasons that the project steering committee, who really do have control, is actually four community people and only three employees. We can always veto something because it only takes one vote to veto. But, in my experience so far, working with this committee in the last 18 months, I don’t see us ever actually using a veto.

Scott: Great.

Robert: In communities you can suddenly get a rift with the company that wants to maintain control, or sometimes you get a rift in the development team itself. Drupal is a good example of that. They had a rift in the middle of the development team, and that resulted in the fork of Drupal. I don’t think that’s a common thing.

Scott: Right.

Robert: If you have a good, well-oiled community, I don’t think that’s likely to happen in a company that’s open.

Scott: Great.

Scott: This was fantastic. This was really informative from our perspective. It’s nice to be able to talk to somebody who’s got so much experience in both realms and can provide some objective insight about their experiences on both sides. So, this was really great for me.

Sean: Yes, same here. And Bob, we’ve extended this courtesy to everybody. We’re just curious if there’s anything else you want to get on the record or another question you’d like to propose, or things along those lines before we wrap up?

Robert: No, I think the only thing we really didn’t get to was one of your questions earlier about QA and documentation; I didn’t touch on the documentation piece. I would say that’s one of our biggest weaknesses with open source right now was that we really don’t have a solution around documentation in the community right now.
That’s something that I think will still take time to formulate. Everybody is busy and nobody has time to write documentation. That’s actually my experience as one of the issues with open source projects.

Scott: Unless you’ve got a sponsor who staffs that, it’s just not the most fun stuff to work on.

Robert: Right. So for now, most of the open source documentation has been staffed by Autodesk. But that’s getting harder and harder for us to continue. So, that’s one of the places that I am, if anything, a little concerned about for the future; the documentation and trying to get a way to continue to staff it.

Scott: I don’t think that’s unique. Part of the problem is the people who really, really work on it, don’t really need the documentation. So, they don’t necessarily feel the need as much as the people who aren’t as active in the community itself.

Robert: Right. But the problem is that it’s hard to get new users on board when you don’t have good documentation. That’s really the challenge.

Scott: Right.

Sean: Well, Bob, I really want to thank you for taking the time. I couldn’t agree more that it was a great interview. It was really, really nice perspective we got.

Bookmark this: