How Software is Built

Interviewers: Scott Swigart and Sean Campbell

Interviewee: Mike Shaver

In this interview with talk with Mike Shaver, Chief Evangelist for Mozilla. In specific, we talk about:

• Governance of the project and how people get involved
• Changing contributors who fill key roles
• Roles of independent contributors versus employees
• Ensuring high usability in an open-source project
• Protecting reliability in a world of many extensions
• Tools used by the Mozilla team
• Reflections on open and closed development models

Scott Swigart: There are a lot of different governance models for open source projects. With Sun projects, for example, they often have it written that control of the project has to rest with Sun employees. Apache has a different model. Apache projects have to have maintainers who are from at least three different companies so that no one company can hijack a project.

Talk a little bit about how the Firefox kind of governance fits in that spectrum.

Mike Shaver: There are some pieces of our model that are quite similar to Apache’s. For someone to get commit access to the source tree, or become an owner of a section of code, people need to vouch for the person, and not all those people can be from the same company. We’re not as concerned about it as Apache is though.

I think one of the reasons that’s the case is because a good number of our contributors are employed by the same company, the Mozilla Corporation. But our role in the corporation is very different from that of a traditional software company. We’re not doing anything other than working on this code. It’s not just part of our business to build the Mozilla project; it’s all of what we do. It’s what we’re chartered to do.

So we’re structurally set up as an organization to be aligned with the project, which means we don’t really have the same kind of risk. No one’s thinking IBM’s going to run off with Mozilla if Mozilla doesn’t have the defenses in place, and they’re big contributors.

The Mozilla project is very healthy, obviously, and a lot of the people who do come into our employment relationship come into that relationship because they’ve been successful as Mozilla leaders and developers.

Scott: Talk about that. What is the way that people advance through the ranks? Because, again, that’s something that varies from project to project. You guys have terms like “sheriffs”; there are special security people, and you’ve got other contributors. How do people get their toe in the water with Mozilla and climb all the way up to the top of the food chain?

Mike: It certainly isn’t necessarily a linear progression. There are some of the things you described, like being a sheriff, that’s really more of a role than a rank. People are sheriffs, meaning that they’re responsible for watching out. They’re on integration watch for a given day, watching as things come into a source tree, making sure that our tests all stay in line, helping get the right people on problems that come up in integration.

Most generally, people get involved with Mozilla initially by scratching an itch of theirs. So we’ll see people come into the project, on the technical side certainly, with a patch or investigation leading to a patch for something that they care about. And sometimes, that’s where the level of involvement ends; people can pop in and out, they only have that one itch to scratch, and it’s scratched now, so they can go back to doing whatever it was that bumped them into that problem. But some people do really “get the bug” and stick with the project.

Generally, the path to getting more privileges in the project is not extremely well formalized. Some pieces, such as access control over the source code repository, we obviously take very seriously, and there are a number of nomination and review processes that happen there, and similarly on the security group side. But for the most part, people’s influence comes from the respect they have from other members in the community.

Whether they’re employed full time, or they’re grad students who can spend a couple of hours a month working on something, if they’ve demonstrated that they understand the technology and Mozilla’s values, those are the things that lead to people becoming more influential.

Most of the time when we have somebody move into a role of ownership over a code, which really means that they’re there to settle disputes and break ties about which way a given patch should go or decide whether something’s ready to check in or not, it’s usually a nonevent within the community. It’s something where it’s obvious that the person’s been acting in that role and providing good leadership and guidance in that capacity. It’s more a matter of formalizing it so that the people who aren’t as familiar with the project know whom to go to and what the effect of the person’s decision is.

Sean Campbell: When we were at OSCON, I watched some guys from Google who are very involved with Subversion. They talked about managing the community, and they talked a lot about how people grow through the community, but then they also had an interesting conversation about – for lack of a better phrase – how you demote somebody.

I think whatever process a project uses, it often comes to the test not when people are being promoted, but when someone has to be removed from a position.

So tell me a little bit about that.

Mike: The most common case is that somebody acts as a bad owner, simply because the person is not invested in it anymore. The person is not spending a lot of time on it and has moved on to something else in their life. And in that case, the news is really more about someone new becoming an owner.

In a couple of cases, we’ve had to actively seek out a replacement, and that tends to come from cases where the owner in question is not thinking or acting with enough breadth of interest in mind, where the person’s specific needs for that code are being met, but it’s not integrating either technically or, in some cases, according to project values at a higher level.

Sean: So people lock too much into their view of the feature and can’t really back out of that.

Mike: Yes, exactly. I mean, there’s taste and judgment involved in the role of anyone who’s a committer, but sometimes someone can go too far.

Sean: Right, right.

Mike: You often have to trade off an old capability in order to get a new capability, or to fit into a new model. And we had an example, to speak with some generality, where there was someone who was an employee of a company and was an owner of a piece of code that was not widely used in the project but was used by a number of downstream projects.

And the patches from this given company were getting checked in, but we went to look at some of that stuff some months later, and there were questions about it. Did this stuff get reviewed well enough? Did it get reviewed in light of the way all the other people are using this code, and not just the people who were most active in the project at the time?

We actually did end up replacing an owner there. It wasn’t really acrimonious. It’s not like the person was, you know, grabbing the door frame and screaming on the way out.

Sean: [Laughs]

Mike: To them it’s sort of old news. By the time you’ve gone through a review like that, and it’s obvious there was a real falling down on the part of an owner, something needs to be done. But it was really fine in this case actually, because even in talking with the company in question and saying, “We’re finding a new owner for this code because the stuff that came in really didn’t go through the things it should have. It wasn’t treated with same breadth of analysis that we need in order for our code to be useful to a lot of people.”

And they were OK with that. The reason they were involved in the first place is because they like the quality of the code and the nature of code that comes out of that collaborative process. So rather than it being a source of tension between, if you will, the core of the project and this company that had some contributors in it – they were glad to see that this code that they obviously care about was going to be that much better owned.

And as far as I know, it didn’t have repercussions on the person’s employment or anything. It wasn’t that the person was hired because he or she was the owner, or that kind of thing, which is good.

But in this case, it turned out really well. And it was an opportunity for us to really bring another development community into not only our project, but also our values. We’re not doing this in a punitive way; it’s because we care about this code, and it feels like it’s not getting the love it needs right now. So we found some people who have other history with it and who are willing to take a different view on it.

The new owner gets support from other people who have been around longer and in ownership roles longer. It’s been pretty positive.

Scott: One of the things I’m curious about – because there’s a Mozilla Foundation with employees – What are some of the things that can be done only by employees vs. people in the community?

Mike: In our view, if something has to be done by an employee, generally that’s sort of a bug. We can’t always afford to fix those bugs, in the sense that it’s more disruption for purity than the value you get at the other end. For example, we do partner distributions with different companies. We need to start spinning that stuff up with them before they are ready to announce it, so employees are involved in things like that.

We have some contractual relationships around revenue that for legal and other reasons need to be performed by employees. Someone’s got a legal relationship and can preserve whatever the legally required privacy and secrecy stuff.

But generally, when you look at technical tasks, when you look at things around how the software’s produced, or reviewed, or determined to be released, I struggle to think of anything where employment is a requirement.

In some cases the role is demanding enough that full time commitment is really important. But that doesn’t necessarily mean that they need to be employed by us. An example of this is Firefox. When we first started out, the lead of Firefox at the time, Ben Goodger, switched from being a Mozilla Foundation employee to being a Google employee. And he continued to be the owner of Firefox for some time.

He isn’t anymore; he’s moved on to other things. And Mike Connor stepped into those shoes quite ably, and Mike is an employee. But if someone else was paying him to do a good job on Mozilla, to contribute full-time and keep up with it, that’s the real condition.

Where we have cases that don’t meet that criteria, where you would need to be an employee to do certain work, we’re very tentative about undertaking that kind of work. Scaling through the community, not just technically but in terms of organizational resilience, is really important to us.

One of the luxuries of being a part of a nonprofit is that we’re not held against next quarter’s revenue numbers. We have only one shareholder, and they aren’t in it for the money. So, we don’t have to produce a return there. We’re really about making sure that we are creating software and a software culture that can last 10, 20, 70 years.

Where we have an opportunity, we try to bring people in who may not be able to commit full time to work on this. They may have other things in their lives, or they just may not be that interested. They want to spend some time on this, but not make it their lives. Those are opportunities for us to really diffuse ourselves through a broader part of the software industry.

And those kinds of things give us, as I say, great resiliency. People sometimes switch from that part-time mode and want to work on Mozilla full time. We get some great hires and full time contributors that way. They bring Mozilla’s values into other environments: around transparency, around support for standards and universality of technology.

And they also bring to us their own perspectives on software development. You know, “This is how we do it at Big Co.” Or “This is how we do it in this tiny little agile Web development shop. Here are some things we learned about how to test, how to build tools, how to instrument things.” The conversation around software development, in some ways I think, is even more important to us than the specific software.

Before we had Firefox, we had the Mozilla Suite, which for its many wonderful attributes was not nearly as successful in the marketplace, as I’m sure you’ve seen. But the project that created it is very much the same. And that we were able to weather having basically no market share for five, six years but still built a culture of software development and a culture around that software – of marketing, of user support, of design and tools, and so forth – is what let us continue to be there until we found our Firefox.

And it might be that in five, 10 years, we’ll need to find the next Firefox. So we need to be about the process that leads to it, as much as we are about the artifact.

Sean: One of the things I like to ask people we talk to is, “How do you ensure usability in an open source project?” We get interesting responses, and you guys, in particular, stood out to me, the same way OpenOffice did, because you, even more than OpenOffice, have the “grandmother is downloading in Iowa” problem.

Mike: Yep.

Sean: It just has to work. It can’t get in the way. I’m curious, because when I look at the new features page for you guys, it’s very much written in a “to you, the developer” tone. But at the same time, I think everybody can look at open source projects at times and say, “How usable is this by a nontechnical person?”

There are a bunch of questions here, but I’ll start by asking how you get in a feature that you know users really need, but no developer is clamoring for it, and maybe it’s not really that sexy of a feature to write.

Having written code, we all know how that works. How do you motivate the community to add in that feature? Do you have examples of stuff that you guys have added to Mozilla that maybe the dev community wasn’t clamoring for, but you got 50,000 user requests for?

Mike: So there are a bunch of interesting questions there for sure. I’ll take them backward. One of the best ways to motivate the developer community at large is to go get yourself 130 million users.

Sean: Right. [Laughs]

Mike: Because the things that are not sexy – like, I’ve got to go figure out why this one plug in crashes when I’m running on Panther and I have Quicksilver installed – that’s not necessarily a sexy technical problem. But it can be exciting to fix something that’s going to affect “only” a couple of hundred thousand people.

Sean: Right, you got all these Quicksilver users. And so you can look like a hero because you can say you solved this problem for them.

Mike: It’s true that at some point you have problems that affect a small enough subset of people that it’s not economical to have somebody work on it. All software, propriety, open, whatever, has this class of problem – whether it’s technical or usability related.

But as you broaden the pool of people who are exposed to your product, and as you make it easier for people to move from being users to being followers – I mean followers of the project, not followers in the cultlike sense. Some people contribute by helping test. You want to increase the chances that somebody who cares about that problem is going to be able to connect with someone who can fix it.

And the motivation piece is helped a lot by the fact we have a really direct connection with our users. You’re not seeing a ticket trickle down through tier 1, tier 2, tier 3, support escalations, and then a big “what do we fix in this bucket” release.

Sometimes we do that. We look for high-profile crashes or bad plug in interactions. But often you can read the bug report and the person who’s hitting that problem is right there. It’s a person whom you are helping, and it’s not just a software artifact.

Now, going the other way on usability issues, that can be a trap, right? We need to write software that’s usable not just for one person, but usable for 130 million, or 150 million, or hopefully a quarter of a billion people. And that’s a hard problem.

I think it’s a problem that’s generally hard for software that scales. I think it would be very hard to do that well at our size without being as open as we are. I would venture to say impossible, but I try not to use that word very often.

Here’s a good example. There’s a thread that’s flaming away right now on one of our application development forums. We’re making some changes to bookmarks in Firefox 3. This is pretty much the first time we’ve changed how bookmarks work since about 1994. But we’ve learned a lot about how people use them. The Web’s bigger now, you need to track more on it, and so forth.

Sean: Sure.

Mike: But a lot of people, especially those who are more technically advanced – they’ve been using bookmarks since 1994. They have muscle memory. They have whole parts of their reptilian hindbrain devoted to how bookmarking works today.

Sean: [Laughs]

Mike: So, making changes there – even when we have strong belief and as much evidence as you can have that it’s going to improve the lot of users – I think it’s going to be good for users and be below the threshold of “It’s too much to learn how to use this.” Even when we have that, there’s still a very vocal portion of our community that’s conservative.

Which is good. That conservatism helps us in a lot of core technical ways. It keeps us from getting ahead of our users. We don’t want to have the Apple Newton problem, where we have something great but no one really knows what to do with it yet.

And that’s a hard tension on the usability side, for sure, between how the Web works today and making it work better. And also, helping users develop new habits and patterns for managing information, for understanding the security characteristics of what they are going to do.

Another good example of this is the anti phishing features we have in Firefox 2. With that feature, you can still click through to the site. But because we flagged it, now a different part of my brain’s working, and I’m being a little more suspicious of this site. Or maybe I’m scared enough that I’m going to ask somebody who knows.

In Firefox 3, we have another kind of protection against sites that have been identified as trying to install malware on the user’s system, and you can’t click through that warning. With phishing you could look at the page and still decide, “I’m not going to put any data in this. I want to look to see how the phishing attack works.” In the malware case, by the time you’ve loaded that page, we’ve lost. We can’t undo that action.

But Firefox is all about the importance of choice, and the importance of putting users in control. That makes it easy to throw stones at this new behavior. We have people who say, “I know better, I should be able to click through.”

But ultimately, we’re responsible as the agent of the user. As we’re building this software, we want them to be able to use the Web in a more reliable and trustworthy way. We’re saying, “If the site’s in this small list, no. Just don’t let the user through.” If you want to, you can disable that feature. You can disable malware protection entirely. And if you are technically advanced enough to do that, then we will give you some rope and try not to watch too carefully what you do with it.

But, for the vast majority of users, providing a click through path would make the product less usable, even though it would be more palatable to some people, especially those who know most about security in our community. And these are discussions that, for better or worse, we have in the open. This is where our process happens. You can see how the sausage is made.

Those cases can be somewhat distracting for us. I think in a more closed environment, where you have good designers, as I very much believe we do, they can get together in a room. They can figure out what the problem space looks like. They can design some good solutions and iterate them.

In our case, the bar for those features is that much higher, where it needs to survive feedback from anybody who wants to give it. Which isn’t to say all feedback is necessarily followed. Even we have bad ideas; other people have bad ideas. And we do have to make hard choices about trading off different capabilities.

But you have to be able to have at least a thick enough skin to go through that process and see all that feedback, which tends to be negative, because if you like something you just kind of say, “Eh, I won’t say anything,” unless you are really excited about it. But if there’s anything about it you don’t like, people jump on that stuff. Stuff has to make it through that process.

I think for us, it’s made us more conservative about adding new user facing features, which I think is good in a tool as ubiquitous as the browser. But it also taught us a lot about how to make the features we do add understandable and easily usable by a really broad range of expertise. So I think the product’s much better for that.

OpenOffice is exactly the same sort of case. And your reference to grandmothers downloading in Iowa was the reason that, for us, the whole of the installation experience is really important. We try to keep the download size small. I think we’re just over five megs now, because we know there’s limited time between “I think I’d like to try this Firefox thing” and “Oh, dinner’s ready.” You come back from dinner and never notice that you downloaded it somewhere in your huge downloads folder, and you’re never going to try it.

Scott: Right. Right.

Mike: Keeping that loop short is really important. When you use Firefox for the first time, it pulls in all your bookmarks, it pulls in all your history, all your saved passwords, et cetera. Because you want it to be really easy to start using and be really easy to stop.

You can go back and use IE if you want, because we don’t want you to have to feel like you’re committing your Web to us forever.

Scott: I mean one of the things that you guys obviously have to focus a lot on is reliability. And like a lot of open source projects, you have this modular, pluggable design. And we found that a lot of times that’s really important for projects to be successful. It’s important on Apache that even if people aren’t maybe up to the level they need to be to work on the Apache Web server itself, the barrier to writing a module is pretty low, right?

Mike: Yeah.

Scott: And Firefox, you have the same kind of thing, right? You might not be working on the browser itself, but you might want to try to write an extension to it.

How do you guys architect Firefox to help keep bad extensions from crashing the browser? How do you, for lack of a better term, keep bad code from making your good code look bad?

Mike: Yeah. Not well enough, I think. People using Firefox sometimes load it up with a pile of extensions and then saying, “Performance isn’t what it used to be” or “I get this one weird crash.”

There is an important trade off there, right? We could certainly restrict the attachment surface for plug ins. We could say, “We are going to run you out of process. We’re going to run you in this safer way. We’re going to give you five things you can do to the browser.” And we could become much more robust that way.

Or we could say that you can run arbitrary code. And it’s up to the users to determine whether the given extension is going to be good enough for them or not.

We err, and to some extent I guess it is erring more on the side of permitting the extension developer to do whatever we could do in the product. And there certainly are challenges that come along with that, from helping the users understand that when they install an extension that they are trusting the extension code to do whatever they trust Firefox to do.

But also we try to help developers understand the impact of what they’re doing on Firefox. People write extension often because it’s something that they are going to use, but it’s also something that could be used by any of these 130 million people.

And an individual extension developer may not have access to the same breadth of feedback and breadth of testing that we have across Firefox itself. And we have a quarter of a million people using just our most recent Firefox 3.0 beta right now. Many extensions don’t get that kind of coverage and that kind of detailed analysis by a large group of people.

What we generally do is try to make the things that people want to do with extensions easy and safe. And so in Firefox 3.0 for example, we have some utility libraries we’ve built in, which we call Fuel, which make it easier to work with bookmarks, make it easier to work with tabs and history and so forth, because we were finding that a lot of extension authors were spending a lot of time on that boilerplate. And we want them to spend more time on their cool idea.

But also they were doing it in ways that made it hard for us to make changes later on without breaking their extensions, or they were doing it in a way that interfered with other extensions or interfered with performance.

So we tend more to deal with that class of problem by encouraging better behavior rather than preventing worse. We are certainly interested in looking at models for sandboxing of content, for having extensions that can run all over our privilege mode. They may not have access to as much data or be called in a different way.

But one of the most valuable parts about the extension model for us isn’t just what other people can do with it. It’s that it is a proving ground for our own ideas.

You see stuff coming out of the labs. You see stuff coming out of individual developers or people who are out in the extension development community, if you will, who have a great idea, like session restore, or some of the new changes made to tabbed browsing in Firefox 2.0, and a bunch of stuff in Firefox 3.0 as well. We can look at those extensions. We can say, “Hey, if we want to put this feature in Firefox, we can find out on a pretty broad scale how people are going to react to it by getting 100,000 people to install the extension.”

Sean: Right.

Mike: And that gives us the kind of feedback that you can’t get through even the most passionate online conversation, right? You can put it in front of people and see, in a classical usability test sense, or through instrumentation, or just through surveys afterward, is this a better behavior? What are people blogging about it? What are people using? Are they leaving it installed? And that’s a really powerful channel for us.

Scott: Some applications, and maybe Firefox already does this, will sort of shame the extension developer a little bit. You’ll start up the application and it’ll say, “Hey, last time I was running, this extension crashed me. You’re going to have to manually re enable it if you want it to load.” Is there any kind of architecture like that or is that something that you guys have looked at?

Mike: So we do something similar for session restore, for example. If the browser crashes and you start it up the next time, it says, “You can try this session again, but if it was something you were looking at that crashed you, I can predict what’s going to happen.”

Scott: Right.

Mike: One of the things that is challenging about doing what you are describing for extensions is that the way extensions really work in Firefox is by sort of mutating the application itself. In some cases we could tell that a given piece of code was running. We can look at some kinds of crashes and say, “Ah, well, we don’t normally take that path and this extension did, so that’s likely what’s happening here.”

But in a lot of cases, the way extensions work are by taking parts of the applications that aren’t connected together and connecting them to each other. So they do their damage, but we’re running Firefox code at the time of the crash. One of the things that we have done in Firefox 3.0 is use a new crash-reporting system called Breakpad, which was developed by us and Google for a couple of reasons.

One is, the previous one we were using wasn’t open source. It was the only non–open source code we were shipping and the source of much heartache for that reason. But the value of those crash reports was just way too high. It improved the product so much. But it also didn’t work, for example, on Intel Macs, of which as I am sure you are aware, a couple have been sold.

Sean: Yeah, just a few. [Laughs]

Mike: Just a few. So we have this new crash-reporting software. It also handles threads, which is something that we have been using for a long time, and the crash-reporting software from the ancient Netscape days really didn’t handle threads very well.

We get better crash-reporting data. And one of those things we do report with crashes is what are the extensions that are installed. So we’re able to do correlation that way and then do outreach to authors. And they can then see: this is what these stacks look like. We are able to work with them with my team, specifically in evangelism and developer relations, to say, “Hey, this thing you’re doing is breaking stuff. Let’s get it updated.”

One of the nice things also is that we have spent a lot of time making the extension update model for Firefox pretty smooth. You can push a new update out and it’ll get to users. You don’t have to make sure they all come back to your site and re download and run an installer. It checks for compatibility. It does it in the background.

And so when we do have a fix for something like that out, or more important, when an extension developer has a fix for that kind of thing out, we can get it to users pretty quickly.

Sean: Cool.

Mike: And that’s something we are looking forward to a lot as Firefox 3.0 rolls out, being able to become even more responsive and agile not only with our own product, but with extensions as well.

Scott: Let me ask a question in a different area. There was this huge news story a while back where Coverity had promoted 11 open source projects to their Rung 2. Coverity didn’t make these claims, but the way the news reported it was almost like, “Hey, here is the silver bullet. Coverity’s product scanned the code and found all the defects. You fix them, and you have a secure product.”

Talk a little bit about how tools like Coverity’s factor into how you guys make a secure product. And then what’s the story beyond that? What are the limitations of code scanning tools? What do you have to do beyond that to ensure a secure product?

Mike: Yeah. So certainly we are engineers and we like it when computers can do work that lets humans not do that work. So we are all about tools of various kinds. And certainly security standards are no exception to that.

Our experience with various source code checkers, and I’m not sure what our most recent results with Coverity were, is that maybe they’re not as useful on our code because we are a project that uses a lot of C++ where most of the open source projects are still largely architected in C. Or it could be because we have a very dynamic architecture. You can’t always tell how the pieces are wired together without understanding it or running it.

Generally, the level of false positives is incredibly high. The number of possible vulnerabilities that are reported as “This could be a problem, though it may not actually be one” are vastly smaller. And the number of actual vulnerabilities that are reported are quite small.

And we do act on those where we see them. But I think if you look at it, the Coverity “rungs” are really ratings of how responsive you are to Coverity reports. They’re not really certifying any absolute level of security in the code.

Scott: Right.

Mike: Which is fine. Responsiveness in security is very important. But what they are really saying there is that these groups have responded to all the problems reported to them.

Well it turns out that’s the way they call it. And that’s certainly something we could invest in. We could go in and make sure that we tweak our codes so they don’t trigger any false positives and so forth. But in our experience, that doesn’t correlate with improved security of products.

Which isn’t to say people shouldn’t be using these tools. I think to say, “You know that there were false positives from Mozilla, so we’re not going to bother looking at it” is pretty reckless. Each project needs to look at the tools and figure out which ones are going to pay off best in terms of improvement to security of products.

We do our own. We released last year at Black Hat a set of tools that we built for fuzzing various network protocols and JavaScript language, which we had given to other browser vendors as well, ahead of that release. We found that those helped find a much larger set of bugs and potential problems in our code than the traditional sort of static code scanners.

Where we do use tools similar to Coverity, they are actually ones that we are building in-house and when working with researchers on projects like Elsa. This will let us do static analysis and transformation of our code. And we have a huge code base, and what we really want to do is be able to update the patterns in it as we learn about new vulnerabilities or performance patterns or portability or take advantage of new language features.

When we started to build Mozilla in 1997, C++ compilers were not nearly as capable as they are now. So there were a lot of features we just couldn’t use if we wanted to run on everything from Win 32 to AIX to OS/2 and so forth.

Scott: Right.

Mike: So we can take advantage of, for example, C++ exceptions in a way we couldn’t before. We are writing tools to help analyze our code and also to write big chunks of the code.

Sean: Interesting.

Mike: So that’s where we are putting our investment in terms of the tools. But I think those that are trying to make a secure product and aren’t doing everything they can to help use the computer to find issues, not as a substitute for human analysis but as an adjunct to it, are doing themselves and their users a disservice.

Scott: One of the things that we do a lot of the times at the end of these interviews is hand the microphone to the person we are interviewing and say, “We covered a lot of stuff that was interesting to us, but what’s maybe interesting to you that we didn’t specifically ask about?”

Mike: Yeah, I think the whole series you’re doing here about the software processes that are different – well, we’ll see whether they are different or more similar – between open source and proprietary model, I think is really interesting.

And I think it’s going to be interesting to see how those differences in process manifest in terms of users affecting things, whether it’s agility of features or improved responsiveness on bugs, or security issues, or days of vulnerability. Whatever we decide we want to optimize for in a given path.

But one of the things that we are doing with Firefox – and we hope with the Web – is making it a more robust and effective way to build an application. So it’s interesting for me to reflect on the questions you’ve asked and look through the various blog posts and so forth.

The kinds of processes that we want to encourage there, the kinds of things that we want to make easier for developers or make their results more effective, is how can we bake those things into the Web?

And we are certainly very active in various standards communities. We are very active in working with language developers and Web application developers and other browser developers to be sure to have these capabilities. But doing them in a way that doesn’t require the forklift upgrade of the development process.

Like, you don’t have to throw out this app if it has everything you need except video and rewrite it in something that has video. We want to take the barrier points away. But I think we will do well to keep in mind the development processes that lead to those apps as we do that, and make sure that what we are doing builds well for people who are using traditional proprietary methods, whether they are doing more collaborative development or whether they are trying to find a hybrid, as MySQL did.

And it looks like it’s worked out well for them, based on the acquisition announcement.

Scott: [Laughs] That’s right. Mike, thanks for taking the time to chat.

Mike: Thank you.