In this interview, Scott Swigart, Sean Campbell, and Richard Bowler interview Michael Howard, a senior security program manager in the Security Engineering team at Microsoft, and an architect of the security-related process improvements at the company. He is the co-author of many security books including the award-winning Writing Secure Code, 19 Deadly Sins of Software Security, The Security Development Lifecycle and Writing Secure Code for Windows Vista.
In this interview, Michael exposes how Microsoft developed the Security Development Lifecycle, which has decreased the number and severity of vulnerabilities in their products. Michael also directly challenges the notion of “many eyeballs” leading to secure code. Highlights include:
Richard: Michael, thanks for taking the time to chat with us. I’d like to start by getting your take, if it’s applicable, on the advantages of open source or closed source in the context of security.
Michael Howard: I think to actually describe it as closed source or open source is really a bit of a misnomer. Really, it’s who controls the process and what the process is. And so, the implication is that closed source tends to have more of a documented process. Whereas the open source guys, perhaps, don’t. I’m not saying that’s true either way.
But if we’re looking at it from a Microsoft perspective, I think one of the advantages that we have here — a big, big advantage — is that we can mandate policies. “You will do this, and here are the criteria that you must meet before you can ship some software.”
The joke is, if you want to be involved in the Microsoft Salary Participation Program, you have to do this stuff. When people say to me, “How many security people have you got working on Vista?”, the answer is that it’s 6,000 people, because everyone has got at least a baseline level of security expertise. We enforce through education, and through the use of centralized tools and that kind of stuff.
With the Security Development Lifecycle (SDL), there are things that you must do. You can’t write code that uses banned functionality. And so on, and so on, and so on. We can enforce that.
I think that’s a big advantage that we have.
Richard: I think that’s the general perception, that process is more enforceable in a proprietary source environment than when you’re dealing with individual contributors in different locations.
Michael: I’ve talked to a few open source guys about what we do here and their comment was, “Well, that’s just draconian and stifling creativity.”
My response to that is, “No, that’s just discipline.” And fortunately, I think the days of software being written by cowboys are behind us. When I say “us,” I mean the industry, I don’t mean Microsoft, I mean the industry as a whole. I mean, you need discipline. And that’s what the process gives us.
Richard: So, about the SDL, can you give us some idea what gave birth to the idea of a set of security-specific processes blended with the SDLC seed that was already in place?
Michael: Five or six years ago, it was obvious to us that software development processes, regardless of who the vendor is, just didn’t give you secure software. We saw that by the sheer pandemic of security bugs across the industry.
Therefore, we needed to change the process, and we had three possibilities. One was to hope that the problem would go away, which was obviously not going to happen. Another one was to completely rip out the existing process and replace it with something brand new. Well, that’s not going to happen either, because, at the end of the day, Microsoft is actually good at shipping software. You can argue about whether it’s on time, that’s another matter, but the company as a whole is good at shipping software. That’s what we do. So, let’s not change that. The third option is taking some existing process that does ship software, and augment it with security requirements.
That’s what really led to the SDL, which is ultimately just taking whatever process you currently have, and layering security requirements on top, with security-related deliverables along the way.
If you have a waterfall model or a spiral model or an agile model, it really doesn’t matter. SDL layers on top of your existing process.
Richard: It seems that Microsoft is being a little bit evangelistic about the SDL, trying to convince others to put it in place in their own development process.
Michael: Well, actually, the funny thing is, to answer your question directly, yes. But to give you the more long-winded version, we’re actually being asked by a lot of companies as well. “What have you done?”
And I think that’s really interesting to see. It means that 1) people are actually seeing that we’ve made some progress and 2) they also recognize that they can possibly make some progress too.
So, it’s actually both. We would like to just go out there and evangelize the stuff, but frankly, we’re also being asked the questions as well, which is absolutely wonderful.
Richard: The SDL seems focused on process to limit the number of security bugs that get written in the first place, instead of the entering a test and fix mode at the end of the process. Can you talk about the relative success of that goal?
Michael: Actually, there are really two goals of SDL. Goal number one is to reduce the number of vulnerabilities in the code. And goal number two is to reduce the severity of what you miss. You’re never going to get all the vulnerabilities, ever.
The thing that sets security apart, more than other disciplines, is that the science is constantly evolving. The person on the other side of the table wants to take you down and will do whatever possible to take you down. It’s a very, very interesting field of expertise.
Getting back to the first goal, which is reducing the number of vulnerabilities in the code, there are really two parts to that. One is essentially going through all the code and just removing vulnerabilities.
For example, we went through the entire Window’s Vista code base, looked for banned function calls, and removed those banned function calls from Vista.
It doesn’t mean there was vulnerability there, but one thing I can tell you is that by removing them, if there was an undiscovered vulnerability there, there’s a good chance that it’s gone now.
That’s the first aspect, just going in and cleaning up and re-reviewing code that may have been written 10 years ago, when the threat landscape was very different. The other aspect is to not add new vulnerabilities to the code. That’s why we have education; that’s why we have tools; that’s why we have bug testing. The whole point is to catch problems before they are added to the code and put into the software.
And, frankly, one of the biggest parts is just education, making sure people are aware: “Don’t do this, instead do that, if you’re writing this type of code. These are some of the assumptions that you need to be aware of about the attackability of your system. Here are the things that you should be aware of.”
Now, that by itself, just raising awareness and holding people accountable for it, exponentially increased the quality of the code. That includes producing fewer bugs. I’m not going to say that we’ll ever get to zero, I’m not going to go there at all. But we have seen a huge improvement.
Scott: I guess, talk a little about that. What, quantifiably, have you seen being the results of SDL?
Michael: The cynic in me says I can always point to a pathological case. I can always show pathological data that will prove that anything doesn’t work. It’s really about a bell curve of products. Some products have done exceptionally well; some products still need to make improvements.
[One of the] products that have done exceptionally well is our Web Server, IIS. When you look at IIS6, it has had two security bulletins, and both of them are in components that are off by default.
Now what is fascinating about those two bugs is both of those bugs also affect IIS5. In IIS5, the vulnerable features are on by default. In IIS6, they’re off by default. One of the goals of SDL is to reduce the severity of what you miss. Well, both of those bugs are less severe in IIS6.
That is one example. Another good example is SQL Server. In Service Pack 3, they really went to town on that product, security-wise. And we have not issued a security bulletin in the SQL Server engine in over two years.
Last year, in the “Month of browser bugs,” IE7 came through completely unscathed. I can’t say the same for IE6 and IE5, but IE7 came through completely unscathed. And that is because of the security work we did. And I can attribute that completely to SDL. You look at Office 2007 and the current bugs that are affecting Office 2003 are just not affecting Office 2007. So there is a lot of progress that is being made.
Richard: I was going to ask you about whether there were bad practices in the past or whether it was just an absence of good practices. I think you pretty well made the case that it was just the absence of good practices, right?
Michael: I think an absence of best practices and a lack of awareness of the real issues. A lot of people are just not aware of the issues that are out there. And I think a lot of people believe that because they are running on a Mac or Linux that they don’t have security bugs and everything is fine, it is just Microsoft with security bugs. Which is clearly not true.
Scott: People who read this will be sure that I’m saying this out of bias, but it does really seem that the latest Microsoft products are significantly more secure than the latest major open-source products.
Michael: Well, it’s funny, I listened to a podcast yesterday at the gym, and they got going on about how insecure Windows was and how they run Macs and they are fine. It’s like “Oh my God guys, have you been ignoring the security updates for your machine?” The sheer lack of recognition of the pervasiveness of security vulnerabilities across the industry just blows my mind.
Scott: Are you seeing any trends where as the Microsoft products become hardened that hackers are going after softer targets?
Michael: Yeah, you bet. Absolutely. We have been seeing that for about two years. The big problem is you don’t read about it, you don’t hear about it. We do, we watch what is going on in the industry, and we are seeing other systems being compromised regularly. Just because they are not getting as much press doesn’t mean they are not getting compromised.
But, you know, to be frank, if we get bad press, I don’t mind so much, because it means that people will know they have to apply the patches. I think that is critically important.
Scott: Playing devil’s advocate here, how would you respond if I
A prominent free and open source software (FOSS) advocate, Stormy Peters joins OpenLogic from Hewlett-Packard (HP) where she founded and managed the Open Source Program Office. As an early adopter of open source, Stormy was responsible for HP’s open source strategy, policy and business practices. She was also a founding member of HP’s Linux Division.
In this interview, Scott Swigart, interviewed