Interviewee: Marc Frons

In this interview with Marc Frons we talked to him about:

• His background with the New York Times.
• Their focus on the LAMP stack and Solaris.
• What he likes about open source.
• The ethos of open source and the ethos of a news organization.
• How pervasive open source is in the New York Times business outside of the New York Times Digital properties.
• What the reporters use to get their job done.

Sean Campbell:  Marc can you lay out a bit of your background with the New York Times?

Marc Frons:  Sure. I’m the Chief Technology Officer of the New York Times’ Digital Operations group. I’ve been at the Times since June of 2006. My role is to oversee technology strategy and operations for all of our digital properties. Since July 2007, I have been directly supervising NYTimes.com’s technology and product development as part of my  responsibilities at the company in addition to looking at potential partnerships, acquisitions, technology strategy and tactics overall.

Sean:  Where are the places where the New York Times has made investments in open and closed source technologies?

Marc:  The history of the NewYorkTimes.com is interesting. It is one of the first newspaper websites and now the largest newspaper website on the planet, as far as I can tell. And for the past six years, it has been a proprietary platform and a very good proprietary platform. The folks long before I got here wrote their own application server, even wrote their own programming language. And it’s highly scalable, very secure, and incredibly fast and it has served us well for the past six years. At the same time it is somewhat inflexible in its current form. So we made the decision a little over a year‑and‑a‑half ago to begin to bring more open source technologies into the shop and develop some of our new products using open source technologies. And that was a conscious decision to leverage the knowledge of what other folks were doing in our industry, which would enable us to move faster and be more innovative. At the same time, that other platform isn’t going away and we are continuing to do some things on that as well.

Scott Swigart:  Since you essentially have the world’s largest news web site with the associated high expectations that brings, how do you approach something like that? What would be your advice to another organization looking at open source, things you can take for a given with open source, and things you need to take a closer look at in order to ensure it is going to be handled?

Marc:  We are making a bet, not a big bet, because we are being incremental. We are not just saying "OK, we are going to redo this entire platform that serves almost a billion pages a month and all sorts of other things on a new platform" and just flip the switch all in one big bang.

We have an architecture that allows us to experiment, hedge our bets and be incremental. And I would advise anyone to do the same when making any platform transition.  Do it a little bit at a time, try out a couple of new lower risk products, things that you can do quickly, things that don’t get a ton of traffic, and then you move up the food chain as you go along. Test your hypothesis; see if these things that you thought were true really are true.

Scott:  So just to make sure I understand that right, you may take some section that is three levels down in the hierarchy and move that over. When you look at a website, most people never really bother to understand what it is running on and so you keep reimplementing more sections until eventually the home page is being served up by the new technology?

Marc:  That’s pretty much our approach, because obviously your new technology is going to evolve along the way as well. You want to try it out on the smaller pieces of your site or smaller sites that are in your infrastructure and not do it all at once because you are going to evolve your open source platform as you along.

:  And so when you say open source and you are focusing on web, what immediately springs to mind are the LAMP stack, Linux, Apache…

Marc:  That’s pretty much what we have.

Scott:  OK.

Marc:  We have Sun Solaris in the house, OpenSolaris. But we will also be moving to Linux just for some things; we already have Linux for some things. We will be doing some more things on a pure LAMP stack.

Scott:  And some of the concerns around that I would assume are the quality of the software, support, servicing, documentation. So what are some of the things that you feel are really good versus things that are going to take extra diligence on your part?

Marc:  Quality is obviously important. We feel like these various pieces of the stack have reached a maturity, certainly on the level of the operating system, that allows us to use them more widely.  This is not something we have to worry about particularly much. On the database level, it has also reached a level of maturity that for our purposes anyway is adequate. On the web and application serving level, it is the same thing.

So if you look at the various layers of the stack, these are all things that we have increasing confidence that perhaps two or three years ago, you might say well these products have a ways to go. I mean others like Apache were more mature, but certainly even in the last couple of years, MySQL has made significant strides and Linux keeps getting better and the kind of support you can get for all of the various pieces of technology in your stack has certainly grown. The documentation that you get with it, the cost is a big factor obviously, but these are just attributes that make it possible to use open source software in the first place. For us, the major factor is when talented developers are excited about the tools they are using and the technologies they are using and are passionate about it, you get a lot of innovative development. You can’t overestimate the importance of that.

:  One question given that you’ve built somewhat of a home brewed solution. With that in mind what do you see about the typical advantages open source gives you to fork it, providing you complete access to the source, etc. Is that part of the equation for you or is it just that you feel the platform has matured to the right point?

Marc:  Access to the source is nice, but I really would rather not be rewriting the kernel.

Sean:  Right.

Marc:  What I like about open source is that you are leveraging the community to help you make improvements far more rapidly than any single company could make them for you with a vendor product.

Sean:  Talk to me a little bit about that. How do you see that playing out in terms of the rapid innovation? Do you see the New York Times eventually contributing more actively to some of these open source projects? The Apache folks would have to look at you guys and say, "It’s a billion web pages." Whatever you would have to say about Apache you think they would listen, etc.

Marc:  I think so. This is obviously a new thing for the New York Times. So there’s a lot of building we need to do internally.  But our goal, in general, is to think about ourselves as a platform‑‑not a technology platform so much as a news and information platform‑‑and adopt some of the same ethos of the open source community in software, when it comes to content, information, news, research and data. So as we move forward into 2008, our goal is to create APIs not only for our own internal consumption, but for the outside world so that you can access the New York Times search archive and mix and match that with other data and information and applications, and create compelling applications out of our content and our other tools.

:   One quick follow‑up to that and then I’ll turn it over to Scott. I’m just going to pull on a thread that seemed to be underlying the previous discussion, but I’m not sure if this is accurate or not. Would you say that, from your perspective, there is an almost symbiotic nature between the ethos of open source and the ethos of an organization that’s journalistic, in that you want to provide information? You want to provide an open sense of community. You want to have multiple eyes looking at something trying to analyze the righteousness of it.

Marc:  That’s right.  We’ve increasingly been moving to open NYTimes.com both journalistically and technologically. We have close to 100 blogs by our journalists and launch new ones almost every week. We allow our reads to post comments on articles and we’ll soon be launching additional community and social networking functionality on the site, built, of course, using open source technologies. Perhaps most significant, last September we ended our paid subscription product, TimesSelect, and made virtually all of our content free. And the response has been pretty amazing.

So it’s true, open source fits nicely with how we think about NYTimes.com as a whole.  If you think about the history of journalism, Journalism 1.0 maybe was the typical "who, what, when, where, why" third person journalist‑‑just the facts. Journalism 2.0, the new journalism, maybe is Tom Wolfe,  Hunter Thompson and others who themselves became the center of the story. And 3.0 is where the audience becomes a part of the story through blogs and other forms of online community, and where the Internet helps inform what you’re doing by enabling you to mash up other data, other news, other points of view. So I think that’s where the New York Times as an organization is going, and really needs to go as we become a part of not just an isolated node on the Internet but an integral part of this vast but still growing community.

Scott:  Tell me if I’m reading too much into what you’re saying, but I don’t think I am. It seems you’re saying if I had a team of developers and they were used to buying their software and they were used to a proprietary model of acquiring their tools, they may look at our business and our information in more of a proprietary way. But if you take developers who are kind of steeped in free flow of information – openness, taking from the community and contributing back – those people come up with ideas. And those ideas probably infuse a lot of different things they do, including ideas they come up with for how the New York Times could collect and contribute information to the planet.

Marc:   I haven’t really articulated that even to myself in quite that way, but I think you’re right. I think adopting an open source philosophy tends to attract a particular type of developer who perhaps‑‑and this is not true for everyone, obviously‑‑but who perhaps is more engaged and more aware of the possibilities of information sharing than someone who is very used to a single product, a single source approach, a more closed approach. A lot of that, too, has to do with personality and skill set and interest.

Scott:  Right, right, but people who tend to work in open source kind of gravitate that way because of their personality I would assume.

Marc:  Yes.

Scott:  People are kind of this sort…

Marc:  Let’s put it this way: it certainly couldn’t hurt, right?

:  [laughter]
Well, let me ask how maybe the digital business contrasts with some of the other businesses in the New York Times. You guys are using a lot of open source technology. Is that pervasive, do you feel, throughout the New York Times, or is it more like look, "We’ve got a lot of office workers. They all run Office all day long." Different businesses but not…

Marc:  It is not yet pervasive, and I doubt that it will be pervasive anytime soon.

Scott:  On the Web and for what you guys are doing, open source makes a lot of sense. Are there areas where you think it doesn’t make as much sense?

Marc:  When a proprietary product is so clear superior to anything you can get through open source, then naturally you’d go with the vendor solution. But I think for systems and products where software does not change very much‑‑where you have a vendor product that meets your needs, that allows you to run without a whole lot of technical infrastructure, a product that gets the job done,  doesn’t have to be particularly customized,  isn’t changing all the time, where business requirements are set, where you just sort of want it to work‑‑because your business really isn’t changing. You have a payroll system. You know you have 10,000 employees. You know they get a paycheck every two weeks.

Unless there’s an open source product that has been battle tested in the market and works and you can just download, install and license it, you’re probably going to go with an established vendor, right?

Scott:  Right, right.

Marc:  So the same is probably true for many of your other corporate systems.

Sean:  Well, let me ask a question on that, because maybe I’m right or wrong about this. I would guess the New York Times probably breaks down into a ton of different IT buckets- three broader ones at a minimum where it might be servicing the needs of the staff, for lack of a better phrase: the accounting teams, and the HR teams? Then you’ve got the web presence, which is huge. But then, I would also imagine you have the needs of the reports which is the heart of what people think the New York Times consists of in any case.

Marc:  Yes.

:  So with that in mind I’m curious. Maybe for the office worker, a closed source piece of software makes sense. Then you’ve got the Web, where you have a variety of reasons you went with open source. How does it play out for reporters who are having to report from almost anywhere on the planet? What type of technology stack enables them to do what they do?

Marc:  It’s a good question. And it’s actually very interesting that you mention that. Shortly after I assumed this role and reorganized the NYTimes software group, I created a group called Interactive News Technologies. Basically, it’s a small team of software engineers and technically-minded journalists who sit in the newsroom alongside the reporters, and are charged with building both tools and very rapid development applications for news stories and news events. And actually they are using Ruby on Rails for some of their stuff.

We’re also now talking about fronting the big print publishing systems with web forms and other tools in order to get structured data, and a lot of the other things that we want to do that we can’t do easily in big, closed systems. So we are beginning to use open source in that way to help our Newsroom.

Sean:  Well, one question on that then; so Ruby on Rails is obviously popular and the Basecamp guys have probably popularized it more than most. As in they are one of the canonical apps where they focus on how easy it is to use, it has good fidelity, etc. So just to drill into that one for a minute, why the choice of Ruby on Rails? Because you obviously laid out a good business problem of, you have team of reporters, and they probably have needs that are immediate. You don’t want to be slowed up in any way. You have to have a development team that is able to move pretty fast. And with all that in mind I’m curious why you made the choice you did.

Marc:  My philosophy is, you tend to go to a large extent with what your team is comfortable with and excited about using. So when I was at Dow Jones, we were a big Java shop and had Java J2EE stack with a mix of open source and vendor components. But we used Apache, and we were moving to Linux, MySQL, and Tomcat and away slowly from Oracle and WebSphere, as these open source pieces of the stack matured.  That is where my guys were comfortable with. Those were the technologies they liked. They proved to me they could make them work. I wasn’t about to tell them they had to use something they were opposed to using.

Our interactive news technology guys said, "Listen, we’d really like to try a Ruby platform as opposed to Django, as opposed to some other framework." And I said, "Well, go for it; show me that it will work and start to experiment, do some things on it." So my feeling was you have to let your people have the tools that they are comfortable with as long as you feel that they have a chance to succeed, and you have to charge them with succeeding.

Scott:  You said something earlier that I think ties into that and, again, let me say it in my own words and see if you’d agree. It is more important that you can build whatever you want with your choice of today’s modern tools. And so you want your developers to be passionate and excited, so you let them use the tools they are going to be passionate and excited about using. Because you are going to get more creativity, productivity, etc. if you are letting them have input into their toolset rather than if you are simply dictating a toolset such as, "Well we are a Java Shop, therefore everybody use Java."

Marc:  Yes, exactly. The typical corporate CIO wants to consolidate systems and technologies and have a monolithic structure because he thinks that is going to save him money. And for some things that is true, but corporate technology is almost always a cost center.  Whereas for the digital businesses, IT is integral to the growth of the business and you could argue that IT is the business — software development. You don’t have a business if you don’t have software developers. You don’t have one if you don’t have journalists either if you are in the New York Times, but that is why technology and journalism are inexorably intertwined in the digital world much more so than they have ever been before and why they will continue to be even closer, even more intertwined as we move forward.

Sean:  So what do you see in the open source community that excites or intrigues you the most and I guess at the same time what excites you in terms of the closed source side of the world. As a CTO, you are looking at technology trends as well as specific implementation issues. What seems to stand out?

Marc:   Let us put it this way, there are so many open source projects, there are so many things happening, there is so much innovation going on that it is hard to pick and choose. I mean I think obviously the whole LAMP revolution has been fascinating to watch. These kind of contained frameworks like Ruby, like Tapestry are quite fascinating in that they put tools in the hands of people who perhaps don’t have the skills of hardcore software engineers, who are busy writing code at the base level.

But what I think really excites me is the move to making a lot of these more sort of all‑in‑one frameworks more scalable because if you are in the New York Times, it has got to scale and if it doesn’t scale, we basically can’t use it for anything really important.

Sean:  Right, exactly. Yeah, a billion pages are simply a billion pages, right?

Marc:  Right. Now the truth of the matter is it is not exactly evenly distributed. I mean some parts of the site get a ton of traffic and others not so much. So if you want to do something on Ruby, you put that on a thing where you know it is not going get creamed. Just because it is the first time you have really done that and it is Ruby, so you have to make sure that it is going to work OK, but as you gain confidence in these technologies, then you begin to roll them out in more highly visible and highly trafficked places.

Scott:  I don’t know how core this is to our investigation, but it is an interesting question to me and you might have an interesting answer to it. This is sort of continuum, right? On one end of the continuum, you have closed source proprietary companies who want to be very secretive about their roadmap, very secretive about their features, not share their source code. And even that is diminishing because even they are realizing that there is business value in transparency.

And then on the other end, you have the free software foundation who feels like all code effectively should be public. Initially they were kind of happy with software as a service because there were people who were using open source and building on top of it. But they might even want to go after someone like Google and say, "Hey, you benefited from open source, share your code." Where do you think it makes sense between those two spectrums to kind of draw the line, because it has got to make sense. You can’t kill the goose that laid the golden egg in either direction.

Marc:  Absolutely. Obviously, people need to be compensated in some way, shape or form for their hard work.  But there are emerging models: Just because you are open, doesn’t mean you don’t charge for it. Look at MySQL, you can download it but then you pay for support. It may be cheaper than Oracle but it’s not really free.  The same is true for various flavors of Linux. So I think people are finding ways to make those models work.

Google, it is true, has used a lot of open source, they have also written a lot of their own proprietary stuff that they have not given back to the community. And I think they have done that probably because they needed to just because of the scale of their operation. It really wouldn’t even be germane to most other businesses, right?

Bookmark this:

Interviewees:Blaine Wastell and Glenn Block.

In this interview with Blaine Wastell and Glenn Block of the Patterns and Practices Group at Microsoft we asked them about:

• About the Patterns and Practices Group
• What makes them different from a typical product development group
• The software development methodology they use
• Some of the things that Patterns and Practices group has produced for developers.
• The role of CodePlex for their group.

Blaine:   I’m Blaine Wastell. I’m a program manager in patterns & practices, and I’ve been in this role for a little over four years now. Most recently my focus has been in what we call the "client UX program". It’s about providing guidance to customers on developing both smart client and web client applications.

For 12 years before Microsoft I was out in the consulting world helping business customers develop enterprise line of business applications, mainly Web applications.

Glenn Block:  I’m Glenn Block. I’m a technical product planner in patterns & practices. I am a newcomer to the client team. I’ve been in p & p for about 6 months, and with Microsoft for about 2 years. Prior to p & p, I worked in Microsoft Learning and was responsible for building online developer training.

For the 10 years prior to Microsoft I was a software engineer and architect. I worked in various startups as well as a few enterprises. I’ve had a touch of consulting, but it’s mostly been really about building both packaged apps and internal apps ‑‑ both on Web and on Windows. A significant part of my background has been in developing software frameworks for those kinds of applications.

And as the technical product planner on the client team, I’m responsible for the overall strategy, understanding what customers want, and setting expectations on what kind of guidance we’re going to deliver.

Scott:  Talk a little about patterns & practices.

Blaine:  We provide "building blocks" that customers use for line of business application. Microsoft has a number of development tools and technologies which include the .NET framework, Visual Studio, SQL Server, the core operating system, etc. We help customers by providing guidance, in multiple forms, on using those technologies to build line of business applications.

Glenn:  The practices part of our name is focused on the latter, where we give guidance, best practices, and design patterns, to educate developers on known solutions so they’re not reinventing the wheel.

Scott:   I’ve used like the Enterprise Library that patterns & practices produced, and it’s really more of a framework than building blocks. Are things like the Enterprise Library are designed to be best‑practice reference architectures, or are they drop-in frameworks designed to give a productivity boost?

Blaine:  Enterprise Library, and the application blocks, are part of what we deliver. We also have written guidance for areas like security and performance.

We also have another type of deliverable called "software factories" that tie all the pieces together. They have the framework, but they also include the common patterns for a specific area. They include automation; they include what we call a "reference implementation" ‑‑ essentially a sample of how to implement proven practices using the frameworks.

Glenn:  When you talk about frameworks, at the end of the day, to us it’s all guidance. In the case of these frameworks, they are incarnations of the patterns we recommed. We think of them as an 80% solution meaning they may not provide all the answers. We write the code with the intention that people who are consuming these deliverable will rip that code up, and will look at how we’re doing stuff. It’s not simply about giving people a reusable toolkit, although there is an aspect of that.

The idea is, "Hey, look at what we’ve done. Rip out the pieces that you think are applicable for your scenario. If you’re writing something new, you can use this as a reference point and know that it incorporates well‑known patterns."

Scott:  What makes your team different from a Microsoft product team. How are you different from the teams building the .NET Framework, or SQL Server, for example.

Blaine:  We focus on what we call ‘customer connected engineering’. We start with an area that we see is challenging for customers, and then we will put together an advisory board that includes customers. As we design a solution, they’ll validate that our scope and approach is really hitting the top set of challenges that they have.

Glenn:  The big differentiator is that product groups build the platform, while the guidance we create uses the platform.

Product teams focus on taking the platform to the next level. We do guidance, as Blaine described, guidance on using the platform. We show you how to put the legos together in meaningful ways to solve particular problems that you’re facing.

In some cases, through the act of putting those things together, we may pull out some new reusable guidance that we call ‘application blocks’. But the block is not the end goal. The block is just one incarnation of the guidance. When I say ‘block’ , I’m really referring to a reusable library.

Scott:  Could you list some of the things that you’ve produced?

Blaine:  You mentioned the Enterprise Library, which has a number of blocks within it. There’s also the composite UI application block, which is a library that helps you create smart clients. There is also a set of libraries to help you create web clients. One is called the composite web application block. And we also have another one called the pageflow application block which addresses challenges around Page navigation. Blaine:  We’ve done a number of things in the web services area.

Glenn:  We have a Web service software factory; which is guidance on building web services. And we also have new guidance that we’re developing called ESB guidance which addresses utilizing BizTalk as an Enterprise Service Bus.

Scott:  Talk a little bit about the software development methodologies that you use.

Blaine:  Sure. We are an agile organization. We’re probably closer to XP. We start with the backlog that we prioritize. Glenn is essentially the proxy for the customer; the one that will prioritize the backlog for us. From that we do the standard iteration planning, and meetings. We do it once a week, and we have stand ups daily where we go through the status of the standard set of questions: What did we do yesterday, what do I plan on doing today, what blocking issues do I have.

At the end of an iteration, which is every week, we publish our results out on our CodePlex community site. Then customers can look at the CodePlex site and get an understanding of where we’re going. A lot of times they’ll say, "Hey, why are you doing this, because I have these other two sets of challenges you’re not covering," which is good. We’ll also have people say, "Hmmm. There’s a bug in line 35." It’s pretty amazing how closely customers sometimes watch us.

Glenn:  And in some cases even supply fixes.

Blaine:  Right. We’ll also do testing, development, and continuous integration. We use in test driven development, so we write the unit tests first, and then the code.

We’ll generally do pair programming. We’re also distributed team with team members located in Redmond, Buenos Aires and India. This sometimes makes it hard to always do pair programming. If we can’t do pair programming we will have somebody else do a code review before checking the code in. We do automated exception tests where they make sense. We do quite a bit of static analysis on the code that gets checked in.

Glenn:  How about code coverage?

Blaine:  We do code coverage. I have a blog entry that I just put out recently that talk about the quality checkpoints that we go through. On the quality assurance side, we’ll do things like proof and security testing, and globalization testing.

Scott:  You mentioned there’s also a community side of what you do. Talk a little about that.

Glenn:  There’s several different ways that we actually interact with the community and our customers. The simplest way is our blogs. A good portion of the team is heavily into blogging, and we try to give as much visibility as possible into what we’re up to.

Blaine mentioned that all of our stuff is on CodePlex. A big differentiator between the work that we do, and the product groups at Microsoft, is that you can get all the source code.

Also on CodePlex we have work item voting. When people come to us with feedback we say, "Great idea. Throw that on as a work item so the rest of the community can vote." And I can tell those work items absolutely determine what we do.

For example, AJAX support in the Web Client was one of the highest voted work items. As a result we’ve done about four months of work on the UI responsiveness and we have a few more to go. That’s a large investment and it’s based off the community feedback.

In some cases, there’s a work item that’s very broad, and we’ll dig in. For AJAX there’s a server side and client side aspect. When we asked which people were more interested in us investing in, people said, "Hey, we’re more interested in the server than in the client."

Another source is our direct interaction with customers. One of the things we do in p & p is bring a lot of customers in to spend time with the team. That gives us an amazing opportunity to directly interact with them, find out what they’re building, find out how our guidance in mapping against their needs, and find out what the gaps are.

We just had a customer that was here for an entire week. We do that with a number of clients and aggregate the information and learning which we inject back into our efforts.

We also do some on‑site, where we go and visit customers. Since I’ve joined, I’ve visited two customers ‑‑ one in the U.S. and one in the U.K. ‑‑ spending a week with their teams.

Further, we have general blog that generates feedback that we get. We have email feedback that we get. That covers the external customers. But p &amp p also has internal customers. Who are those? These are the product groups.

Lately we’ve been we’ve been working very heavily with Scott Guthrie’s org. We’re doing this is because we want to understand what kinds of challenges the platform is going to be addressing and how it will be addressing itWe want sure that whatever guidance we’re giving is in line with this direction and not opposed to it.

And we also key in to their demand for deep content. They say, "Hey, there’s nobody who’s really telling people how to put these pieces together." We’re also getting very involved with OBA [Office Business Applications]. We’ve had the OBA team coming to us saying, "Look, we need some solid guidance that will give customers architectural guidance on how to develop an OBA application."

Sometimes all these inputs are in conflict with each other and we have to look at the external feedback, the internal feedback, the industry trends, open source, and even our competitors, and decide what we’re going to deliver.

Finally we have our Customer Advisory Boards. This is a key aspect of our customer‑connected engineering strategy. For every one of our deliverables, we have an advisory board of individuals that are in the industry, that are or will be using this guidance. For the Web Client Software Factory, every two weeks for about the past four months, we have an hour meeting where we present to the advisory board what we’re doing, and we get their feedback.

The advisory board is a group that’s able to look at what we’re doing and raise flags and tell us, "Hey, you’re going in the wrong way." Or they may be saying, "You guys are missing this big opportunity over here that really affects all of us." And when we build this advisory board, we select a group of experts across the industry in both small and large companies, some located in the U.S., some located in Europe, some located in Australia. We try to get a really good mix.

They have different needs. They give us a proper perspective on the overall landscape so that we can make those intelligent decisions. I think that covers it. As you can imagine, it’s not an easy task to drill through all of that input and decide which way to go.

Scott:  Right now, the patterns & practices code is released under a custom license. It’s not under the Microsoft Reference license or the Microsoft Public License — which used to be the Microsoft Permissive License.

Blaine:  Everything is migrating to the MSPL.

Glenn:  Yeah. It’s essentially been MSPL without the name. It is going to be MSPL.

Scott:  OK, so the MS Public License.

Scott:  The thing that’s really interesting that now that’s an OSI‑approved license. In the past, Microsoft would say, "We put the source out there. It’s open source," but to a lot of people in the industry…

Blaine:  It wasn’t really.

Scott:  Yeah. To them, unless it’s an OSI license, they don’t know if it’s really open source the way the OSI defines it. So now it is. That leads to the question of, are you now starting to do some of the things that a traditional open source project would do? You’ve got builds. You’ve got bug tracking and issue tracking that people can participate in.

Do you ever envision a day where you’d be taking community code submissions?

Glenn:  We’ve taken steps in that direction with what we call our Contrib community.

Before I worked at Microsoft, I used to work with a bunch of different open source products. I worked with NAnt and NUnit. NAnt has what’s called a NAnt Contrib Project. The NAnt Contrib project is where developers who have the source, are extending NAnt. They’re building plug‑ins, etc., because a lot of these products have a plug‑in model. They want to have a way of driving that back in so that the community can benefit from those enhancements or extensions.

We decided about six months ago to follow the same model and we created a set of Contrib communities. Actually, we didn’t create them, and I think it’s important that we didn’t create them. We worked with the community. For example, with Smart Client we engaged with people like Kent Boogaart, who created WPF CAB, which was his own extension to CAB that would allow you to build WPF applications with CAB.

We also worked with people like Bil Simser, who’s a known MVP who developed Smart Client apps. We worked with a whole group of people, Chris Holmes is another. We brought them together and said, "Hey, we want to make you guys as successful as possible and enable you to take our deliverables, extend them how you want, without us telling you which way to go, similar to the way you can with other open-source projects. We’ll provide a place for you to do that, and we will also work with you going forward. We’ll also hear your concerns and consider how we can fold that into the product."

We do also work with external resources as part of our core guidance deliverables. We have our advisory boards which I mentioned which participate in the design and give us feedback. We have our CodePlex Workitems where the community submits ideas that influence the design. And we even have members of the community (partners, SMEs, etc) that write the codebase itself.

Scott:  Aren’t there models where other people have dealt with this? You take a look at something like the Linux kernel, right? Nobody really owns it. It’s an open source project, it’s covered under an open source license. IBM is a contributor to it. And if stuff gets found in there that infringes on somebody else’s intellectual property, a lot of times the community, or IBM, replaces that with code that doesn’t.

I realize it’s a gradual process, and Microsoft is moving towards more community involvement, but it seems like down the road there may be ways to structure this to where Microsoft is a chief contributor to these projects, but every committer may not necessarily be a Microsoft employee.

Glenn :  In the Linux kernel, yes you do have community participants, but it is not a free-for-all where anyone just comes along and checks in code. It’s a regulated process with a core set of contributors. We operate in a similar fashion. We do have externals that contribute, but there is a process around those contributions.

Scott:  Another thing that’s interesting, when you’re under an OSI license somebody could fork your code.

Glenn:  Yep. And I just want to make clear that we don’t own the Contrib project. We’re not claiming any responsibility for the kind of code that goes into that. We might suggest stuff, like, "What do you guys think about doing this or doing that," but it’s completely up to the contributors what they want to do.

It is the model that you described. We may pour some stuff into Contrib, and we’ve talked about doing that, but essentially the people that own it is the community.

Scott:  Right, right. And those different Contrib projects are essentially forks of your code, and because…

Glenn:  That’s exactly what they are. In some cases, complete rewrites. For example, if you look at the EntLib Contrib, there’s actually an EntLib refactored project where somebody who works at Microsoft ‑‑ Scott Densmore, who helped write a lot of our deliverables ‑‑ is actually ripping apart EntLib and providing a new version.

Similar kind of things are happening in the CAB space. We just recently shipped another version of Smart Client Contrib that actually contains updates to our Updater Block, to get it to work on Vista. This is completely being driven by the community and it’s taking things to places where we don’t have the resources to take them.

Scott:  That’s what I was wondering. Maybe down the road, the only thing that exists is the Contrib space, and people in Microsoft are contributors just like anybody else.

Glenn:  That’s certainly a possibility that we would be open to, but it’s not really about us and what we want, it’s about the customers. Many customers that have serious concerns about using community contributions that might contain code that violate IP. In the case of patterns & practices deliverables because it’s from Microsoft, they have less concern. Other customers however are willing to take that to take that leap of faith. These are the same customers that also use open-source deliverables like NHibernate, Log4Net, Castle Windsor etc.

Scott:  To me it’s really fascinating how the different models are shaking out. We also interview CIOs for things outside of "How Software is Built" and they’re very fond of saying, " I want one throat to choke. I want to know that I’ve got support. I want to know that I’ve got a company that’s standing behind it. That’s really important to me. It’s not about the cost of the software, it’s not even about free as in freedom. I just need stuff that works, and I need someone to hold accountable."

And then on the other hand, on the other side of it, you’ve got open source, right? Where you don’t seem to have those same assurances, but on the other hand the software really does hit the mark. There are people who are experts in their domain. There are people who build really interesting things that a "big company" wouldn’t necessarily think of. There are people who do some innovative things, or start with someone else’s idea and take it in a very different direction.

There’s benefits in all of that. So it’s curious to me what the landscape is going to look like. You take a look at a company like RedHat. They’ve got this open source product, but they don’t really have control over the Linux kernel, or Apache, or the things their customer absolutely depend on. Still, they guarantee that they’ll provide support on top of it, enterprise‑level support.

You’re saying that if p & p’s work was full open source, and the only out that existed was Contrib, some of your customers would have trepidation because they would feel like there was nobody they could really point to who would guarantee the quality of the code, who could guarantee fixes, and those sorts of things.

I guess what I’m saying is it’s a very interesting spot that you sit in.

Glenn:  In a way, it can be the best of both worlds, with some caveats of course. What it means is there may be significant functionality that will exist because of the energy of the folks in the open source community, and that functionality will show up in Contrib but not in our core deliverables. There are other things that the Contrib community might do, that we might decide to incorporate into the core in the future.

Scott:  There’s another thing that makes your position interesting. On one hand you might get feedback from the product team saying, "Hey, we’re not going to have this in the next version. It’s going to be a pain point. It’s something that you guys could take a look at."

On the other hand, you might have a lot of people in the community asking for a feature, but because you’re part of Microsoft, you know that the product team is already building that into the next version of the product. You don’t want to invest in what the community is asking for, because it’s already being built, but you can’t tell anyone it’s being built. The product team still has their plans under NDA.

Glenn:  That certainly happens all the time. That’s certainly a challenge. But what I’m happy to say is that more and more we’ve been able to drive back the experiences customers have had with our guidance to the product teams, which has resulted in many cases in product offerings that come out based on that.

Scott:  When I looked at stuff you produced a long time ago, for example, the Data Access Application Block, it seems like it was almost destined to be obsolete. Eventually Table Adapters got built into the .NET Framework, and that provided the same functionality.

Blaine:  Our goal is to actually help migrate customers from what we produce, to the core product. That’s really our ultimate goal. If the core product addresses scenarios we’ve heard from customers, that’s a good thing.

Scott:  Yeah. Like I said, you guys are in a fascinating spot in terms of being an open source project, or open source projects, you’re really pretty unique it terms of where you sit. Can you share anything about the directions in which you’re going? You’re obviously moving towards more open. There’s certain constraints on that. Anything that people should look for over the next few months?

Glenn:  One of the things I think we’re moving on, and Blaine can touch on this, is bringing more of what we’re seeing to product groups. As an example ‑‑ and we’ve been trying to collaborate much more tightly internally ‑‑ one example is this new MVC framework. Want to talk a little bit about some of the work?

Blaine:  At patterns & practices we’ve been supporting, as Glenn talked about, many of the common patterns in the UI space. MVC [Model View Controller] is a common one. MVP [Model View Presenter] is another one. And since we’ve been in the space, we’ve been working with the ASP.NET team to help define the MVC work that’s coming out of there.

A big part of this is really Scott Guthrie and his team’s vision. We are bringing some of the feedback that we’ve heard from customers to the ASP team.

Glenn:  We’ve actually been working on the design with them. It’s really been a partnership. I think those kinds of things are efforts that you’re going to continue to see.

The other big thing that I think you’re going to see, is the fruits of a transition we’ve been going through internally within patterns & practices. We’re re-evaluating carefully at how we deliver guidance and what is the best method. , and we’re also looking at taking the principles of SCRUM that we apply within our teams and bringing it up to a higher management level with p & p. This includes having having a cross p & p product backlog that we can be very visible with customers about. We want to show, "These are the things we’re looking at. This is what’s a high priority. This is what’s not."

On the subject of how we deliver guidance, although many customers have nothing but good to say about our factories, there are other customers, that say they are too complex. They also tend to not be as applicable in in what we call "Brownfield scenarios" which are existing applications. Instead, they tend to be most useful when you are starting from scratch. Another thing we’ve seen is that industry-wide there has been a shift away away from comprehensive “do-it-all” type frameworks to a more pluggable services approach. Finally we’ve had a set of customers who tell us that the biggest value we can provide is to focus more on patterns themselves and how to use them (including samples and reference implementations) rather than framework type libraries that implement those patterns. We’ve heard thse customer concerns and are taking them seriously. We’re looking at how we can break things into smaller blocks. We don’t mean exactly like the earlier application blocks, but instead how to ship our deliverables in an incremental fashion that offers customers more options as to what they use.. For example if I want to use some aspect of the factory without having to take everything, then I can. And, we are also looking at patterns and how we can make them more a first class citizen in our guidance.

Around this, we’ve put a stake in the ground with the current release of the Web Client Software Factory and with our newly announced WPF Composite Client.

By getting these things to be smaller, we can ship more of them more frequently, we can make more intelligent decisions about what we ship and what we don’t ship. We can build guidance that has more applicability in both the brownfield scenarios and well as greenfield — which is new development. We also give customers more choices as to how they incorporate our guidance within their environments.

Ultimately, we can reduce the complexity in evaluating and using these deliverables to make them approachable to a wider audience.

As you can tell, we’ve certainly got a lot of work ahead of us.

Scott: Thanks for talking. You opened up an interesting subject at the end, and maybe we could talk again in the future about it.

Blaine:  You bet.

Bookmark this: