How Software is Built

Interviewers: Scott Swigart and Sean Campbell

Interviewee: Gabor Cselle

In this interview we talk with Gabor Cselle. In specific, we talk about:

• Deciding between open and closed development models
• Strategies around handling QA challenges effectively
• Maintaining high usability and user security
• Business models and corporate customers as opposed to individual users
• The relationship between open-source and commercial software
• The value of avoiding feature creep

Sean Campbell: To get us started, could you please give us a little bit of background on Xobni, your role with the company, and a little bit of your work history before that?

Gabor Cselle: I’m the VP of engineering here at Xobni. I run our engineering team, manage QA and support, and was the first employee. Previously, I worked at Google as a software engineer, both in Zurich, Switzerland where they have a big engineering office and in Mountain View.

Xobni was started as a Y Combinator company. Y Combinator is a seed funding group that tries to encourage young college graduates to start companies, instead of working for “the man.”

The company was started by a graduate of MIT and a Penn State graduate in 2006. For about the first year, they were working on what we call Xobni Analytics today, which is a package that analyzes your Outlook emails just like Google Analytics analyzes your web traffic. It would chart out how many emails you got this month, how many people you emailed with, and so on.

After a while, we realized that people wouldn’t use that more than once or twice, and then they would let it fall by the wayside. We really wanted to build something that people would use for hours every day, and so we got the idea for today’s product, which is a sidebar for Outlook.

We got funding from Khosla, so after I joined, we built the current product in about six months, and launched the private data in September of 2007. Since then, we’ve been adding features and broadening the project’s scope, and we have some new exciting projects as well.

Sean: Given the overall open-source momentum these days, what was your thinking on making the project closed source from the start?

Obviously, some of it was driven by the fact that you are going to plug into Outlook, but that didn’t have to be a complete limiter, in and off itself.

Gabor: I don’t think it was a conscious decision in the very beginning. It was more about the fact that eventually we wanted to license this to enterprises, because we were working on the Analytics product back then. And obviously, Outlook is closed source–there’s the thin layer around Outlook of APIs that Microsoft makes public, but everything else is closed source.

I think we are just kind of imitating that model, and eventually we may have versions with extra features as something that you can purchase. I think that’s why it’s still closed source, and we’re probably going to continue to follow this route.

Sean: I guess the natural follow up to that is to ask what kind of push you have gotten from people to extend what you are doing with Outlook and put it in Evolution or Thunderbird?

Gabor: Actually, the day after we released, we had about 400 emails in our support queue that all said, “I’m not an Outlook user. I’m a Thunderbird user,” or “I’m a Gmail user and I really like the functionality,” etcetera.

So this is definitely something that we’ve looked at very, very intensely. We decided not to pursue the path of integrating with any other desktop clients, at least for now. We would need to revise a lot of the interface, but the main reason is really that Outlook is the number one email application in the world. There are 400 million people using it, and a lot of these people are in pain.

Outlook doesn’t do what they want it to do. They can’t search for messages quickly, especially with the older versions. We solve that one pain point really well for Outlook. It’s a huge addressable market, so that’s why we are sticking to it. The future might hold a version for other clients as well, though.

Sean: One of the things that I find interesting is how you have handled the breadth of QA questions you must have come up against. I mean, everybody uses Outlook, from the most computer illiterate to the most sophisticated users, so what they’re doing with their machines is radically different from each other. Notepad may be the only thing that gets more universal access on a Windows box.

What was your process around QA from the beginning, and how has it evolved? I’m sure you have to build something that is absolutely rock solid, and yet who knows what people are running alongside it?

Gabor: My first reaction is to say that this issue has been a lot of really hard work. When we were originally building the product and the team was just four people, we outsourced QA to a manual testing company in California. We had a good experience with that, but the problem is that the number of possible configurations and the number of things that can go wrong is very high.

I think the best decision I’ve ever made here is to bring QA in-house and to hire really good people and just try to get these issues taken care of by adding more bandwidth to them.

We hired an excellent QA lead and testers, and they’re not just testing the product–they’re testing the different integration scenarios with Outlook. They’re testing scenarios like how it works with Exchange, IMAP, POP, and various combinations of plugins.

That’s a lot of work, and it’s not particularly sexy, because everyone wants to work on a great new feature, rather than what happens when a particular ActiveDirectory setting is at one instead of zero. I’m very happy to say that we have QA engineers who are very passionate about getting these things right.

Scott Swigart: In a lot of open source projects, code’s checked in and people download stuff before it’s officially called a release, and they work with it. The model is often just to have good bug tracking, but not a lot of overt structure beyond that.

What do you think about the notion of handling bug fixes that way? What do you think would be the net result of a model like that in your case?

Gabor: That’s a very good question. I don’t think it would necessarily work for us–mainly because people don’t want to be exposed to bugs in their Outlook, and they don’t want to be the guinea pigs, because Outlook is their data store. It’s where all their personal data is, and they’re necessarily very sensitive about protecting that.

I think that our case requires in-house testing. That said, however, we originally did a staged private beta release of our software, to a limited population who had signed up on our web site.

That gave us a really strong test audience of people with a bunch of different configurations, and only once we were pretty comfortable with that did we go to the next order of magnitude.

There’s an effect at work that we call the “10X effect.” Every time you increase the number of users by a magnitude of 10X, new stuff starts popping up. For every 10X increase in user base, we get all sorts of weird scenarios that we haven’t seen before, and we go through these cases very methodically and just kind of adjust them one by one.

Scott: Another thing that open source does a lot is to use very distributed teams. You can have people working on one project from all over the globe and lots of different companies.

What kinds of advantages does it give you because you can pull everybody together in a conference room and you’ve got people working in the same building, versus what may be some of the challenges of using a highly decentralized distributed model?

Gabor: Everyone is in the office during work hours–no one really works from home on our current team. We have meetings about support, about QA, about where the product is headed, and about the current status of each of the projects that we’re currently working on.

We find that face time is very important, because if you want to describe a problem, you can just go up to the whiteboard and sketch it out. You don’t need to start up some virtual conferencing solution and try to drive your discussion through a window on the screen.

Really good ideas pop out from hallway conversations; we actually just had a conversation about how sometimes the Outlook UI would freeze when we were syncing data in the background:

We were able to fix that issue, but if we had not had that face-to-face conversation, we wouldn’t have been able to get a solid theory about why that bug was happening.

Scott: Another thing that springs to mind is that, aside from the architecture and what is going on behind the scenes, do you feel like a closed-source, centralized model facilitates getting where you want to be with the user experience and those kinds of things?

Gabor: To answer that, I should first take you on an excursion through history. The interface of Xobni has been highly praised, and some people call it “Star Trek”-like, because it looks like the computers on “Star Trek, The Next Generation.” The way that interface came about was basically that, in the very early days of Xobni, I just tried out different things in Photoshop. I came up with three basic designs: one looked just like Outlook, one looked like Outlook with bigger icons, and one I made as a joke about how far out we could take it. That is what we use now. And that’s how the basic look and feel of the interface came about.

It was basically arrived at with experimentation and talking to the other team members at the time about what they felt about the different designs. Lately, we’ve moved into a slightly heavier development model. We have a user experience expert and weekly usability meetings, we view mock ups, and we do hallway usability testing.

We also go to coffee shops. There’s a Starbucks we frequent a block from the office. We just ask people, “Do you understand this mockup? Do you want to try it out on our laptop and tell us whether it works for you?” One of us actually stands in line and says, “We’re going to pay for your latte if you have five minutes to look at some screen shots.”

Sean: Wow. Ethnographic research with a latte–it’s the grass roots ethos, for sure. [laughs]

Gabor: We’re in the financial district of San Francisco, so it’s a really good spot, as these are our target users. They all use Outlook, because they work at Morgan Stanley or Wells Fargo. They’re exactly the kind of people we want to talk to, and they’re exactly the kind of people that need to understand what our product does.

And so, back to the open source discussion. I don’t think that in a distributed team, these kinds of impromptu usability sessions would really be able to take place.

Sean: There’s the issue of reliability, and nobody wants to see the “Outlook is checking your inbox” message for the 5,000th time, just because you happen to shut it down. But at the same time, there’s a security issue, especially as you get into those large companies in the financial district and so on.

So what is your process around security? In the open source model, it’s well-known, and it boils down to “many eyeballs creates a more secure code.” I don’t mean this at all in a pejorative way, but closed source companies, by nature, need to protect their IP, and so they’re a smaller team that keeps it in-house.

What has Xobni done around the security aspects of the development model? Are you following a particular framework, whether externally developed or internally?

Gabor: Our basic premise is that we don’t ship any personal data back to us, including any part of your email, any part of your calendar, or any part of the data on your desktop. That’s been the case since the very beginning of Xobni, and it’s absolutely vital, because people are very sensitive about this issue.

Beginning in the very early versions, we shipped with something called “The Customer Experience Program,” which was an opt-in program, and it still exists, if you go into the options dialog. If you enable that, we ship back an anonymized data set about your usage patterns. It basically tells us that a user identified only by a randomly assigned number uses feature X and Y, but not Z. So, that is the one time where you can opt in and volunteer to give us some anonymized user data.

Sean: That’s an interesting bridge to another question that I have had in mind–what types of issues arise when you support a corporate enterprise sale? I imagine that there might be some concern on the part of IT organizations about supporting a broadly distributed plug-in like this.

Gabor: Actually, the first-line answer is that we’re a consumer company, and we want to speak directly to users and get them to download our products in order to address pain points that they are experiencing. We want them to be happy with our products and to be able to find the stuff in their inbox and to deal with it efficiently.

That’s much, much easier than trying to convince an IT department to get Xobni for their entire corporation.

That being said, a lot of the more progressive IT people have contacted us proactively and said, “We’re a company with 200 or a company with 5,000 people. We want Xobni on every single one of our desktops.” Of course, we encourage that, and may work out licensing that meets their needs.

I think in the future, we might have a support model specifically around this type of deployment, but that really hasn’t been worked out yet, mostly because the company is quite young and also because we are very focused on making a brilliant product that consumers really enjoy.

Scott: In the open source world, there’s the notion that the source code is free. People can download it, and companies have to figure out a way to monetize around that. They might sell support or have a more enterprise version that they charge for, with features that aren’t in the free one.

As you look at the landscape, what’s your impression around business models? Do you have a sense that the traditional model of having people download an eval version and then pay for the full version is still viable?

We run into the impression on the part of some people that everything’s moving to an open model and that open source software is built on a superior development methodology. On the other hand, there are new and innovative companies like yours that didn’t choose that route.

Do you feel that the traditional business models are alive and well, or do you see things moving to that “give it away and figure out how to charge for something” approach?

Gabor: People ask us this question a lot, and it basically comes down to, “How are you going to make money? Can you really charge for an add-in?” Our answer is that right now, we’re not focused on that. Right now, we are very focused on building a great product. We believe that if you build it, they will come.

And if you have millions of users that enjoy your product every day, somehow you will always be able to make money. Like Google. Google didn’t get started by saying, “Oh, we have search. Let’s find out how to monetize it.” They were, in the first few years, just focused on getting search better every single day, and later they found ways to monetize it.

There’s a story that the initial Google pitch stack was 70 pages long and explained how they would make money by selling search technology to enterprises, which was completely wrong at the time, but they later found a different way to monetize it, which has worked pretty well for them.

Our mindset, right now, is the same. I think we’re just building a great product and we’re going to find out how to make money eventually. People are asking us to buy pro versions. People are asking us whether they can pay if they want to install it into an enterprise.

We’re definitely looking at these models, but it’s really too early to take those conversations on in earnest, though–that still needs to wait six months or a year.

Sean: That’s fair. Let me ask you a different kind of question. There are obviously many, many good open source applications. At the same time, though, the little holy duopoly of Exchange and Outlook never seems to really find a strong open source competitor. I don’t think that people love it, but it’s the best that the world has at the moment.

Every time you try some other combination, whether it’s Zimbra or Evolution or Thunderbird or Sunbird or you try some other backend, like IMAP, POP, gmail, or whatever, they never seem to have that fidelity.

Why do you think the Exchange and Outlook combination never really sees the same strength of potential competitors as RedHat versus Windows, or MySQL versus SQL Server, or any of these other parts of, let’s say, the LAMP stack? Why is it that in that one spot, there doesn’t seem to be a parallel, even though there is almost everywhere else?

Gabor: I think you’re totally right, and I don’t know why that is. There are definitely great applications out there, such as the experience that Gmail provides. Also, Zimbra is great, and there’s just a fantastic variety of new applications in this field.

I think that Microsoft has the strength they do in the enterprise world right now because they know exactly what features they want, and they know exactly what these customers need and how to get into these enterprises. And once you have Exchange Server installed, it’s pretty hard to switch, because Exchange and Outlook have tremendous lock-in. I think that’s the key to their success and to their strength.

Sean: As a follow up to that then, since you came from Google, it’s an easy one to throw out there. What do you think about someone like Google’s relationship to open source? Or to give another example, what about Apple? Apple is seen as this amazingly open company, but they’re certainly not very open when it comes to their roadmap, or talking about other stuff like that.

With Google, you’ve got a company that builds a lot of stuff on essentially free open source, but yet they build a lot of commercialization on top of that, so there’s an interesting story there I think. Do you see any tension there?

Gabor: First, I have a really high opinion of Google, and I think that they’re excellent at software development. Also, I think their fundamental tenet of not being evil and doing good for the world is genuine, and they’re certainly sticking to it. They’ve open-sourced a lot of software already – such as Google Gears or protocol buffers, and I’m sure they want to be open and share the wealth with the world, but have to balance that with being a strong company that can generate profit.

Scott: During development, does open source fit in anything you do? I know as a developer that open source is really, really popular as developer tools for unit testing, automated builds, and for all kinds of different things. From the perspective of peeling back the covers of your organization, where do you find it useful or do you not really use it for much?

Gabor: We do use open source for build tools, release tools, unit tests, and so on. The infrastructure provided by the open source community is very useful to us, and I could imagine us contributing once we have a little higher level of resources, but so far our interaction has been very limited.

Scott: Would you mind giving a plug for some of the build tools and development tools that you really like?

Gabor: There’s always the wish list that you have. I think that NAnt, NUnit, or CruiseControl are really good, but I wish that they could each do this one more thing. There are other tools we use, all of which are an integral part of our development cycle.

Scott: One of the things that we have observed before is that open source really shines in projects that are ‘by developers, for developers,’ rather than software that targets a non-developer user base. In the case of developer tools, for example, the people building the project have an intimate understanding of what the product needs.

Open source projects tend to be very good at building a lot of infrastructure stuff–operating systems like Linux, web servers like Apache, and so forth. That seems to be where open source really innovates toward excellence. On the other hand, the really end-user-focused innovation like the iPhone, Google Maps, and the Xobni UI examples we have been talking about don’t tend to come from open-source projects.

I feel like open source is very concerned with architecture and performance and modularity, but when it comes to building a peak experience product, a lot of that’s still coming from the Apples and the Googles and the Microsofts and those kind of companies.

Do you feel like I’m on to something there? And if so, do you feel like there are fundamental reasons for that, based on how the different kinds of software are built? Or am I kind of just drawing a circle around a few things I have observed and calling it a trend?

Gabor: I have a very controversial opinion around this issue, which is that if you want to make a product with a peak experience like the iPhone for example, it’s not as much about what you build as what you take away, and about making those cuts at the right point.

For these types of decisions, you need very strong product people who understand exactly what the consumer wants and who understand exactly what the minimal feature set is that covers the maximum number of usage scenarios.

And this is something that I, as a developer, am not necessarily good at. There’s always the impulse to think of the next feature, and I always want to add, add, add, add. That’s why all of the open source tools have zillions of command-line flags and options and so on.

Whereas, if you want to make a consumer product like the iPhone, you have to make the cut somewhere, and make those things that are inside of the cut just outstandingly good instead of making a lot of things that are sort of a 75% solution.

Scott: That’s interesting from the standpoint of a lot of work I have done as a consultant for software companies. They say, “We’re going to add this feature to the product, because one important customer wants it.” And especially with smaller companies, a lot of times they can be pushed around by their bigger corporate customers to add things into the product.

Even something like MySQL does a certain amount of non-recurring engineering, and they add a certain amount of features, because a person wants it really bad for some specific scenario. I’ve never heard anyone explain it exactly the way you did, but that makes perfect sense. The iPhone’s good not because of what’s in it–it’s good because that’s all that’s in it. They really did just focus on making those things really good.

So how does a company like yours balance those two impulses? You must run into people who say, “We’d buy it if only you would add this feature.” How do you balance that with the philosophy of less is more?

Gabor: I think this question comes back to what I said earlier, which is that we’re really focused on consumers. We don’t, as a company, listen to the types of people who say, “If you just added this feature,” because right now we’re not even selling a product. So, the bottom line is that we can’t really consider a scenario where if we just added the foreign feature, we’d have $100, 000 more revenue.

And a related thought is that it’s been very painful for me personally, because I’m one of those “add, add, add” people, and other people on the engineering team are as well. Xobni used to have in the private data stage two tabs, which were “email” and “organize.” And one was about your email and the other one was about your calendars, your appointments, your outstanding tasks and so on.

We found through the customer experience improvement program that I talked about earlier, where we send back anonymized data about features if you opt in, that no one was using it.

We thought, “We built this great thing and no one is using it. Why? We should make it more prominent and maybe we should make the button blink or something.”

[laughter]

But then really, honestly, this is really and indication that people don’t want that functionality. And we decided to cut it out. It was probably one of the best decisions that we ever made, because we focused the product, and we made the things that people really do use even better. It’s worked great for us–it was just very painful at the time.

Scott: That reminds me of Firefox. We had Netscape, and then Internet Explorer became dominant, and it kind of became the browser with one more thing and one more button. And when Firefox first came out, it really made people sit back and ask what they really needed from a browser. They asked whether they really needed a lot of extraneous stuff, and a lot of them decided that Firefox really had everything they needed.

When it kind of caught fire, I think it was because it had that minimalist approach. I wonder if it depends on the size of your target, in the sense that if you target a niche, then you may need to be able to get pushed around by the niche a lot. And you may need to build a feature for a customer, because the niche is only so big.

On the other hand, if you are targeting millions of users like Xobni did, maybe you’re automatically freed from a lot of that. In essence, it seems like you asked where the pain point is for hundreds of millions of users, and that freed you from trying to be all things to all people. You just have to be the right thing for a bunch of people.

Gabor: Exactly. But finding that spot is really hard.

Scott: I can imagine.

Sean: Well, thanks for a great conversation. I’ve really enjoyed this.

Gabor: Absolutely. Thanks for setting it up.