Not The User's Fault

In light of the idea that openness is hard work and that improving the discovery path for new contributors is a constant struggle, I’d like to do something to help make Test Pilot a little more discoverable, and possibly gain some useful feedback as well.

Why not publicly post the code for upcoming Test Pilot studies? The Mozilla community can check it out and verify for themselves that we’re upholding our promise not to collect any sensitive or personally identifiable data. They can also point out anything that’s potentially wrong with the code, things that we’re overlooking that might make the collected data less accurate, etc. That sort of feedback would be very useful for us developers.

Of course, it’s not like we’ve been keeping the Test Pilot study code secret up until now, or anything. They’ve always been, and will always be, available through our public Mercurial repository. (Look in the /testcases/ directory). But that’s not exactly easily discoverable.

So, I’m going to start announcing upcoming studies on this blog and linking directly to the code, to make it easier for interested parties to offer feedback.

The next study we’ll be releasing is called the Firefox 4 Beta Interface Study v2. It’s a revised version of the earlier Beta Interface study that we ran near the beginning of the Firefox 4 Beta program. It’s been updated to include instrumentation of the new features in the beta, such as Sync, Panorama, and App Tabs; we’d really like to know how many people are using these new features, and how they’re using them – e.g., how often do people sync? If they use app tabs, how many app tabs do they have? If they use Panorama, how many tab groups do they use? And so on. We’ll also continue tracking the same things that the first version tracked – frequency of use of menu items and toolbar buttons. That way, we can look at how people are using the toolbars and menus now,
compared to how they used them earlier on in the beta cycle (summed up in this heatmap), and see if there are any trends there.

Of course, as always, we’re not collecting any URLs, search terms, names of sites, or names of bookmarks. You don’t have to take my word for it; take a look at the code and see for yourself!

You can read the latest Firefox 4 Beta Interface Study code on the web via the Mercurial repository.

Or you can take a look at the Bugzilla tracking bug for the study; the code is attached to the bug.

Finally, the code for the study is also mirrored on GitHub, so feel free to look at it there if you prefer to use GitHub’s code review tools.

So pick your method and take a look at the code; comments, questions, and criticism welcome.

One of the reasons that Openness is a lot of work is because you need to create a path for new contributors to A. discover your project and B. figure out how to make their first contribution.

It’s not easy to get this path right! I myself have “bounced off” of several open-source projects, so I know what it’s like. Potential contributors come in eager to contribute something to the project, but they get confused about how to proceed, or they’re met with a total lack of enthusiasm, or they have their contribution rejected due to technicalities like not following correct coding style. (I know that coders have strong feelings about whether to put spaces inside of parenthesis or not, but is it really more important to enforce this than to gain contributors?) Anyway, they get discouraged, lose their motivation, and “bounce”. Even when people are willing to contribute for free, their time is still valuable to them, and if they don’t feel a sense of progress soon, they’ll invest their time elsewhere.

I recently gave a recruiting talk at a university, and met a bunch of students who were eager to contribute to Mozilla. They asked me how to get started, and I realized that I don’t have a good answer. If someone wants to contribute to Test Pilot (yes, please!) do I start by directing them to the Google discussion group? The #labs IRC channel (which is dead most of the day)? Do I point them to the open Bugzilla tickets? Or do I ask them to read several pages of Wiki documentation (some of which is may be out-of-date) just to get up to speed, before doing anything else?

If you want to get involved in Test Pilot, the openness is there and the information is there; but the information is spread out across a world of different websites, mailing lists, and other services, and it’s all rather overwhelming. Where does a prospective volunteer get started? This is not a problem unique to Test Pilot, of course; all of Mozilla struggles with it to some extent.

I’d like to hilight a couple of links that show promising approaches to removing obstacles from the discovery path.

First is the work Richard Milewski, a community member (and frequent Design Lunch participant) who pointed out to me how hard it was for someone outside the Mozilla organization to make sense of the raging river of information we output and figure out what was important. Rather than just complaining, though, he went ahead and did something about it – he built a website called Mozilla Express to aggregate in one place all the information sources he was interested in.

The second is OpenHatch.org, which aims to be a one-stop portal where you can find open-source projects that need help, find something that needs doing, and find out exactly how to do it.

I’d be interested to hear from you about your experiences on either side of the discovery path – if you’re in a project, what have you tried to do to make it easier for newbies to get started? If you’ve been that newbie trying to join a project, what was your experience like and what made you stick with it or give up?

You can’t just make something “open” and expect magic to happen. Openness is a lot of work. This is true whether you’re making an open-source software project, a website with user-generated content, a political movement, a charity, or any other kind of organization where you expect volunteers to show up and start doing work for you.

I emphasize this because openness has become quite the buzzword over the last decade, and I worry that people are starting to attribute near-magical powers to it. I wonder if books like Here Comes Everybody: the Power of Organizing Without Organizations by Clay Shirky, or The Wisdom of Crowds by James Surowiecki are over-selling the idea. Being in Silicon Valley I hear a lot about startups based on a “crowdsourcing” business model, or new open source projects run by people who assume that just because they open-source their code, they’ll magically get contributors. Lately the phrase “… ANYONE can participate!” seems to be de rigueur in every presentation.

Let’s say I’ve got an idea for a software project and I want to make it open source. So I put up a public code repository on my website, write a page about my patch submission policy, and start an email list for discussion. Great! I’m done now, right? I can declare my project “open” and go back to hacking now?

This project may be “open” by a technical definition, but if you look at the commit log, I’m still the only one working on it or using it. Where’s my windfall of free volunteer effort?

If I want to be open in a practical and not just a technical sense, I need to get people interested in participating. That means proselytizing my project, building a community, selling people on the idea: doing marketing. That’s a lot of work right there, and none of it gets code written.

Next, suppose some interested volunteers contact me and ask how they can get involved. If I say “Go read the issue tracker and find something to do, I’m busy”, I guarantee they will not remain interested for long. I need to invest some time in building a connection with each and every volunteer I want to integrate into my project. I need to explain to them how things work, answer their questions, find easy tasks to give them, and continue to act as their mentor until they understand the project as well as I do. I am making a substantial investment up-front that I hope will pay off in the long term. Until then, it’s an additional burden on top of my existing workload.

(more…)

Hey Internet,

I’ve been neglecting this blog all summer. I was traveling, and it was crunch time to get Test Pilot in Firefox 4. (..and, uh, I was playing kind of a lot of Starcraft 2.)

There’s a lot of stuff I want to write about here, though. These are interesting times for interaction design, open source, Internet freedom, and all that other stuff I care about.

So, consider Not The User’s Fault hereby re-launched.