Blake (who generated all the cool graphs I’ve been using to present Test Pilot results) has used the session data from the Week-in-the-Life study to form an interesting hypothesis: That Firefox crashes per user follow a power-law distribution. If true, the power-law distribution means that “mean crashes” and “typical experience” are two very different things.

I should emphasize that the Week-in-the-Life study was not really designed to look at crashes, so I’d rather do a follow-up study specifically targeting this hypothesis before I support it with any confidence. But, as Blake says:

If our crash data follows a similar distribution, the average crash per user metric tells us little about the experience of a typical Firefox user.

Anecdotal evidence supports this hypothesis. While we all know people who swear by Firefox’s stability, we also know people who complain of frequent failures.

…

With this in mind, I suggest we use Test Pilot to run a longitudinal study of true Firefox crashes.

Agreed! And it’s great to see more hypotheses coming out of the Test Pilot data!