Differeneces in tab usage by age and sex?

Jeremy Singer-Vine at Slate magazine has posted his own analysis based on Test Pilot’s Week-in-the-Life dataset!

He looked at tab use, and while his general conclusions are similar to what we found from the earlier tab study, his chart of tab use by demographics shows something interesting I’ve never seen before:

Jeremy rightly points out that the data behind this graph may not be the most reliable, due to the severe undersampling of women in the Test Pilot user base, as well as the fact that most Test Pilot users who submitted data did not fill out the optional demographic survey. It should be taken with a large grain of salt. Still, by suggesting that there might be significant differences in tab use patterns by age and sex, it points to what might be an interesting area for future experimentation.

This might be a good time to remind you that the deadline of the Open Data Competition is December 17. So there’s still time to do a visualization of your own and enter the contest!

 

Patrick Dubroy’s CHI 2010 talk

While we were working on Test Pilot studies, Patrick Dubroy was doing his own research on Firefox tab usage patterns. He presented his findings a paper at CHI 2010 last week. Now he’s put up an excellent blog post summarizing what he found out. Go read it right now!

 

Two opinions on 3.6 tab behavior

My wife strongly dislikes the new Firefox 3.6 tab behavior (where tabs opened from links appear immediately to the right of their parent tab, instead of at the extreme right of the tab bar).

I do like the new behavior, because by keeping related tabs closer together, it reduces the amount of time I have to spend interacting with the tab-bar scroll buttons (my least favorite UI element in all of Firefox).

She dislikes it for consistency reasons: when you open a new blank tab, it still appears at the far right. So now tabs can appear in two different places, depending on where you opened them. It violates the principle of consistency, which is generally considered one of the most important UI principles. This inconsistency hasn’t really bothered me personally. I’m not sure why; maybe it’s because opening a tab through a link, and opening a new blank tab, feel like different actions to me. There’s a difference in what I’m thinking about. But I can certainly understand how it feels like a consistency violation to other people.

My wife also doesn’t like that there’s no way to change Firefox back to the old behavior without going through about:config. (If you’re interested: type “about:config” in the location bar and hit enter, then do a search for a preference named browser.tabs.insertRelatedAfterCurrent and set it to True or False, as you like.)

 

Test Pilot Results!

We’ve had over 5,000 users submit data from the Test Pilot tabs study!

Considering that people had to first hear about Test Pilot, then opt in by installing the extension, then opt in again by choosing to submit the data, 5,000 is a really good number. Better than we had any right to expect, certainly.

Over the past couple weeks, I’ve been sifting and analyzing the data, and working with Blake Cutler from the Mozilla Metrics team to generate graphs of interesting statistics about tab usage. I’ve just put up a results page showcasing several of these graphs.

We’ve also posted samples of the aggregated data which are free for anyone to download and use. There was some discussion on my previous post about how to aggregate the data in a way that was still useful to researchers. What we ended up doing was building files that include row-level data from a random subsample of the users that fit particular criteria. It’s stripped of any information on the language/locale, operating system, or installed extensions for any individual user in the sample.

Third-party researchers have already begun using the data to do their own analysis! Andy at Surfmind.com has a post containing some very cool-looking visualizations and has proposed an interesting theory about there being two classes of heavy tab users.

 

Don’t worry about “skewing” the data

I get a lot of people contacting me by email, IRC, forums, or blog comments to say that they’re “worried that if I join Test Pilot I’ll skew the data” because “I’m sure that my tab usage is atypical”.

People! Don’t worry about being an atypical user!

First of all, we have already had almost 5,000 Test Pilot data submissions. One outlier isn’t going to do much to “skew” a data set of that size.

But more importantly, you shouldn’t assume that you’re abnormal. We don’t know what “normal” tab usage is! That’s why we’re doing this experiment, to find that out. If we started out with an idea of what normal tab usage looked like, and threw out things that didn’t match our preconceived notions, that would be a clear case of experimenter bias. Then we’d really be skewing the data.

For instance, I was surprised to find out that there are users who have over 500 tabs open at a time. Over 500! They’re surely outliers, but they’re not abnormal users &emdash; they’re just users. That number isn’t skewing the data &emdash; it is the data. Thanks to those users’ participation, we now know that having 500 tabs open is something that people do with Firefox, something we might not have known otherwise.

As I said in a previous post, I do believe we have a major oversampling of the power-user / early-adopter demographic in our current Test Pilot user base, and that we need to work on fixing this by reaching out to a wider sample of users. But note that word: wider. Excluding yourself because you think you’re atypical isn’t helpful. If you really want to help our sample — and I’m touched that so many of you do want to help " the best thing you can do is to let your less-techie friends know about Test Pilot.

 

Design Challenge, summer 2009

If you take a look on the right side of the page, you’ll see a Mozilla Labs design challenge logo. That’s because we’re running a Summer 2009 Design Challenge, which is currently open to submissions! The topic is “Reinventing Tabs in the Browser” (a topic I am very interested in).

Click the logo in the right sidebar (or click here) to go to the page with the detailed description of the design challenge, including the deadlines and the instructions for submitting your concept. Go check it out!

 

Updated tabs UI for Weave on Fennec

Fennec Beta 1 was released yesterday! And there was great rejoicing!

To go with it, we’ve released Milestone 5 of Weave, including a lot of bug fixes and UI polish improvements to the Fennec version, and an improvement to the caching algorithm that makes syncing up to 3x faster. Weave is now hosted on addons.mozilla.org, so you can get it there.

Anyway, I noticed that a lot of people have been linking to my previous post, with its embarassingly ugly proto-UI for selecting a tab to open. I wanted to show you the new and greatly improved UI, with a more informative display, selectable sort order and much larger click targets:

 

How do People Use Tabs?

I saw a great presentation yesterday by Patrick Dubroy, who presented some preliminary results from his research on how people use tabs in the real world.

[UPDATED: You can now read Patrick’s whole presentation on his blog.]

The sample size was small (22 people) so we should be cautious about any conclusions we draw from this study, but it’s pretty exciting to have any real scientific data at all on this question (as opposed to anecdotes, personal observations, etc.) The study also points in some exciting directions for further research.

The big thing I took away from Patrick’s presentation was that among heavy web users, tabs are enabling new styles of browsing behavior that rely less on bookmarking and less on the back button. According to Patrick, the back button is getting used less and less as the years go by, and for all but two of his test subjects, switching tabs was a more common action than hitting “back”.

This change seems to coincide with the rise of web applications, where a user might spend a long time interacting with data on a single page. Web applications live comfortably in tabs (I always keep gmail open in my leftmost tab, for instance) but did tabbed browsers help popularize web applications, or did web applications help popularize tabbed browsing, or neither?

One anecdote from the presentation really stuck with me: apparently some users love opening multiple links in separate tabs, but they use a laborious manual workaround to do so, because they don’t know about command (or control) -clicking a link to open it in a new tab. This tells me that the feature needs to be more discoverable somehow.