Not The User's Fault

Over the last couple of weeks I’ve been participating in the beta test of upcoming game Starcraft II. In an interview with Gamasutra magazine, the lead Starcraft II designer, Dustin Browder, had this to say about the data they have collected from the beta test:

The danger with a lot of this data is that you have to be very careful how you use it. With unit stats, I can tell you that, for example, in a Protoss versus Terran game, 12 percent of the time the Protoss build carriers. And when they build carriers, they win 70 percent of the time. You could say, “That must mean carriers are overpowered!”

That’s not really true, though. It could just be that as you get towards the end of the game, if the Protoss have the extra resources to waste on a bunch of carriers, they’re probably going to win anyway.

…

Of course, it doesn’t mean the carriers aren’t overpowered either. That stat alone actually tells you nothing. It’s a very dangerous stat. If you listen to that stat, you can make all kinds of mistakes

…

If we look at the stats and we say, “This doesn’t actually back anything we’re experiencing online,” I’m very suspicious of that number. We get information from a lot of different sources, and then we use the other sources to refute or corroborate. We look at another source and say, “You know what? What they’re saying online matches my play experience, and it matches the stats. This seems real. Let’s talk about what some possible fixes can be.”

So very, very relevant to Test Pilot and Firefox!

Design Lunch is back this week. Back with a vengeance !

John Wayne Hill is one of our interns, working with the User Experience team this summer, and he’s working on the design of the new Home Tab post- Firefox 4. He’s looking for feedback on his ideas for the design.

For some more context on what the Home Tab is going to be, you might want to check out Alex Faaborg’s blog post “Browsing Your Personal Web”, as well as the Winter 2009 design challenge which invited people to submit Home Tab designs.

As always, the Design Lunch is at the Mountain View office, in Ten-Forward, at 12:30pm pacific time this Thursday (June 4). You can watch it remotely on Air Mozilla and/or call in by phone to ask questions; see instructions here.

Hope to see you there!