We’re trying something new with the “Week in the Life” Test Pilot study. Instead of running just once per user, it will automatically recur about once every two months (60 days, to be precise) and run for one week each time. The idea is to let it recur over the course of a year, and see whether we can detect any long-term trends in the data that would indicate user habits changing over time. For instance, a lot of us who work in web browsers have a hunch that our users are using bookmarks less, and tabs more, compared to a few years ago. But does the data actually support this?
Because the “Week in the Life” study recurs, and because we never submit any data without the user’s explicit permission, we’ve got a potential user experience problem: Test Pilot is going to ask you whether you want to submit the data from the study every time it recurs. Do you want to submit the data? Do you want to submit the data? How about now? How about now, huh? How about that data, do you want to submit it?
Isn’t it annoying being asked the same question over and over again?
To mitigate this problem, I’ve added a new UI widget to the Test Pilot status page:
The principle of “Don’t pester the user” is important, but so is the principle of “Make sure you have the user’s permission before doing anything with their data”. These principles are natural enemies. Finding a compromise between them is not easy! I know that my little drop-down menu is not a perfect solution. What do you think? Is it self-explanatory enough? Too wordy? Is there a better approach to this problem?
December 8, 2009 at 8:47 pm
I have test pilot at home and school. I didn’t know the “Week in the Life…” was recurring but I did purposely select, “Always submit data…” I was thinking that that select-box was for test pilot overall and not just that particular test.
December 8, 2009 at 8:59 pm
Hm, that’s interesting. I guess as it stands now it’s kind of ambiguous, isn’t it? We should make it more clear that it applies only to that one study. It’s already pretty wordy, though. I’m worried that if I add more explanatory text nobody will read it.