iA


Time-Aware Research

by Nate Bolt. Average Reading Time: about 3 minutes.

An excerpt from our 2010 book, Remote Research, published by Rosenfeld MediaThe soul of remote research is that it lets you conduct what we call Time-Aware Research.

By now UX researchers are familiar with the importance of understanding the usage context of an interface–the physical environment where people are normally using an interface. Remote research opens the door to conducting research that also happens at the moment in people’s real lives when they’re performing a task of interest. This is possible because of live recruiting (the subject of Chapter 3), a method which that allows you to instantly recruit people who are right in the middle of performing the task you’re interested in, using anything from the Web to text messages. Time-awareness in research makes all the difference in user motivation: it means that users are personally invested in what they’re doing, because they’re doing it for their own reasons, not because you’re directing them to; they would have done it whether or not they were in your study.

Consider the difference between these two scenarios:

1. You’ve been recruited for some sort of computer study. The moderator shows you this online map Web app you’ve never heard of, and asks you to use it to find some random place you’ve never heard of. It’s This task is a little tricky, but since you’re sitting in this quiet lab and focusing–and they’re not going to let you can’t collect your incentive check and leave until you finish–you figure it out eventually. Not so bad.

2. You’ve been planning a family vacation for months, but you’ve been busy at work so you procrastinated a bit on the planning, and now it’s the morning of the trip and you’re trying to quickly print out directions between finishing your packing and getting your kids packed. Your coworker told you about this MapTool website you’ve never used before, so you decide to give it a shot, and it’s not so bad; that is, until you get stuck because you can’t find the freaking button to print out the directions, and you’re supposed to leave in an hour, but you can’t until you print these damn directions, but your kids are jumping up and down on their suitcases and asking you where everything is. Why can’t they just make this stupid crap easy to use? Isn’t it OBVIOUS what’s wrong with it? Haven’t they ever seen a REAL PERSON use it before???

Circumstances matter a lot in user research, and someone who’s using an interface in real life, for real purposes, is going to behave a lot differently–and give more accurate feedback–than someone who’s just being told to accomplish some little task to be able to collect an incentive check. Time-awareness is an important concept, so we’ll bring it up again throughout our Remote Research book to demonstrate how the concept relates to different aspects of the remote research process (recruiting, moderating, and so on).

bttf2

(We understand that as a commercial entity, there is no legal premise of fair use for this image, so we’re clearly violating all kinds of copyrights by using it…)

Remember that diagram in Back to The Future II? Doc argues that messing with time has sent the world crashing hopelessly toward an alternate reality where things are horrible: the “Wrong 1985.” And that’s sort of what happens when you try to assign people a hypothetical task to do, at a time when they may or may not actually want to do it: you’re meddling with their time, and it’ll create results that look like the real thing but are all wrong.

When you schedule participants in advance and then ask them to pretend to care, you’re sending your research into the Wrong 1985. If you don’t want to create a time paradox–thereby ending the universe–you should do time-aware research.

  • http://www.linkedin.com/in/10uxd Tristan Naramore

    This is one of the more eye-opening posts I’ve seen regarding user testing. I’ve always shrugged off the fact that usability tests happen under rather artificial conditions, whether in the lab or remotely. I figure if the moderator is good and the test script is solid, we can use sophisticated analysis techniques to smooth out any biases introduced by the test environment itself.

    That said, I think the idea of catching users “in the act” is absolutely fascinating, if not fraught with challenges: Planning would be tricky; Getting test subjects to cooperate/behave under real-life conditions; Real-time moderating could be mind-bogglingly difficult if user is distracted or multitasking.

    Still, I’d love to hear more about this concept of Live Recruiting. I look forward to reading your book.

  • http://boltpeters.com bolt | peters

    Thanks Tristan! All the challenges you’ve brought up are exactly what Chapters 3 and 5 of our book will deal with. Our book will be out at the end of January, if all goes according to plan. You can also check out our company website for more about live remote research: http://www.boltpeters.com/

  • Pingback: Interview with the Authors of Remote Research | UX Booth

  • Pingback: 好文翻譯:Quick and Dirty Remote User Testing | UX map

  • Pingback: How A Story From World War II Shapes Facebook Today « NYC Real Estate News

  • http://twitter.com/switchedonUX Sarah Keogh

    Really nice post, and a great way to explain to people why they can’t ‘make up’ stuff and need to ask realistic tasks, taking into account real contextual information. I would take slight umbrage to the fact that time aware research is the soul of remote research however. I got really predictive results in over 3 years of testing week in week out a different design or application. We tested 30 times a year perhaps, and still got really insightful results. Of course you are right, catching a person just as they are about to ACTUALLY do something is better, but well researched scenarios and following good conventions of asking them to explain their real world scenarios so we are aware of the pressures our design has to work under also works pretty well.