His eyes were dancing over the functional prototype on a 17" monitor. Tiny furrows were raising up on his brow. He was biting his lip so hard, he almost drew blood. Safely behind the glass, protected from the fury of his white-hot gaze, we could practically feel the heat emanating from his sandy brown hair.
Clearly, the interface wasn't faring so well.
If you haven't observed users interacting with a Web site in a formal testing environment, you're not alone.
Many seasoned UI designers that I know have spent little to no time having their work tested formally. They use Google Analytics or WebTrends and sift through their Web site metrics for benchmarks as to how those sites are performing. For those who are seeking a dollars and cents understanding of the value of usability, this is where you can track exactly how sticky your site is for your audience. For most clients, this is good enough proof of site performance.
Then, you step over to the other side of the glass. You actually see how, behind the metrics for a poorly designed site, there are a number of confused, angry, and upset customers.
While metrics/analytics programs are very powerful with regards to time spent with a system, what page elements are trafficked and clicked, and so on, they can't give us a mainline connection to our user's emotional state. Seeing people express their frustration with a product gives me 200% more insight into what's going in within the mind of someone using the interface -- more so than any other method of testing my Web UI designs.
There's definitely an "Aha!" when you've watched a series of people play with your designs -- especially ones who have no idea where you're coming from. Once you've lived through this process a few times, you're hooked.
Usability testing. Whoa. That's expensive, both in terms of time and hardware. Do we even have time in the schedule to do it?
You bet. If you can't roll a private beta, just throw away the 2-way glass, the fancy catered lunches, and the 2 hour sessions where you hammer away at every last detail of buying a pair of blue pumps through your shopping cart system. You need to get laser-focused on the interactions that will make or break your site.
Plus, some of the same observation techniques we use when testing functional prototypes and live Web sites in a formal usability testing environment can be easily scaled, in a relatively cheap way, to your studio practice. With new technologies like the Mino Flip camcorder or the Silverback app for any Mac that has an iSight camera, you can do guerilla usability testing with practically no overhead or heavy file transfers from camcorders. I think within the next few years, if we can disseminate the right tools to designers and developers, we can start integrating usability testing into our Web site and app designs on a very practical level.
If you've been thinking about testing your designs, here are some quick tips as to how to pull it off. Disclaimer: This is in no way meant to replace doing some research and developing a script, a release form, and practicing on an unwitting friend or co-worker before unleashing your usability mojo onto an unsuspecting test subject.
First things first...
Reread Steve Krug. You should definitely start with (re)reading Steve Krug's book on usability, Don't Make Me Think, as that covers all the basics of usability and provides a simple master script that you can adapt.
You need 4-5 users from a persona group to make your test client-facing. You can do an internal test with 3-4 users from a persona group and cover most of your bases informally. But be prepared to find 4-5 if you're going to try and get some recommendations in front of the client. (Thanks, Jakob Niselsen.)
If you are working with developers, testing needs to be integrated into their build cycle. If you're working in an agile manner, sometimes you can sneak the usability into a few sprints. Otherwise, you need to slot into the waterfall a spot or two where you can bang on the working interface.
If you have little to no time, test what really, really counts. You already know the key points in your user flows where people might get stuck -- especially if you're asking them for critical data, like checking out of a shopping cart or interacting with some kind of novel Flash thingy on your home page. Those are the ones that need to be tested first. If you have little to no time, go straight for what you know is most critical. Doing just two hours of testing on 5 people in your office could totally save your bacon.
Don't lead them on. Try not to tell people what to focus their attention on what you think is an issue, as in, "Hey, do you think that headline on page 2 makes sense?" Get their feedback in the context of fulfilling tasks. Then, afterward, develop a simple survey to walk through after their experience (both in written feedback and by rating ease of use of the system on a 0 to 5 scale). This will help you set a benchmark for user satisfaction if you are going to continue to make updates on the site and measure improvement.
Be mindful of who you cast to participate. If you're selling high-end bikes, try to find high-end bike owners. You could pull five people off the street to participate, but try to align them with your personas. If you've got really narrow audience groups, finding people can often take longer than the entire process of testing! A great trick is to save yourself the time by posting to Craig's List in your area and offering a small honorarium that's been factored into your project fees. Also, if you plan on doing usability testing a lot, start a database of people that you can call in last ditch for pizza and beer. Also, always overcast by one. A person will always cancel at the last moment.
Be sure they sign a consent form. If you're going to share direct feedback from your testing with a client, you should protect each user's confidentiality. At the same time, you also need to give people who participate the option of walking away mid-test. The consent form should cover both of these scenarios, and also clarify their compensation and the exact activities they will be participating in. There are legal ramifications here that must be covered here, even if the participants are your employees. If you're working with children, you need consent and a parent there for the test. (By the way, doing this with kids rock. They just don't pull any punches!)
When you're going to present results to the client, here's some simple ways to structure your presentation:
Limit your major findings to five major points or less — the ones that need to be addressed right away. Results from testing are usually ranked by level of severity, with the most pressing issues given the lion's share of the attention. Keep everything else to a list on a later slide.
Keep the findings logical, not chatty. This is one place where I actually advocate not being conversational in how you describe things to your client. "Half the users were really pissed that we didn't offer a back button in the shopping cart." Nope. Even if they were hopping mad and screaming, you shouldn't say it. You should...
Visually document the level of resistance to the interface, and serve it up as Exhibit A. Don't want to revise the wonky screen that your boss loves? Let's go to the videotape... I've seen clients that were adamant about preserving key interface features completely flip their mindset based on watching videos of people tripping over them. Repeatedly. However, that said:
Be neutral about what you got "right" and "wrong." Don't ever, ever consider saying "I told you so," because we don't do usability testing to just "prove ourselves right". Scenarios always come up where you didn't anticipate problems, and you're telling the client that your awesome strategy that you've been recommending since the new business pitch is totally unusable. Be humble. That's why we are user-centered designers, not self-centered designers.
Usability testing isn't a substitute for thinking about how users might use your interface. It lets you better understand the mindset and emotional world of your users, and how they see your site through their own eyes. Until Google Analytics has a feature called, "Just how pissed is your user now?", usability testing is a great way to get some critical objectivity on how your site is going to be perceived out in the real world... and without pissing off thousands of users in the process...