We are in the middle of another round of usability testing and after having many conversations about it this week, I wanted to share some of our learnings and ideas:
Ideas for Recruiting Usability Testers:
1) Look Around: We’ve had a lot of luck getting some amazing feedback from people we work near everyday. It’s a great way to introduce other people in your office building to your website and at the same time get some valuable feedback.
2) Ask the Pros: We’ve met many rock-star product and UI people at partner companies, customer companies, or even competitors. Surprisingly, many have responded when we humbly asked for their professional opinion of our site or a new design concept.
3) Be mobile: Sometimes, it is a pain for people to come into the office, so when we are flexible and offer to meet them at a coffee shop, restaurant, office, or class room….we have good luck and sometimes they will do it for free.
4) Be heard: Don’t be afraid to post on both physical (Panera Bread, Retirement facilities, Borders, etc,) and online bulletin boards. We’ve had the most success posting to Craigslist and offering a gift card to a fun store.
5) Be smart about your rewards: If you spend a little extra planning time, you can buy discounted gift cards or gift certificates in bulk.
6) Screening: If you are looking for a certain type of user, try using Survey Monkey or Zoomerang to get more information about potential testers before you invite them in.
Usability Testing Environment
1) Location: It’s best to be somewhere that the usability tester will not get distracted, but also helps if you are not in a completely quiet or lab-like environment, where the person feels uncomfortable or under a micoscope.
2) Give up all control: Let the user type URLs in and search on their own. You learn a lot about how people will actually enter your site, misspell words, use the mouse vs. shortcut buttons, etc.
3) Photographic Memory: If the user agrees, I would recommend using a screen and audio recorder…after sitting with about 5 users, things will start to blend together for you and your test session notes can be difficult to decipher, because the user is often moving faster than you can type or write (and you always want to be watching them).
4) Be clear about the time line: If a user thinks that the faster they blow through tasks, the earlier they get their reward, your results will not be as authentic. Tell them they have 5 minutes to complete a form, or that for the next 10 minutes you will be getting feedback about feature X, etc. It is also important to let them know they will be finished at a certain time, so they do not get impatient or wonder when the session will end. In my experience, 45 minutes is a great time frame for the testing and I usually plan for 5 minute prep and info gathering with a 5 minute end of test survey and a de-brief. Total time: 55-60 minutes.
Conducting the Usability Test
1) Scenarios: Develop about 10 fairly broad scenarios (ex. “You then search for a car based on your hobbies and important vehicle features.”) and a list of follow-up questions for each scenario after you have observed the user (ex. “What did you think about he list of hobbies?”, “Were the suggested vehicles reflective of what you searched for?”, etc.). This way, you can get good feedback even for people who are not very vocal. Not every person will need the follow-up questions, but they help when you are facilitating the usability test.
2) Set them at ease: Make sure the testers know that there is no wrong answer. “This is not a test. The only thing we ask is that you would use the site just as you would normally use any other site. Be honest about what you like and don’t like. You will not hurt our feelings. Learning from you is helpful and will only make our site better and our company more successful.”
3) Starting off: Resist the temptation to talk about the purpose of your company or website. You don’t want to guide the tester in any way. If you need to talk, just ask questions or talk about the process/time line.
4) Don’t help unless you have to: Sometimes it is hard to see somebody looking right at the big red button in the middle of the screen and then click on the small blue link on the top right of the page. You want to point to the screen and scream “it’s RIGHT THERE”. Don’t. Let them struggle, let them click on the wrong things and try to find their way back. When they are done, revisit and ask them why they looked at the link instead of the button. You will learn the most from these “alternate paths”.
5) Keep focus: If a tester gets distracted by something on your website or wants to talk about their new puppy, take note and gently guide them back to the task at hand.
6) Don’t be emotional: At a start-up, often times the people who work on the product are the same people hearing about what people hate. Remember that you are building the product for the user and not your personal taste. Be thankful for their honesty.
7) Don’t lead: Be careful when introducing concepts or designs to the user. For example, instead of asking “Do you like this design or the new design better?”, stay neutral and don’t talk about which is newer: “Tell me what you like and don’t like about each design. Which do you prefer and why?”
Take good notes: The data you gather during usability testing is valuable and you may need to refer back to it at times. Keep good records.
