holy crap, it’s andy budd.
holy crap, it’s mark trammell.
usability testing and user research should be at the beginning, middle, and end of of a project. regularly scheduled user research sessions can start interweaving user research into the entire process.
- Presenters
- Mark Trammell (Digg)
- Juliette Melton (User Experience Mgr, Lumos Labs)
- Nate Bolt (Bolt|Peters)
- Carla Borsoi (VP Research & Analytics, Ask.com)
- Andy Budd (Clearleft Ltd)
- Date
- Monday, March 16
Tools
- Most useful user research tools from the panel include direct questioning, interviews, and observation, Beginning–ethnographic/one-on-one interviews, Middle–analytics, a/b/multivariate testing, End–customer service feedback/surveys. Behaviors over opinions (one on one observations) or feedback in the course of using a tool. Surveys are a great way to access overall sentiment and demographic patterns. Task analysis.
- Least useful/helpful user research tools include eye-tracking (smoke and mirrors and problems with analyzing the results), focus groups cause too much groupthink that gets in the way of the detail you really need and are about opinions.
Culture of Genius
Requires that everyone is aligned toward the same goal. Can get into a situation where you assume that you know enough about usability that you can keep on knowing how your users are changing. Companies who do not rely on user research as heavily as others often iterate much more. So user research is for the rest of us.
Innovation in technology comes from a process of consuming feedback or iterating until its right. User research can help move your business model toward your user base.
Remote Testing
Remote testing is how you observe someone from another location (screen sharing, data tracking, etc.). Observation in the user’s native environment can be extremely more valuable than a theoretical lab test where usability testing isn’t being handled well. However, the location of the user’s environment may not impact usability itself, more the desires and sophistication of the user.
Getting the Team Involved
Show them! Tying user research into the organization can be difficult when people think everything is fine with a product. Silverback is a way to record the user as they are using the website. Sharing snippets of actual observed behavior can help prove that there is something wrong with an app.
Make it part of the development process! The more people who get to participate early in the usability testing the more integrated it will be into your process. Show a success you had from a decision made based on usability results. Market and become known in the organization as a researcher about user behavior. Be pro-active about letting people know what is going on. Find the person who cares and show them that area of the app and get them to champion your work.
Share the numbers! Repeat the statistics that make an impact until it becomes common knowledge.
Introducing User Research
Just go out and do it! Time can make it difficult to introduce user research, so just guerrilla test and get the testing done. Convince people that it is important (“we should care whether or not we’re building good shit”) and that you should use user research to drive the creation of the product, not just validate the success of a project.
Quantitative vs. Qualitative
A four week a/b test (retention, frequency, task completion) produces very utile information about the “what”. However, quantitative requires a high level of statistical complexity. Qualitative testing gives you the why (insight) behind the what, provides the story behind the data (satisfaction). The weight given to one or another depends on the culture of the organization. What is valued more, what can be used to make decisions that the company can get behind?
Cost
Qualitative testing can be cheaper to start with and can get the ball rolling (Silverback, SurveyMonkey). Guerilla research can be easier to keep going throughout a development process. Do just enough research to make decisions.
These are notes from a session at sxsw interactive. My own take on topics are mixed in with what the presenters were actually saying, so do not assume all of this content is my own.