The Lean Product Playbook: How to Innovate with Minimum Viable Products and Rapid Customer Feedback by Olsen Dan

Author:Olsen, Dan
Language: eng
Format: epub, mobi
Published: 2015-07-02T20:02:03.322000+00:00

HOW MANY CUSTOMERS SHOULD I TEST WITH?

I recommend conducting user tests with one customer at a time for the best results. You can speak with more than one customer at a time, but you usually get suboptimal results due to group dynamics. You especially see this negative affect in focus groups, which involve anywhere from 6 to 12 people at once. Some participants may not speak their mind openly for fear of being judged or criticized. One or two outspoken people often dominate the discussion, drowning out other voices. Participants also often experience groupthink, which leads to all or most of the group artificially converging on the same opinions—and which leads to inaccurate data.

By speaking with one customer at a time, you don't experience any of those negative group dynamics, and you're able to have a richer, more in-depth conversation. The customer is much more likely to speak up and share his or her true feelings, especially if the moderator is the only other person present. In my experience, the more observers you have, the more worried about being judged some customers can be. Many researchers like to have a note-taker present so they can focus on conducting the user test, which is fine. I personally prefer taking my own notes—that way, I'm certain that my insights get captured, and it's truly a one-on-one interview. If you want observers to be able to watch testing sessions live, then using a webcam that projects the video feed to a monitor in another room is a good alternative. If the user test is remote, then the observers can join the screen sharing session.

I have conducted user testing with two and three customers at a time. It worked out fine because I was getting feedback on printed mockups; we were all seated at a table and could see and point to the papers on the table. It probably wouldn't have worked so well if each customer was looking at the designs on a laptop. I took this approach instead of one-on-one sessions because my client wanted to obtain the results of the research very quickly, and this allowed me to speak to more customers per day. Sometimes research subjects don't make their appointment; no-shows are just a reality of user research. So another benefit of having two or three people scheduled for each session was that I wasn't left twiddling my thumbs if one person was a no-show.

Product teams often ask, “How many customers should I test with?” If you talk to too few, you run the risk of not catching all the issues you need to address. And you might discover opinions that aren't really representative but not realize that's the case. On the other side of the spectrum, talking to too many people takes additional time and resources. You can go past the point of diminishing returns where you just keep hearing the same feedback and aren't learning anything new. I've found that testing in waves of five to eight customers at a time strikes a good balance.

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.