Avoiding a common research pitfall
Years ago, I was conducting in-depth interviews, usability tests, and brainstorming sessions for global retailer. Since the team was a cross-functional group, including success, development, and a VP, we’d spent hours training everyone on how best to conduct and observe users. Given all the work we’d already put in, we all were incredibly excited to be there and get started.
However, one of the team members dominated his sessions. He interrupted users to finish their sentences; talked for minutes about why he agreed with them; helped them complete tasks; and, in one instance, took the mouse from a user to show them how the system worked. The results from his sessions were practically unusable. When looking through the recordings and transcripts, we didn’t find insight into participants’ thoughts and mental models, we found his.
Unfortunately, this scenario is all too common. Moderators (new and old) can easily get wrapped up in the conversation and, in the end, accidentally overwhelm a session with their own comments. Instead, you must stay intently focused on the user and their responses in order to understand their experience and point of view.
The participant is the star — don’t interrupt.
In 2009, Kayne West had the best interruption of all time. It was the “I’ma let you finish…” heard around the world. He walked on stage during an award speech, snatched the microphone from Taylor Swift, and proceeded to disparage her win.
Taylor was stunned and didn’t know how to respond. Even after he walked away, she stood there silently, looking out into the audience. We don’t hear her speak again in that segment. While this was awful for Taylor Swift, it offers us a very clear example of what not to do as it shows how interruptions can completely derail words, actions, and thoughts.
By interrupting, you can break a person’s thought process and lose what they were going to tell you. So, it’s incredibly important to give users sufficiently time to answer a question before interrupting, clarifying, or asking a followup. Waiting silently can be hard, but remember that you’re there to get honest, unfettered answers, which you won’t get unless you let the person talk.
Of course, sometimes, you might feel that they have been silent for too long. If that’s the case, use a prompt to get them talking, such as asking them to tell you what they’re thinking or to recap the last few seconds for you.
Guide and reflect, don’t lead.
There are many ways to accidentally influence a participant’s responses, from the setting to question phrasing. You must be incredibly mindful in order to maintain a neutral moderation.
To me guiding a conversation means being able to refocus, bring the participant back to the question, and dive deeper, if necessary. This isn’t always easy — participants interrupt their own thoughts, switch direction multiple times in one breath, give hard-to-understand analogies, and, sometimes, even stop talking mid-sentence.
- If the question was only partially answered, use the 5 Whys to dive deeper and learn more
- To refocus the conversation, use phrases like “You started out by talking about…would you elaborate on that?” and “Earlier you said…why is that?”
- When an answer is long and/or complex, summarize their answer and reflect it back saying “Ok, so I heard…did I miss anything?”
Be Prepared and Active.
It is almost impossible to practice too much for a user session. There are usually curve balls so knowing the script and understanding your questions will allow you follow your participant down a path without losing track of your goals.
- Sticking to the script is great, but more importantly, stick to your goals. New questions and directions are totally fine, but make sure still get the answers you need otherwise you’re adding holes to your data set.
- Double check the setting. Is the lighting harsh? Do you have a beverage available? Is the chair stable? Is all of equipment queued up and ready?
- Be Present. Give the participant your focus and attention.
- Show That You’re Listening. People are often more perceptive than we think and will notice our feet are tapping or when we look at the clock.
- Have you thought about what you’ll do if you have extra time? Don’t waste it! Be prepared with additional questions and tasks to get the most data you can.
If nothing else, practice, practice, practice. Simply running through your session with colleagues or friends will help you identify and understand the unique things that you might be doing to bias your session and influence your data.