Ellen Isaacs: Interviewing customers

For the past five years or so, I've been teaching a one-day tutorial on the process of interviewing customers. The tutorial is intended to help technology designers, developers, and marketers to better understand their customers' underlying needs so they can better make decisions about feature requirements and design choices, and also to help uncover new product ideas. The tutorial grew out of my experience at Sun interviewing customers before each major project. I found that my previous experience as a reporter, as well as the insights I learned from my degree in cognitive psychology, gave me some useful skills in conducting interviews and in understanding what you can and can't learn by asking people directly what they want. My psychology training also helped me tune a methodology for distilling the vast amount of qualitative data into a set of concise, specific findings.

I originally designed this tutorial as part of a customer-focus process at Sun, and then went on to expand it and take it on the road. I've given it twice at ACM's Conference on Computer-Human Interaction, once at the User Interface '97 conference, and several times on at private companies, such as Interval Research, Bose Corp., and Icarian, Inc. I'm happy to give it on request to groups of 10-60 people. It's a highly interactive tutorial, with a heavy dose of demonstrations, examples of good and bad interview techniques, exercises, and discussion. Over the course of the day, participants develop interview questions, conduct two interviews, and do some preliminary analysis of their data. I'm pleased to say that at CHI '96, where they collected and analyzed extensive feedback data, this tutorial was rated 3rd out of 30 tutorials for "worth the time and money," and I was rated 5th out of 50 instructors for "effective communication."

The following is the description of the tutorial, as published in 1996 in the Conference Companion to ACM's Conference on Computer-Human Interaction in Vancouver, BC. Click here for a PDF version of this description. Some people have asked me to make the slides for the course available. If you would like to view them, please contact me, explaining how you plan to use the slides, and I'll send you the password.

Abstract

Product designers typically talk to customers in an effort to better understand their needs. However, without interviewing skills and an understanding of the types of information people can provide about themselves, interviewers may collect little useful information or even misleading information. This tutorial provides a practical approach to interviewing customers. It focuses on three areas: (a) the types of information you should (and should not) expect to learn from interviews, (b) good interviewing techniques, and (c) methods for analyzing the large volumes of information collected in interviews. The tutorial makes heavy use of demonstrations and exercises to give the participants hands-on experience with preparing and conducting interviews as well as analyzing information collected.

Keywords

Interviewing, customer needs, requirements gathering.

Introduction

Your company, traditionally technology-driven, has started using terms like "Voice of the Customer" and "focus groups." In its enthusiasm for becoming more customer focused, it has been encouraging all its employees to go out and talk to real live customers. You are asked to visit some customers to find out their "real" needs. What do you do?

For many people - engineers, tech writers, managers and human interface professionals alike - interviewing customers is an unfamiliar and challenging task. If they have not been trained in the mechanics of interviewing, the experience could be uncomfortable or uninformative or both. Even those who speak to customers frequently may find that they are not getting as much useful information as they would like. If they have not been trained in techniques to uncover customers' underlying and future needs, they are likely to return with a list of bugs and short-term feature enhancement requests, many of which do not fall within their area of interest. Worse yet, they could easily wind up with information that misleads them about how to make their customers happy.

This tutorial is designed to teach people how to interview customers to learn about their underlying technical needs. It does so in three main ways: (a) by teaching people what information they should (and should not) try to collect through interviews, (b) by training them in the mechanics of interviewing, and (c) by showing them how to analyze the large volume of data they will collect during the interviews.

It is worth noting that this tutorial does not arise out of any specific CHI tradition for gathering user requirements (e.g. contextual inquiry, ethnomethodology, participatory design), but rather focuses on skills for gathering customer information that may be useful from any of those perspectives. To the extent that any of those techniques involve asking users questions, the skills taught in this tutorial would be useful.

Also note that the term "customer" is used broadly. The tutorial is equally relevant to researchers who are designing prototypes for users, to in-house developers who are designing applications for their employees, and to engineers who are designing products for the market.

Content

The tutorial covers the following topics:

What you can learn from interviews
Preparing for interviews
Conducting the interview
Interpreting the data
Publicizing the results

The following sections describe each of these topics and discusses the means by which the information is taught in the tutorial.

What you can learn from interviews

This section discusses the type of information that can and cannot be revealed through an interview. It provides context by contrasting interviews with other methods for getting customer input, such as questionnaires, usability testing, focus groups, and field studies. It discusses the information that each technique is designed to collect.

The main point of this section is that interviews rely on people telling you about themselves, and people have significant limitations on the kinds of information they can accurately report about themselves. In particular, people are not good at predicting what they will like or will want. They are also not good at estimating how much they like a single option (as contrasted with comparing two options) [Glass and Holyoak, 1986; Anderson, 1985]. Yet the most common questions asked by many interviewers is "What features would you like?" and "What do you think of this feature/product we're thinking of building?"

Rather than asking people to consider hypothetical scenarios, interviewers should ask people to tell them about their current practices: what they are currently doing, how they do it, what they are trying to accomplish, what problems they face, how they handle those problems, etc. From this information, the interviewer can learn not only about customers' current needs, but also about opportunities for satisfying future needs. They can learn about limitations the customer is accepting without noticing, which are especially good indicators of useful products and features.

This section includes numerous demonstrations and an exercise to make it very clear just how poorly people can answer "what features would you like" questions accurately and how much better they respond to "what are you currently doing" questions.

Preparing for interviews

This section provides a collection of practical information about the preparation phase. It covers the following topics:

What information would you like to know?
How do you structure the interview?
How do you record the information?
Whom do you talk to?
Where should you talk to them?
How do you contact them?

An exercise is conducted in which participants prepare questions for an interview, which they will conduct later.

Conducting the interview

This section covers in great detail the mechanics of interviewing. It covers many interviewing techniques that are based on the following principles:

The interview is about them, not you
Ask open, unbiased questions
Ask the question and let them answer
Follow up
Adjust your questions to their previous answers
Ask questions in language they understand
Be flexible
Listen to their complaints, but look for other problems
Pick up on examples

Many good and bad examples are provided from recordings of real interviews. The participants are also given the opportunity to conduct an interview, using the questions they prepared earlier.

Interpreting the data

In this section, we focus on analyzing the data collected in an interview (or series of interviews). The participants are taught a bottom-up approach to data analysis that allows the major points and themes to emerge from the raw observations, rather than by imposing a preconceived structure on the data. The advantage of this approach is that it minimizes the influence of the analyst's pre-existing biases. In many cases, people who do not use this approach instead use the organization of the interview questionnaire to organize their findings, which sets them up to find only what they anticipated finding. Other major themes are unlikely to emerge.

During this section, the class participates in analyzing the data they collected from each other.

Publicizing the results

In this section, we briefly discuss ways of making the most out of the information within the organization. Customers are a valuable resource, and it is important that any group who spends the time, effort, and money to interview them share their information with other groups in a way that respects the confidentiality agreement with the customer.

References

Anderson, J., 1985, Cognitive Psychology, W.H. Freeman and Co.: New York.

Glass, A.L. and Holyoak, K.J., 1986, Cognition, Random House: New York.