It’s been almost three years since my first qualitative HCI study that involved interviewing. I’ve since used semi-structured interviewing in two more studies, so I’ve still got lots of room to grow, but I want to share some things I’ve learned about how, and why, to use semi-structured interviewing in HCI research.

• Why Interview?
• The “Trick:” Be Specific!
1. What, When, Where, Who vs. How, Why
2. Last Instance vs. Typical Instance
3. Criticize vs. Compare and Contrast
• Tools and Tips for Collecting and Analysing Interview Data
• Recommended Resources

Why interview?

So what is semi-structured interviewing, anyway? Unlike with structured interviewing, the researcher has freedom to ask questions not in the protocol (a list of questions to ask during an interview) in order to get richer data and get more detailed responses. Unlike with unstructured interviewing, there is a protocol and a specific goal of the interview.

In HCI, the method can be employed to understand the values and priorities of potential users when it comes to systems; for example, we might want to know how important privacy is relative to the ability to share content online. Interviews can also be useful to HCI researchers who want to understand what kind of system(s) can be built to support a particular task in a particular situation; this involves talking to people to find out about the processes and relationships relevant to that space. For example, when observing and interviewing students in foreign-language classrooms, my team ultimately wanted to figure out what kinds of technologies can support the kinds of communication that happened in that space.

The “Trick:” Be Specific!

(1) What, When, Where, Who vs. How, Why

One of the most subtle things about interviewing is that “what” “when” “who” and “where” questions are easier to ask without confusing or misleading the interviewee*, but when exploring a research topic it’s the “how” and “why” questions we care about. So the benefit of semi-structured interviewing is the capacity to break down a nebulous “how” question into bite-sized “what” chunks. For example, if I really was to ask, “how do you go about looking up health information online?” I could instead start with “what websites do you use to look up information?” “can you recall when you found them?” “who told you about them? Have you recommended any of these sites to others? Who do you discuss what you find with?” The latter questions get at a lot of detail, whereas the first one is so vague that most people wouldn’t really know what to answer, and could describe using a search engine rather than getting into the details of social information search or browsing that they may partake in.

(2) Last Instance vs. Typical Instance

Another big thing is going after specifics, which is related to the above idea of making questions as concrete as possible. For example, “describe a typical in-class activity involving translation” is not going to get a response as detailed and unfiltered as “can you tell me about the last time you had an in-class activity involving translation?” Even if what you mean is to ask the first question, asking the second will get you an answer and then some. I asked this type of question of people across two studies, and on multiple occasions responses included comments like, “but that was a really unusual day,” which you can use to talk about what made it unusual. This technique gets the participant thinking about the concrete, rather than the abstract, which can help them to just talk freely rather than try to package things up concisely. Although being concise sounds nice, it’s generally not a good thing, because depth and richness are key advantages of interviews (as opposed to, say, open-ended surveys or structured interviews).

(3) Compare and Contrast vs. Criticize

The last concept I want to share is about speculative questions. As HCI researchers, there’s often a need to make a sketch, approach a person, and ask, “will you use this?” And the answer to this is almost meaningless. Think of all those abandoned New Years resolutions – people are awful at having any idea what they will or will not do. But sometimes you still want to ask the question, at which point it might be useful to present alternatives. Asking “which would you use, A or B?” and then some questions that try to get at the “why” can yield a far more detailed look at the priorities and values of the participant than just asking “will you use this thing I made?” Also keep in mind that if you’re friendly and nice, most participants in your preliminary studies will want to avoid hurting your feelings. Asking to compare and contrast can help people be honest, rather that placate you by avoiding discussion of (even glaring) shortcomings of the systems or prototypes.

Tools and Tips for Collecting and Analysing Interview Data

Generally, interviews can be audio-recorded. If they’re not, try to see if someone other than the interviewer can take notes, so that the interviewer isn’t scribbling frantically the whole time and actually being personable. Even if they are recorded, I strongly suggest jotting things down on a printout of the protocol. You’ll need to refer to it anyway, and jotting things down lets you come back to topics later on.  One caveat: don’t gravitate too much between not jotting anything down and looking excited and scribbling lots of stuff. That makes it seem like only some of what the participant is saying is useful, and they’ll try to filter what they say more to make you happy (people agree to be in studies because they want to help you!). The problem with that is, for one, your participants probably won’t be able to guess what the goals are very well, and, two, you don’t really know what’s important, either. One of the main virtues of qualitative work is the capacity to discover phenomena you didn’t even know to look for, so casting the net wide in the interview stage might be a key aspect of the methodology.

On a less philosophical note, some tools I’ve found invaluable for audio recording, and then transcribing, interviews:

• Voice Recorder! In a voice recorder, you’re looking for efficient battery use and ease of transfer. This recorder, for example, just plugs into a computer’s USB port. I use the Olympus one in the link, as do a handful of my colleagues.
• If you’re going to pay someone else to transcribe: Production Transcripts, which runs about 2 dollars per minute (varies on sound quality). I used this service, at a colleague’s recommendation, and they were very responsive and nice about rush orders and the rare events of errors (file not uploaded correctly).
• If you’re going to transcribe things yourself: Express Scribe is a tool that is fabulous, and it is free. Very customizable, and lets you change the speed of playback arbitrarily without too much loss to the quality of sound. Also has nice way to program hotkeys on your keyboard when foot pedals are unavailable and you want to type in another word processor. When it’s minimized, the taskbar icon (on a pc) shows the current hour:minute:second.ms mark. Simple and kind of obvious, but really handy and unique.

Once you have that data, how to analyze it? As someone partial to qualitative research, my approach has been to  iteratively code the data and emerge with the theory of values/processes/behaviors that is grounded in the data. Most of what I do on this end has been heavily influenced by Constructing Grounded Theory by Kathy Charmaz, which I have posted about on this blog. Another useful book is Writing Ethnographic Fieldnotes by Emerson, Fretz, and Shaw, which contains some very helpful chapters on analysis.

Much of the time, analysis involves coding, for which there is lots of software. I really like Atlas.ti, which is not free but far cheaper than similar options and far better than cheaper options. There is also the free online cloud-based Saturate App, which seems like a very promising tool, if lightweight. Be aware, however, of privacy concerns and the possibility that putting your data in the cloud may not be within the scope of what is approved.

Recommended Resources

Ultimately, the hows and whys of this research method are far more complicated and far more nuanced than I could summarize in a blog post. I didn’t just stumble upon everything above, I read some excellent materials on how to interview that I highly recommend:

• Learning from Strangers: The Art and Method of Qualitative Interview Studies by Robert S. Weiss
• Interviewing as Qualitative Research by Irving Seidman
• Human-Centered Design Toolkit by IDEO (full book available in PDF for free)

Footnotes

* In a lot of literature about interviewing, and qualitative research in general, there is at least a small note about what to call the people whom you interview. In HCI, “participant” and “interviewee” come up pretty often, and sometimes “informant.” I rarely see “subject,” and I usually cringe when I do, because in HCI the most common “subjects” of the research are systems/designs/ideas/etc, not the people whom we ask about those things.