Dimensional Research

Technology Market Research

Phone Surveys or In-Depth Interviews: What’s the Difference?

by

A while back we blogged about phone surveys, and argued that they do not have a place in technology market research. Several comments were made both in the blog comments and to us directly that made us realize that there is confusion about the difference between phone surveys and in-depth interviews (also called IDIs). Here’s how we view this:

Phone surveys and in-depth interviews are similar because:

  • Both are conducted over the phone.

But that’s pretty much the only similarity.

The differences between phone surveys and in-depth interviews:

Phone surveys: A quantitative research method that includes a large numbers of participants.

  • Capture input to a common set of questions with pre-set answer options
  • Administered by a phone survey professional that has a pleasant voice and is trained not to guide the responses of the participants in any way
  • Are usually short: 5-20 minutes as rule of thumb
  • Usually no incentive is given to participants, although there may be a small amount given.

In-Depth-Interviews (IDIs):A qualitative research method that uses a smaller number of highly select participants.

  • Screener is written to allow significant discovery, open-ended questions, and drill-down
  • Administered by a trained moderator who is versed in the client’s business, the goals of the research, the topic of study (in our case technology) as well as techniques for putting the client at ease and getting the kind of feedback desired
  • Usually longer: 30 minutes to 1 hour is common
  • Typically generous incentives are given to participants to compensate them for the time commitment

An easy way to think of the differences are that phone surveys are pretty much like web surveys, but conducted over the phone. Given these definitions, we stick by our earlier recommendation that phone surveys have no place in technology market research. They are significantly more expensive to conduct than web surveys without adding value in a corporate IT study (although agree that there are audiences phone surveys are appropriate for).

That said, there is nothing inherently wrong with phone-based research. In fact, in-depth telephone interviews have become one of our most valuable research methodologies, particularly as clients become more global and travel costs are becoming more of a factor in evaluating research budgets.

Web Surveys vs. Phone Surveys

by

One approach to market research that we haven’t talked much about in this blog is phone surveys.

Phone surveys are when a person with a nice voice calls people and asks questions from a script. Answers are recorded in a spreadsheet, and the final result is very similar to the graphs you’d expect out of a Web survey.

Phone surveys are different from in-depth interviews because the script is asked exactly as written, unlike an in-depth interview where you have a researcher who is a technology expert asking the follow-on questions needed to drill down into answers.

Dimensional Research usually doesn’t recommend phone surveys to our technology clients. Even for consumer marketing, phone surveys are becoming less useful according to Jay Leve of SurveyUSA: “There is [no] future for any form of telephone research that is predicated on the researcher being able to barge in at will and seize the respondent.”

Several more compelling reasons why phone surveys don’t work are outlined very nicely by Jeffrey Henning at Research Live. The ones that relate to market research for technology companies include:

– Expense of dialing. More and more phone surveys are done via cell phones, since more and more people (and this includes many technology startups) don’t use landlines. In the US, by law, you can only use automatic dialing for landlines, not for cellphones. Manual dialing is much more labor-intensive – and expensive.

– Online surveys eliminate the expense of data entry. The respondent to a web survey is, in effect, donating the data entry cost, as they select the appropriate choices and type in their answers. With a phone survey, you are paying a call center representative to transcribe each respondent’s replies.

– The visual medium of Web surveys lets you easily show people visual concepts, such as ads or core messages, and get their response. Surveys that require respondents to react to visual concepts can’t be conducted with phone surveys alone.

– Web surveys have the allure of confidentiality. People today feel more comfortable sharing information on the Web than answering the prying questions of a phone interviewer.

– People prefer Web surveys, because a Web survey can be done at the respondent’s convenience, rather than at the moment the phone interviewer happened to call.

Jeffrey does make a case for phone surveys in certain situations, but almost none of these are relevant to IT market research including:

  • Major Account Research – In these cases we recommend in-depth interviews or online customer advisory boards. Why wouldn’t you take the opportunity to have a deep conversation with your biggest customers?
  • The Human Touch – This argument only works with Corporate IT if you are also knowledgeable about technology, so again, it’s better to do in-depth interviews.
  • Some People Aren’t Online – This is obviously not an issue for technology professionals.

Our recommendation, after years of doing market research with Corporate IT, is to avoid phone surveys when doing technology market research. Instead, use Web surveys, in-depth interviews, or a combination of both.