I admit it.  Although web surveys are one of our most popular research offerings, I strongly prefer qualitative research (focus groups, in-depth interviews).

I can’t help looking at the findings in web surveys and feel like I want to dig in and ask more questions.  The participants say things that don’t make sense to me and I want to know why.  Or they make short comments responding to open-ended questions that leave me with a dozen follow-on questions I don’t have the opportunity to ask.

When I present qualitative findings, I know I can answer any question that comes up with complete confidence.  With quantitative findings, I always know that there will be questions where the answer is, “We can’t draw any conclusions based on this survey.”

This came up again last earlier this month:  We just did a series of in-depth interviews where we wanted to understand perceptions about the cost of various alternatives.  All the participants in this study had identified themselves as product decision makers who had full visibility into costs – a requirement for the study.   If you had just looked at the first level of answers people gave, you would have thought that one of the tools we were looking at was very inexpensive compared to other options.

But because this was an in-depth interview, I got to ask that all-important “why” question. I quickly realized that while all the participants had been educated on the “line 3” costs that were billed directly to their organization, not everyone was aware of the additional “line 10” costs that had to be added to support this different approach.  When you added both of those up, the tool that originally appeared less expensive turned out to have a similar TCO to other options.

Now, it’s true that we could have found this out by writing a good web survey, but one of the secrets to writing great web surveys is to know the answers to all the questions first.  We continue to recommend web surveys as good vehicles for quantifying concepts that you know well, but want to put an accurate % by each of the options.  This is a valuable thing to do, especially for market sizing, external marketing and PR purpose.

But for finding out the answers that you don’t really know, start with qualitative research – and by all means do a web survey next to put those %s in place once you know the statements to put the %s with.