We hope that we’ve made it clear in this blog that Dimensional Research is a big fan of Web surveys. Web surveys are a great market research tool. They make it easy to get immediate results right at your fingertips.
However, you need to have a certain level of knowledge before you can run an effective Web survey. Here are three signs that might indicate you are not ready for a Web survey, and you should do some qualitative research first.
- If you write multi-choice questions and are completely guessing the answers, you’re probably not ready for a Web survey. Of course, you always need to have an “Other” section to cover the corner-cases that you just didn’t think of, but your options must capture most of the likely responses to effectively quantify a finding. Let’s admit it, survey takers will often pick a a presented option that isn’t quite right rather than take the time to fill in an open-ended option, so you can’t rely on “other” to cover your lack of knowledge.
- If you’re asking a lot of open-ended questions, you’re probably not ready for a Web survey. But what is “a lot?” Good rule of thumb, no more than one open-ended question for every 20 survey questions – not including that important last question “Is there anything else you’d like to tell us?” that you put at the end of every survey.
- If you get only one shot at an audience, don’t waste it with an uninformed Web survey. If you don’t completely understand a new market, and have a participant list that you can use only once, mitigate the risk that you’ve gotten something wrong by doing a few interviews with a couple of list members, or doing some small trial surveys with more open-ended questions that you can use to form a better Web survey.
Don’t waste a Web survey. If participants are taking the time to give you feedback, make sure you’re getting the most from them. Don’t be scared to add some test surveys or interviews to a project schedule – the extra week or two needed in the schedule will give dramatically higher results.
We just wrapped up a great project that started as a stand-alone Web survey. We needed to quantify some specific purchasing metrics, so the survey was clearly the right methodology. But as we drilled deeper, we realized that we were making far too many guesses about actual purchase motivations.
We added a series of 15-minute customer interviews prior to the Web survey that gave us some great insights into the scenarios that were driving behavior, and then developed a much better Web survey that gave crystal clear data about all scenarios. Most importantly, it allowed us to eliminate a group of customers that weren’t motivated by the conditions we were evaluating (although their behavior was similar) and would have skewed our data significantly if we hadn’t excluded their responses for certain questions.
Going into the final presentation, we were very glad that we’d done the interviews. We knew our stuff cold and had the backup we needed to defend the results and ensure they were taking seriously – a must to influence the business outcome.