There are many ways to make sure your online survey is efficient and effective. One of them, of course, is to avoid asking bad questions.
Tip 1: Craft your questions carefully to avoid unwanted results
This tip was inspire d by Seth Godin. He explains that “Every question you ask changes the way your users think. If you ask, ‘which did you hate more…’ then you’ve planted a seed.”
Mr. Godin makes a great point. I recently booked a trip with Travelocity. It was a great trip. I was happy. I did have a small issue with the airport transfer getting home, so I filed a complaint to see if I could get a refund. It was a small matter – about $30 credit – so I wasn’t too worried about it. Travelocity’s reply email was pretty typical, asking me for more information (which I had to get by opening up the email THEY sent me, so a bit annoying).
BUT… Then they sent me a survey asking about my experience with Travelocity. One of the questions on the survey was “Do you know about Travelocity’s guarantee that your booking will be right, or we’ll work with our partners to make it right, right away?” I actually didn’t know, but this question clearly did NOT describe the experience I just had.
Travelocity did make it right – it took them about 30 days to do so – but that wasn’t what their guarantee said and their survey pointed that out to me. By including that question in the survey, they planted a seed that they didn’t want to plant and I ended up being less happy overall than I was before the survey.
Tip 2: Ask at least one question that participants actually WANT to answer
It’s important to ask the questions your customers are actually interested in answering. Too often, marketing departments are so focused on the company’s newest offerings that they ignore the products their customers have come to depend on.
I use QuickBooks for my business. I have LOTS of feedback for Intuit on the core QuickBooks product, but they never ask me about that. They constantly survey me, asking if I want to buy checks or do payroll or take credit cards, but there is no “Thanks for your time, is there anything else you’d like to tell us?” that would enable me to give them the feedback I WANT to give them.
Another example from my own professional life: I use Zoomerang for my surveys. It is a GREAT product – with a few caveats. One of the problems I have with the product is that they have a horrible interface for “choose the answer that most closely applies.” It’s a small button that’s almost impossible to see. On the other hand, their interface for “choose all that apply” is great – a nice square with a big check mark. I want to tell them about this issue, but they keep sending me web surveys about other things and I’ve never had a chance to give them important feedback that I really want to give them. Maybe they’ll read my blog and I’ll get to them that way? <CORRECTION: They did in fact just send me a survey this week that allowed me to give that feedback. I’ll wait and see if they act on the feedback.>
Tip 3: Reward your participants, wisely
This tip is inspired by Patricio Robles: “offer users who respond to a survey a discount, an entry in a drawing for a prize, something of value. It will boost response rates and make them feel like they’re investing their time wisely.”
While I generally agree, I would add that when offering a reward to participants, it’s important to consider your target market. A gift that is too nice motivates people who aren’t qualified to complete the survey. Then you have to wade through junk or set up lots of qualifying questions to weed them out. If the reward is nice enough, some people will game the system and try to guess what you’re looking for, so the filtering questions won’t always work.
When you’re doing a technology web survey, this is especially important, since people who don’t know your topic can really throw off the results, as they are not informed. If possible, giving a copy of the final report on a topic is a great incentive, since only people who know and care about the topic will respond to this type of reward, keeping your input very clean.