Whenever we get a chance, we love to participate in market research done by other companies. Doing so gives us the opportunity to think about how we’re answering questions about the stuff we care about. Being on the “other side of the glass” is a great reality check.
We recently upgraded some important business software that we use here at Dimensional Research, and had the opportunity to do a follow-up survey on the experience. We quickly discovered that it was a very poorly designed survey. The company wasted their time and energy (and ours too!) on a survey that was so badly designed, it will do nothing to help make things better next time.
Here are the things they did wrong:
1) To get the product installed, you first had to register, get a new license key, download the product, then run the installer. Four distinct steps – and we had problems with the first three! But the survey asked only about the step that actually worked – the last one. There was no opportunity to tell them that we had to register three times, that we ended up phoning somebody to get our license key, and that the first site we were sent to for the download didn’t have the right file.
2) They were clearly using the same exact survey for people who were first-time buyers and for those who were upgrading. Even though we purchased three years ago and our reasons for purchasing aren’t relevant anymore, we were required to answer questions about how we chose this vendor, etc. Even if their lists are so bad that they don’t know we were an upgrade rather than a new purchase, a better designed survey would have given us the opportunity to skip new purchaser questions and continue onto the relevant product-related questions.
3) They did not ask any product-related questions aside from the installation. We’ve been using this product for three years now and they’ve never asked us any questions about the product – only about purchasing and installing the product. It definitely left us with the impression that all they care about is getting our renewal dollars .
4) There were no open ended questions where we could inform them of our issues, so that they could fix them, or to find out if we were an isolated case or if all of their customers have had the same problems.
Today’s fabulous, easy-to-use online survey tools are enabling a lot of bad survey behavior. We trust the readers of this blog will have better judgment!