Dimensional Research

Technology Market Research

Customers Don’t Talk Like Marketers Do

by

One of the important jobs of researchers is to “translate” between the lingo of marketing people and the way that their customers talk.  It’s an obvious thing when you say it, but a good researcher makes sure they are constantly doing a sanity check and purging “lingo” from their conversations.

I was reminded again of this basic concept on a recent project.  We were hired by a client’s Web team to learn about what type of marketing assets were most compelling for a particular kind of audience. It was a great research project – a clearly defined problem, with a clearly defined target audience.

Customers don’t think like marketers. When an IT professional goes out to get information, they don’t really think too hard if it’s a Web page, a PDF,  a video, or a even a call from their vendor’s account manager. They think about getting an answer to their problem. IT professionals don’t think of “consuming assets”, many of them don’t even know what a marketing asset is.  They just think they “get information from their vendor.”

By asking questions that used their own words to talk about real experiences (for example, “The last time you purchased this type of product, did you visit the vendor’s Web site?”) or showing them examples of specific Web sites and seeing what they’d be most likely to do, we talked to them in their own words and got great, usable answers.

The final step was to translate that back into the client’s marketing speak, because of course they needed to know if they should develop a video or a white paper or an interactive assessment or….

Market Research: Living With “Don’t Know”

by

We’ve read with great interest Alastair Gordon’s recent article on research methods measuring emotional response.

Mr. Gordon points out that sometimes, “Don’t know” may be the real answer after all. He adds, “Many people choose brands automatically, as a matter of habit and with little thought. Reasons for attachment can get buried and forgotten. Autopilot purchase behavior prevails in lots of categories.”

Mr. Gordon is referring to consumer products, but his argument applies to technology market research as well. While “autopilot purchases” are less common in Corporate IT than in other areas, they still happen.

We recently completed a study for a client that included talking to recent purchasers of their product. The topic of the study wasn’t competition, but there were some great competitive anecdotes that popped up, so we included them in the report. After all, who doesn’t love hearing about their competitive wins?

There was one particularly interesting scenario where the research participant already owned the competitor’s product and had done a successful implementation of the it. Still, they shelled out what was for them a significant amount of money to purchase our client’s product.

When this came up during the interview, I drilled down on the reasons for not using their existing solution – missing features, total cost of ownership (even though they product was free, maybe there were support or infrastructure costs?), relationship issues with the account manager – but it really boiled down to something else entirely.

There was no “logical” reason for the purchase. It was the power of the brand – although the research participant didn’t realize that and was unable to articulate it. He “didn’t know.”

Of course, this was very good news for my client, they have a great brand that wins them competitive deals. But the participant couldn’t articulate that. It was one of those cases where “Don’t know” was the real answer after all.

5 Questions To Ask Before Engaging A Market Research Firm

by

I went back and forth about writing this post in case it seemed self-serving, but it’s a question that we get a lot, so I wanted to offer some guidelines to consider when choosing a vendor for a market research project.

1) What is the firm’s expertise?
Does the firm you are dealing with know anything about your target market? At Dimensional Research, we work exclusively with technology. Our researchers have spent their entire careers working with the people, processes, and technology used by Corporate IT, and I can say very confidently that we know our stuff.

This matters for two reasons. If it’s a technology-related research project: (1) We’ll get better results than any research firm that doesn’t know technology, because we’ll understand the language being used and the content it’s being used in.  This means we do better follow up and analysis. (2) We won’t consume a lot of your team’s time with “hand-holding” while the researcher comes up to speed. We’ll dive right in and be productive immediately.

2) What is the researcher’s expertise?
Once you have found a firm that is a match, make sure you talk to the person who will be doing the actual research – conducting the interviews, moderating the focus groups, writing and analyzing the Web survey. Is that person an expert also? Not every researcher at a firm knows everything the firm has experience with. Ensure you have the right person or team of people doing the hands-on work or the experience the firm brings will be essentially irrelevant.

3) Does the firm demonstrate expertise in their own marketing?
So you think you’ve found a good market research firm? Now look critically at their marketing. This will depend to some degree on the project of course, so look at the appropriate items:

  • If you have someone doing Web usability research, is their own Web site design usable?
  • If you’re doing a message validation, how good is their messaging? Is it compelling? Did they understand your pain?
  • If they’re doing competitive research, what do they know about their own competition and can they articulate a differentiator?
  • If it’s a customer satisfaction project, how satisfied are their own customers? Do they track customer satisfaction? How?

4) Has the firm/researcher done a similar project before?
If the firm’s answer to this question is “yes,” get a reference call set up, or a very, very good reason why not to set up such a call. Researchers tend to develop very close relationships with their clients – during a project there is a lot of communication – and if you can’t get a reference, that’s a big red flag.

If the answer is “no,” that’s not necessarily a deal breaker – there is a first time for everything – but is your research firm honest about that? Are they willing to be open throughout the process so mistakes are caught early?

I’ll make a confession – the first time Dimensional Research tried using social media to recruit it didn’t go very well. We were still learning the ropes and didn’t understand where our audience was hanging out. But the client knew we were trying something new, and we had an extra two weeks built into the schedule to course correct. Good thing, because we needed it, but the end result was a very successful project.

5) THE MOST IMPORTANT ONE:  How have they evolved their own business based on their research?
I was shocked to meet a fellow researcher recently who essentially hasn’t changed their business in ten years.  If a research firm doesn’t fundamentally understand the value of research enough to use it to make their own business better, DO NOT hire them.  A good researcher always has their research hat on for their own business.  Dimensional Research treats every engagement as ongoing research into our own business to launch new products like our ROI analysis.  We also constantly test our assumptions, as in this example of what we did with stipends.

The other great thing about question #5, is that it can be answered by any firm.  One of the frustrations of market research is that the vast majority of our work is confidential, so it’s impossible to share our biggest successes.  But any firm should be able to articulate how they use research to drive their own business.

Should Market Research Be Fun for Participants?

by

I was reading an interesting market research article by Tim McAtee. This line near the end of the article really jumped out at me: “…this approach can make research more enjoyable for the subject by making survey vehicles less boring.”

This raised a couple of questions for me:

  1. How much fun do our participants have when they are engaged in a project with us?
  2. Does it matter to the outcome of the research?

Fun can certainly add value. Any interviewer knows that getting someone to relax during the interview results in more honest answers and better research findings. We purposely write our interview guides to start off with more general topics so we can get the participant engaged.

Of course, you do have to cover the topic of the research well, but if the participant is qualified to talk about a topic, they probably really enjoy talking about it. And of course a completely boring survey, especially with too many ranking or rating matrix questions, will have a huge drop-off, or if the stipend is really good, they’ll speed through the survey to get the completes without giving thoughtful answers.

On the flip side you can have some bad results in a focus group when there is too much positive energy floating around the room – which is just as bad as an overly negative atmosphere. You can’t have an environment where people are hesitant to share negative viewpoints because they don’t want to bring everyone down.

IMHO there are some definitive statements about fun and market research:

  • Boring is NEVER good for market research.
  • Participants who enjoy the process  give better feedback.
  • Fun cannot come at the expense of research goals.

I have one more theory: If the researcher is engaged and enjoying the study, that helps the outcome too. Here at Dimensional Research, we love what we do! We find the IT professionals we have the pleasure to work with endlessly fascinating. They have a truly difficult job and their creativity, tenacity, and smarts consistently impress us.

Technology Market Research: Best of 2009

by

As New Year’s approaches, anyone surfing the Internet knows that it becomes increasingly hard to avoid “best of” lists. We went back and forth on whether to publish such a list, then decided that since blogs are arranged in a chronological order, highlighting the “recent” rather than the “best,” it’s sometimes difficult for readers to find the best material in a blog. This list should help – it highlights our favorite 2009 market research articles, as well as reader favorites.

Best of 2009: Reader Favorites

  • In-Depth Interviews, Focus Groups, or Both? When doing qualitative research, we need to decide which is right for the client: in-depth interviews, focus groups, or maybe a combination of both. The choice depends entirely on the client’s goals.
  • Top Three Questions about Competitive Research. As technology market research experts, we get a lot of inquiries about competitive research projects. This article includes answers to some of our most frequently asked questions.
  • Customer Satisfaction Surveys: Avoiding “Survivor Bias”. One of the biggest mistakes in doing customer satisfaction surveys is allowing “survivor bias”.  This happens when you surveying your existing “surviving” customers – the ones that stayed with you no matter what – and ignoring the customers and business that you lost.

Best of 2009: Our Favorites

We know that these articles are relevant, because we keep sending them to clients! Among other issues, these articles answer the important question of “what is technology market research?” and explain how to measure market research ROI.

  • Technology Expert Defined. Technology market experts are experts on Corporate IT and how it works. They are familiar with Corporate IT and its processes and can talk comfortably and knowledgeably about technology, processes and people in the IT industry.
  • Message Validation: Market Research with Clear ROI. Hard ROI is easy to demonstrate when your marketing programs get two times, ten times or even greater response from the same spend, because your message is more compelling.
  • Involve Your Customers In Market Research. Customers love being involved in market research. They don’t see it as a chore or just another thing on the “to do” list. On the contrary, they love being involved and appreciate the opportunity to voice their opinions.

These are our best, and most useful, technology market research articles for 2009. There are many more articles on this blog (over 50 articles in fact), so do feel free to browse. Some of our favorite categories are Quantitative or Qualitative and Research Questions.

Happy New Year!

Technology Market Research: Meet the Real World of Corporate IT

by

One of the realities of doing technology market research is that you end up dealing with people in the real world. For those who work in roles that deal only with the hottest new innovations, it can be a bit of a shock to shift gears from the cutting-edge of hype and the  super-early-adopters that use new technology.

There is definitely good news. Your technology market research project will give you a good dose of reality and a much better understanding of the market that you’re actually marketing and selling into. And any good market research firm will help you to find exactly the group you need to hear from: whether a cross-section of the entire market, a group of early adopters, or conservative corporate IT executives.

However, bear in mind that the market research project may not feel like the rest of your life. Everyone you talk to on a daily basis may know about your technology and your space, but that doesn’t mean everyone in the world does.

A few important things to remember:

a. “Buzz” usually isn’t happening with the entire market. It may feel like everybody is talking about cloud computing these days, but in reality they aren’t. There are plenty of smart, informed people that simply haven’t got cloud computing on their radar because they are focused on other things.

b. Your competitors are not “everywhere.” We know it feels like that to you, but in reality, only a small percentage of the market uses your competitor’s tools.

c. Even your own customers aren’t as educated about your product as you are. Don’t expect to have the same deep conversation with them that you have in your internal meetings. Remember:  you spend 120% of your time thinking about your product. Your customers probably spend only a fraction of their time doing the same.

d. In the real world, corporate IT doesn’t get as excited about change as technology startups do. It may feel like a wet blanket to hear corporate IT research participants finding the negative aspect in the amazing new technology that you know is going to change the world.  But the reality is that it’s much better to hear the objections, so you can deal with them.

You should work with your research provider to make sure that you understand exactly who you want to talk to, and it helps to be realistic about the level of effort it takes to find exactly the right people and engage them in a beneficial conversation.

Defining Terms in Market Research and Beyond

by

The other day I was talking to a client about cloud computing, and he was frustrated. He finally said, “Does ‘cloud’ stand for something in the technology, or does it stand for the lack of clarity around the concept?”

His response made me laugh, but it also reminded me about how important it is to be very clear about terms when you want people to understand you.

When I was on my high school debate team, and again later on when I studied mathematics, you always had to start by defining your terms. It was a given that you couldn’t have a discussion or prove anything unless you knew what you were talking about in the beginning.

Talking with corporate IT you have to do the same thing – especially with new technology and acronyms! When you’re talking about PAAS, SAAS,  cloud, application virtualization , ITSM, or any other jargon, be kind and give the audience a clue about what exactly you’re talking about. Never assume they already know.

And when doing technology market research, don’t guess!!! Either tell the participants exactly what you mean, or ask questions about what the participants think it means. Put in the effort to make sure you understand each other.

Technology Market Research Should Be Done By Technology Experts

by

I was really surprised to be pointed to this article from Zoomerang (our main online survey tool provider, who I think makes a great survey product). The article states that more than 40% of online IT panels contain survey takers who are not really in IT.

Perhaps I shouldn’t be surprised.

It’s challenging to find appropriate IT participants for research – online or in person.  It requires a very specific knowledge of the people, processes, and technology in IT.  As this article clearly points out, generic market research providers don’t do well here.  I do genuinely think they have TRIED to “fix” this, but I know it’s hard and don’t believe that they really have.

The challenge is that IT professionals are compensated well  for participating in technology market research, so the motivation to misrepresent skills is high.  A better way ensure that you’re talking to real decision makers in corporate IT is to work with a provider who specializes in technology and has years of experience building up the resources to find these people, and has the technical savvy to know immediately if something has gone wrong with the recruit.

Dimensional Research specializes in technology market research, and work extensively with companies that sell to corporate IT.  That’s all we do. And because of that focus, we do it well.  We know the reliable sources to find participants that match your needs.  Just as importantly, we understand technology ourselves,  so we can quickly catch someone who is BS-ing.

Market Research: Quantitative or Qualitative?

by

Scott Anthony recently advised, “In Market Research, Use Numbers with Caution.” He added, “Companies too frequently default to quantitative research because they think there is safety in numbers. It’s a lot easier to justify a strategy by saying, ‘The data suggests’ than by saying, ‘My intuition suggests.’ But sometimes numbers provide false confidence and obscure real opportunity.”

Anthony’s point of view is quite different than the point of view presented by Robb Mandelbaum, who recently said in Inc., “Given limited resources … it generally makes sense to go quantitative.”

Both articles are excellent  – and both present very different points of view.

So how do you choose between quantitative (surveys) and qualitative (focus groups, in-depth interviews) when performing market research projects with technology participants?

Sometimes, quantitative (surveys) research is better

1. When you need data to support a claim with investors, press, or internal stakeholders.  Dimensional Research has done a number of Web surveys that our customers have used for PR purposes, including these recent ones on anti-virus and desktop power management.

2. For trending purposes, quantitative studies are also the best.  Dimensional Research has a number of clients that follow the “Would you recommend this company to a friend?”  question promoted by HBR, and watch the responses to that trend over time.  It’s a great way to track trends in customer feedback.

3. Of course, if time is the greatest consideration and you need some kind of quick feedback – Web surveys have a big advantage.  The are FAST.

Sometimes,  qualitative (focus groups/in-depth interviews) research is better

Numbers can be deceiving and there is no better way to find that out than to talk live to people who give you numbers.  I was recently doing a competitive study, speaking to end users of a client’s competitor’s product.  My client, as is natural, was most interested in the negative feedback about the product.

So to introduce the topic of what was good and bad in the product, we started by asking the customers to rate the product being discussed on a scale of 1-5.  And of course, asked the important market research follow up question, “why?”

I was consistently surprised by users who had raved about the product, saying it was perfect and there was nothing they would change about it given the chance – and they’d assign a 3 out of 5.  And other users who basically spent the whole conversation whining and complaining about the product would give it a 5!

Numeric scales can be misleading and in these cases, qualitative studies such as focus groups or interviews are better.  I would always recommend qualitative studies when you are looking for thoughtful answers including:

–          Messaging validation for products that are new to the market

–          Market validation

–          Understanding objections and barriers

–          Product feedback for enterprise products (web surveys do a better job with consumer products that are simpler to understand)

Qualitative vs. quantitative is a good conversation to have with your market research provider – although of course you should understand their expertise. If you only have a hammer every problem is a nail, so expect a quantitative-focused research house to tell you to survey thousands of prospects, and someone who only does focus groups to emphasize qualitative approaches.

A good research firm will know if they don’t have a fit with your needs and will point you in a better direction.  For example, Dimensional Research does not do conjoint analysis, but we have a great partner that we can refer you to if that’s what you need.

Getting the Feedback that Matters Most

by

This blog post can be summed into one sentence: “I talked to some people I know” is NOT market research!

Of course you should talk to people you know about your ideas, but you need to ask yourself, who is your target market?

Will your drinking buddies ever buy your enterprise software?  Maybe they will, and count yourself fortunate if you play poker with only CIOs. But you probably interact with a lot of other people who are not actually in your target market.

It’s a good idea to get ideas from everyone, but you should put significantly more emphasis on feedback from the people who are actually part of the community you target.

It may be obvious that your high school buddy who runs his family’s (very successfull) car dealership doesn’t know enough about technology to give you feedback.

The really problematic conversations are usually the ones with people who are in the periphery of your target, just not IN it – people who sell to IT in other companies (especially ones with big established brands!), VCs who invest in tech companies, journalists and bloggers – even your fellow co-workers.

All of these people will have insights for you, and you should certainly pay attention to them, but you should never use the info you get from these people INSTEAD of having conversations with your actual target market, the people who will eventually buy your solution. They are the ones you really need to talk to.

If your day-to-day routine does not easily facilitate those conversations – make a point to make it happen.  Find those people and talk to them in a way that gets unbiased feedback. Any market research firm would be happy to help.