Dimensional Research

Technology Market Research

Research Bias – How Different Participants Yield Different Insights

by

According to a recent survey from a specialist PlayStation 3 site, geeks make the best lovers.

Let me clarify right from the start that as someone with distinct geek-ish tendencies, who is married to a confirmed geek, I have no intentions of arguing with the study’s conclusions – just with its methodology.

The problem with the survey is that it was biased. Given the audience likely to peruse a PS3-specific site, it’s fair to assume the respondents skewed more toward the “geek” end of the spectrum.

When conducting research you must know who you’re talking to in order to interpret the conclusions correctly. If they had done the same survey with a running site, would they have found out that marathoners are the best lovers?  

In technology market research, identifying your research target audience is especially important.  Customers will give you different insights than prospects, employees, partners, or even the customers of direct competitors! 

If you’re talking to your customers, you need to acknowledge that by definition they have the pain that you’re selling to, or they wouldn’t be using your product. So, you shouldn’t do a customer survey and then announce, “100% of the MARKET does this.”  Of course, it’s completely fair to say, “100% of our CUSTOMERS do this.”

In comparison, if you’re talking to the general market who isn’t as fully educated on your solution, you’ll need to tailor your questions accordingly. If you ask questions that are too detailed and specific to your solution, you’ll get uninformed answers. The type of questions that are appropriate for customers who are familiar with your product, are not the same ones to use for the general market.

For example, if you’re asking about product features, you will get vastly different answers from existing customers who know your product, and from your competitor’s customers who know an alternative approach. Both of these will be different from the answers you’ll get from prospects who don’t have any product experiences.

As with everything in market research, it comes back to goals. Your very first step should be figuring out your business goals. Then, figure out what you need to know in order to achieve these goals. Next, figure out who has that knowledge, and who doesn’t. Only once you’ve answered all these questions, you can dive into the details of the market research project.

How Much Time Does A Market Research Project Take?

by

So, to be honest- it really depends.  But I know that’s not an answer to a very pragmatic question, so let me give more details here, bearing in mind that it really does depend on your specific project.

Here are suggested minimum schedules assuming stakeholders respond in a timely manner.  However, these timelines are never compressed – only made longer when you hit unexpected roadbumps.

A typical market research project takes six-seven weeks. This is pretty standard for a small project, say 10-30 in-depth interviews or 4-8 focus groups.  Larger projects take longer, of course.  And we’ll talk about Web surveys later.

Here’s a pretty typical schedule for a single stage project with one series of focus groups or in-depth interviews – excluding unforeseen “bumps” or unique requirements:

Week 1: Project go-ahead. Write and approve recruiting guide. Identify source for recruiting participants – internal or external.

Week 2-3: Recruit participants. Write and approve interview guide or moderator’s guide.

Week 4-5: Conduct research. This may take less than two weeks, depending on your goals. Four focus groups all in the US can be done in just one week if everyone is available.

 Week 6-7: Write and present market research report.

WARNING: The biggest schedule slippage that happens, aside from getting all the approvals in place for the project go-ahead of course, is with identifying internal participant contacts.  Using in-house lists or asking account managers to give us their contacts for recruiting may be time consuming and the schedule should be adjusted as needed.

Web surveys are usually faster. A typical schedule for a straightforward web survey project is four weeks. For example a survey with only a few demographic cuts (role and geography), 20 questions or less, 1 matrix question and no open-ended questions could have a schedule that looks like this: 

Week 1: Project go-ahead. Determine goals. Identify lists for participation.

Week 2: Write and approve questions. Upload them to survey tool. QA. Run test survey.

Week 3: Adjust survey as needed based on test.  Field survey.

Week 4: Close survey. Do analysis including filtering and correlating findings. Write market research report.

If the survey is being conducted as a collateral piece to support outbound marketing efforts such as PR or lead gen, add another week for copy-editing and layout.

Market Research: Getting to “No”

by

One of the most important skills in moderating any market research discussion is to make sure you’re not “guiding the witness” to tell you something that is not accurate.

I’ve discussed here the importance of asking market research participants open-ended questions such as “can you tell me more about that?” and “is there anything else?”  I also written about the importance of getting honest feedback, and suggested ways to make sure you do.

Today I’d like to emphasize the importance of giving the participants permission to NOT know.

Market research participants are smart, competent people who know what is going on in their environments. It’s natural that they’re only human and don’t want to come across as unknowledgeable. Nobody wants to admit, “I don’t know.” 

However, YOU need to know the truth. If they don’t know about a product, you need to know about that and adjust your marketing efforts accordingly. It’s just as important for you to uncover that participants are unfamiliar with a product or an idea as it is to know their opinion about it.

The key is to ask questions that allow a participant to tell you the truth without feeling like an idiot.

A question that is in many research guides is an awareness question. “Are you familiar with product X?  What do you know about it?”  I don’t like asking the question this way. It almost guarantees that some selection of the participants will answer, “Yes, I’ve heard of it. I read about it in a magazine” just so they don’t have to admit they have never heard about the product.

Instead, ask “Are you familiar with product X – and please feel free to say no, that’s just as important for me to know.” This will give the participant permission to not know without feeling stupid. 

Another option is to ask, “What do you know about the following product – if anything?” that also leaves the door open for participants to admit they don’t know without worrying about disappointing you.

Online Focus Groups: How Do They Work?

by

Online focus groups are a highly effective market research tool.  My clients love them.  They save the time and cost of travel while making it easier for participants and observers.

Most importantly, these benefits don’t require you to tradeoff for lower quality research. Online focus groups still facilitate the “brainstorming” dynamic that you get at an in-person focus group, where one participant’s comment sparks a comment by another participant and so on. The brainstorming dynamic enables the group to dig deep into an issue, so that the client can get a very clear picture of market acceptance of their product and of any issues they need to deal with.

Participants love online focus groups too, because they give them an opportunity to listen to the feedback of others in the industry. This is an important benefit for participants, especially in customer advisory boards, which encourages them to join.

Online focus groups are conducted using conference calls and Web-based meeting technology such as WebEx, GotoMeeting, etc.  In many ways they’re similar to traditional face-to-face focus groups, except you’re not in one room. We recruit participants in the same way, set a time for the meeting, and prepare a moderator’s guide. If relevant, we can review a presentation with the participants. Just as in a traditional focus group, where the client can listen in through a one-way window, the client can listen to an online focus group on a muted phone line. And we can use Web technology to poll participants similarly to a visual “show of hands” or “nod of heads” in an in-person group.

Despite the many similarities between in-person focus groups and online focus groups, there are also a few differences. The most obvious one is that you don’t get the non-verbal clues from the participants. Because of this, the moderator needs to make an extra effort to draw out all the participants and balance the input. To facilitate this, we typically have a smaller group (4-6 participants per group rather than 6-8 for an in-person group) for the same amount of time. We may do more groups as a result – 5 instead of 4 for example. 

Of course, there is also the option of conducting video online focus groups.  As technology evolves, these are becoming more common and are easier to set up.

Another major benefit of online focus groups is that they reflect the majority of real-life interactions with customers. When selling to Corporate IT buyers, more and more meetings are being conducted online rather than in person, so online focus groups reflect that same scenario.  If your messages work in an online focus group, they will translate well into an online sales call or a Webinar.

Technology Professional? Want your opinions to be heard?

by

Are you a technology professional? Do you like sharing your opinions? You might enjoy being part of a Dimensional Research Project.

The only real qualification is that you work with technology in your profession – we’re especially interested in people who work in corporate IT.  We welcome participation from around the globe!

Important information for participants:

  • This is market research – no one will every try to sell you anything.
  • Your input will be anonymous. Nothing you say will ever be attributed to you or your company in any way.

What do you have to do?

Register here (you may need to scroll down a bit). We’ll ask you a few questions about yourself. Then, as research projects come up that you seem to fit, we’ll contact you with some more detailed questions and information about the specific project.

What is involved?

The actual level of participation depends on the project. For our Web surveys, we’ll send you a link and you can go online to complete the survey. For in-depth interviews we’ll contact you and schedule a time to talk – usually about an hour. For focus groups we’ll let you know when (and where, if necessary) and see if you’re available – usually about 1½ – 2 hours.

What’s in it for you?

This depends on the project. For a Web survey, it may be a copy of the final report or a chance to win a cool prize. For a focus group or interview usually it is a cash stipend payable via Paypal or US dollar check. And of course, with all projects you have the knowledge that you’re influencing the future of technology in some way!

REGISTER HERE>>

Using Market Research To Uncover False Beliefs

by

We just completed an extensive series of in-depth interviews with great IT participants from around the globe. These participants were smart and articulate. They knew their job, and were very good at it.

But as smart and capable as these participants were, we discovered during the interviews that many of them – at least half –  held a completely false belief about my client’s product.

During the interviews with these participants, they kept repeating the same idea.  The details are confidential to that client, of course, but the important point is that this audience really believed that their perception of the products was true. This wasn’t a minor technical detail – it was a major assumption about environments the products operated in.  But the problem was IT WASN’T TRUE! This was verified by my client, by independent evaluations that we found while investigating the conflict, and by some of the other participants in the study.

So here you have a very well established fact about a product, but at least half of the customer population believes something else. In other words: even if something is not logical, and even if your customers and prospects are technology vendors who tend to be logical, they may still believe in something wrong. And it could negatively affect their buying decisions.

This type of dynamic is extremely important for vendors to understand. If there are things that your audience believes about your solution that are not true, you’d better know about it so you can take action.

You’ll notice that I’m referring to this as a “belief”. Beliefs are harder to respond to than competitive FUD or outright lies.  These are things that your target market thinks are true because they’ve never really thought about it.

A formal market research methodology is the best way to uncover and to root out these false beliefs across your target market. You can’t rely on day-to-day conversations with customers, because if these issues do surface during such a conversation, you tend to correct them immediately with that specific customer and then forget about them since it didn’t make any sense to you to think that way.

The Importance of Segmentation in Market Research

by

In my last post, I talked about the similarities across Corporate IT groups when it comes to pain and product benefits.  This week I’d like to talk about key differences that are important to consider when doing research.

This is where segmentation comes into play – making sure you have enough participants of a particular type to represent the differences in your audience. How you segment, and whether you segment at all, depends on your goals. 

For example, if you’re doing a message validation where the goal is to evaluate if you are communicating well about a specific pain or the benefits of your technology, there are a lot of similarities between different geographies and verticals. In this scenario, you don’t need to have a dozen participants from each of the 10 vertical industries you sell into to quickly get a very real sense of how your messages are resonating with a corporate IT audience.

However, in some scenarios, you will need to get representation from different areas. Of course every project is different, and we at Dimensional Research work with our clients to really understand specific goals and make sure we recruit the right representation from any segment to reach those goals. Here are a few general areas to consider. 

The “Silos” of IT

Application Development, Production Operations, Security, …  They all act very differently from each other. They have different cultures:  development likes new features, while “change” often equates to downtime for production operations, and security would ideally prefer that nobody gets access to anything.  They purchase differently.  And (let’s be honest) these frequently have challenges communicating with each other.  If you have a product that crosses IT silos, you need representation from each silo to understand purchasing behaviors, objections, etc. 

Technical Buyer vs. Economic Buyer

The technical buyer will discuss features and functions.  This is a comfort area for many technology companies and they’re more likely to “get” you and your product.  However, the economic buyer is vital to tell you the non-product reasons why people buy – brand, sales execution, channel, financing, etc.  You will need both types of participants to understand competitive dynamics and purchasing behaviors.

Verticals

Government, Education and Non-profits can purchase very differently than IT in other vertical industries.  If you are targeting those verticals, you need to talk to them specifically about how they buy. 

Financial Services spends more on IT and is most likely to consider a new technology for competitive advantage.

ISVs and SAAS are organized and make purchase decisions much differently than corporate IT – although they often are an important market for technology vendors.

Geographies

There are big differences in how these geographies actually purchase products.  Culture can play a big part in acquiring a technology.  European countries frequently prefer working with local representatives – regional offices or channel partners.  Japan absolutely requires local language and culture in their purchasing process.  Canada acts like an extension of the US in the way they purchase and can be easily combined with a general North American study.

Technology and Process Maturity

If you are selling a cutting-edge technology, the best places to do market research are the San Francisco Bay Area, New York, London, and Sydney.  These are all cities with cultures of trying new things. 

On the other hand, if you want ITIL process maturity, Europe is great. And I absolutely love the mid-west (Chicago, Minneapolis, etc.) for identifying “objections” in a very straightforward and honest way 

Bottom line: You should work with your market researcher to figure out your goals and to determine which segments of the market you need to talk to. Then make sure you get the representation from these segments when executing your market research project.

IT Vendor: Things Your Customers Think That Might Surprise You

by

I worked in vendor marketing roles (product marketing, product management, campaigns, etc.) for the first half of my career, before becoming a market researcher.  When I started doing technology market research and spending my days talking to corporate IT without any sales motivations, the conversations changed and I learned a few things that really surprised me and I wanted to share them with you. 

  1. Customers do not spend anywhere near as much time thinking about your product as you do. Even customers that use the product on a daily basis spend most of their time thinking about other things – from people issues to process issues to the other 14 tools they use every day.  They really appreciate it when you make your product and messages simple so they can get on to the other parts of their job.
  2. Customers really WANT to give feedback – happy or cranky, mature or brand new.  They want to talk about their experiences.  Some are very busy and don’t have time, but in general customers love being asked to join customer advisory boards or participate in research projects. There are a few caveats of course:
    • They don’t want to jump through a bunch of hoops to get permission – but that’s easy to solve by offering them anonymity for participation.
    • They want their feedback to go somewhere and yield real results.
    • They want to talk about the products they own not just the new stuff you want to sell them.
  3. Customers HATE IT when you change the names of the products, or product features, they know.  It took them a lot of effort to learn your product, and they don’t like having to relearn everything, or struggle to talk about a product they love because they don’t know the lingo anymore.  And just to be clear, this is not a “minor annoyance.” It’s a HUGE negative.
  4. Corporate IT is surprisingly consistent across companies and markets. There are differences of course, but there are also a LOT of similarities.   <UPDATE:  THIS POINT GENERATED A COUPLE OF QUESTIONS ABOUT WHAT IS DIFFERENT ABOUT CORPORATE IT, INSPIRING THIS POST.>
    • Geography: America, Europe, APAC all talk about the same pain, and same benefits of products. There are definitely differences in the way they buy products (channel vs. direct especially) and each region has different competitive dynamics and maturity of processes and technology, but for message testing or evaluating opportunities you don’t need to go far to get a view of the pains experiences by IT around the globe.
    • Vertical Industry: If an application is mission-critical, it makes very little difference if it’s financial services or manufacturing. The needs are very similar. Government, education, and non-profit organizations buy differently, but again, the pain is the same.

The discovery that surprised me the most? Realizing that customers actually WANT to give feedback and are willing to take the time to do it. As long as they know their feedback is impacting the company and its products, customers are more than willing to share it.

Market Research: It’s All About Listening

by

When you don’t listen to your customers or prospects, you risk losing them.

According to a recent report, “Did not listen to me” is the most widely experienced problem faced by 38% of services buyers. Additionally, 55% of buyers said they would be “much more likely” to consider hiring the provider “if they listened better.”  As Drew McLellan points out, if you add up the “didn’t listen” and the “talked too much,” that’s 63%.  Two thirds of your sales calls are being wasted simply because you talk too much and don’t listen enough! 

Market research is all about listening. It is also about asking the right kind of questions so that you’ll get the right kind of feedback. 

Here are three tried and true tips that I always use in my market research to make sure I actually listen to participants. These tips are applicable to every area of customer and prospect interaction.

  1. Shoot for the 80/20 rule.  Always ask yourself, what percentage of the time are you talking, and what percentage of the time are the participants talking?  I shoot for the 80/20 rule.  If I’m doing more than 20% of the talking, then I’m probably not really listening.
  2. Ask questions that encourage talking – not just “yes/no”.  Are you telling the person what their pain is, or are you letting them tell you about their pain?  A good researcher doesn’t say, “here is your pain – isn’t that right?”  They ask about pain to see if the pain they believe the prospect has actually comes up as a priority.  Of course you can always guide a discussion to keep it relevant, but you need to ask open questions in order to get real feedback – and give customers and prospects a real chance to say “no, I don’t care about that”.
  3. Look for what you don’t know.  Never ask a participant “Do you agree?” 90% of the time they’ll say yes, because that’s the easy and polite thing to do. Your goal is to create an opportunity to hear something you DON’T know. 

To quote the Greek philosopher Epictetus, “we were born with two ears and one mouth, so we can listen twice as much as we speak.”  Thanks for the inspiration Drew.

Market Research: Ensuring Honest Feedback

by

I’ve sat through many advisory board and sales meetings with customers who have just been wined and dined. Then, during the formal part of the meeting, a product manager presents a new roadmap presentation and wraps up by asking, “what do you think?” with a face that clearly shows how proud they are of their work. 

You are very, very fortunate if you have customers that will actually give real, honest feedback in a scenario like that. They want to keep things positive and upbeat – they just saw pictures of that product manager’s kids or heard about their latest adventure trip. They don’t want to get all negative, even if that is what they really think.

One of the things that market research does really well is to intentionally CREATE an environment for honest feedback and for uncovering information you haven’t anticipated.  To create such an environment, you need to:

            1) Make sure the conversation is not about the personal relationship. Of course you want the relationship, and you want to leverage that. But the feedback can’t be influenced by whether someone likes you or not.  Of course they like you – you’re a tremendously likable person!  But the feedback must be about the topic, the market, and the pain points, not about what they think about you. 

You need to use verbal and non-verbal cues to set up the discussion so that it is open, non-judgmental, and the client is never aware that what they’re saying is bad news for you. You need to get the bad news now – not after you’ve been through a long, involved sales cycle that ends because there is “no budget,” which really means “not enough pain.”

            2) Dig for negative feedback.  Don’t just ask, “what do you think?”  Encourage people to talk about what they DON’T like.  You can’t make it better if you don’t know what’s wrong.  Ask, “What do you LIKE about what I just showed you?” then ask “What DON’T you like about what I showed you?” 

            3) Put feedback in context.  It’s not enough for a user to connect with the pain you solve – it has to be a priority!  I recently was in a focus group where a participant described a situation troubleshooting a particularly elusive problem in a complex application. It was a dramatic story with dozens of IT staff from different technology groups getting on bridge calls, pointing fingers, escalating issues, and finally getting the CIO involved in a 48-hour saga that ended up to be a rather minor configuration issue. I could feel the client behind the glass get excited because this was exactly the type of problem they envisioned their solution solving.

But then we asked how important it was to solve this kind of issue so it didn’t happen again. The participant explained that it wasn’t important at all.  It was the first time in 15 years it had happened and they had solved it.  They had moved on.  The pain wasn’t a priority.  Fortunately that group did express other pains that the client could solve, and they learned important lessons on ways of positioning their product that matched the most important pains.

Employing these techniques a great way to encourage your customers to give you real, honest feedback and to help you uncover pains you were unaware of.