Dimensional Research

Technology Market Research

Market Research: Listening for the Unexpected

by

Most businesses have an existing product or market that they are targeting, and their goal is to learn how to serve that marketplace more effectively. Any reasonable researcher should know that if you have a virtualization management solution, it isn’t that useful to talk about HR problems.  But while you need to stay on topic, you also don’t want to miss the chance to uncover information you are unaware of and may be a huge opportunity or a weakness you must address. This is one of the real advantages of qualitative market research where you can spend time deeply understanding the nuances of customer input.

Treat “unprompted” feedback with special importance.

Sometimes the information you don’t know that you need to ask about, is the most important information to hear.  Of course research must have a goal – but work with your researcher so they know your business and can use their judgment to understand what threads are worth digging into and what will take them into a rat hole that does not forward your business. 

The most important tool for unprompted feedback is this question:  “Is there anything else?”  This is the final question Dimensional Research uses for any market research interaction. After you’ve asked all your carefully crafted and specific questions, closing with an open-ended question can yield incredibly useful information.

Of course, this is a very important balancing act and research projects need to focus on their goals – not interesting side topics.  The core questions must be asked. But Dimensional Research makes a point of going through all input from every project with a special eye for the unprompted information.  Most of it is one-offs, with each participant having a special thing that only they are interested in.  But often you can spot trends in unprompted information that are really important. 

We’ve done projects where it turns out there was a critical development tool that needed to be integrated with the client’s product – not something they had realized was vital until the majority of participants talked about it, unprompted, in a series of customer and prospect focus groups.

In another project we didn’t realize that customer loyalty was almost non-existent, until we went through the unprompted feedback from interviews with the highly satisfied, happiest customers, and discovered they regularly evaluated the competition – not a question that we asked about specifically.

So do stay on topic.  But keep your ears open for the “+1” opportunities or customer issues you hadn’t recognized – especially if you hear them from multiple sources.  Possibly minor product changes or messaging tweaks can help you address a different pain, target a new and lucrative market, or address important weaknesses.

Market Research: Why Methodology Matters

by

“You are uniquely unqualified to talk to your customers. You simply don’t think they way they think.”

Jay Ehret caught my attention with this very strong statement. I don’t entirely agree with him – everyone MUST talk to their customers!  But there is an important truth here, which highlights the importance of a formal market research methodology and the key role a good market researcher can play in establishing better communication with your customers. 

The problem is that when you are so close to your product, to your company’s initiatives, and to what you think about your product, you may miss what your customers are trying to tell you. Because, as Jay says, you KNOW TOO MUCH, TOO WELL. 

In this all-too-common situation, a good market researcher can offer a valuable outside perspective on your company and on your product.

I recently got an email from a client after presenting a report. It perfectly captures the value of Dimensional Research:

“I must admit that I was very surprised at how good the results of this project were, especially since you hadn’t worked directly with our solutions or this specific target market previously.  In the course of this project, you learned what’s going on with our market well enough that you articulated what’s happening better than we have been able to ourselves. Thank you.”

Of course, I appreciated the feedback, but I wasn’t surprised.  A good market researcher quickly figures out what’s going on, in part because they don’t know as much as you do.  External perspective can give important insights, not because you don’t know the information, but because you know so much MORE than the basics, that the really key information can get hidden.

Applying a formal market research methodology can bring important information about your business to light and provide valuable clarification. You can then act on that information, combined with your deep knowledge of the product, to drive business results.

New from Dimensional Research and KACE: Survey on Windows 7 Adoption

by

KACE – the leading systems management appliance company and a fabulous client – just announced the results of a new Dimensional Research Web survey that revealed 84 percent of IT staff polled do not have plans to upgrade existing Windows desktop and laptop systems to Windows 7 in the next year—despite early enthusiasm from beta testers of the new operating system

Other key findings include:

  • 84 percent of survey respondents have no plans to upgrade existing Windows desktops and laptops to Windows 7 next year
  • 72 percent indicated they are more concerned about upgrading to Windows 7 than staying with an outdated XP operating system
  • 50 percent revealed they have considered moving from Windows to an alternative operating system, and 27 percent of those cited Mac OS as the top alternative
  • Almost 60 percent of survey respondents do not presently have a tool in place that automates operating system migration
  • Economic factors, such as budget freezes and staff reductions, were cited as other reasons to not immediately adopt Windows 7

You can download a full copy of the report, as well as detailed information on the methodology used at http://www.kace.com/resources/Windows-7-Adoption-Survey.

 Here’s a few links to some news coverage of the survey:

Message Validation: Market Research with Clear ROI

by

One of my favorite market research projects is message validation. When we go and get direct feedback on high-level product messages, I always know we are going to learn something very important that will dramatically increase the results of marketing programs.  Hard ROI is easy to demonstrate when your marketing programs get twice (or more) the response from the same spend, because your message is more compelling!

So how does message validation work?  We evaluate your messages – the benefits and features that you give your PR department, your marketing programs team and your agencies – and break them into digestible chunks.  We set up one-on-one interviews with both customers and prospects (messages are experienced in isolation so this is not appropriate for focus groups).  And we walk through a systematic process of vetting the messages. 

We never ask, “What do you think?”  That doesn’t force participants to share what is really going through their heads.  You need to go deeper.  Instead of “what do you think?” you must ask, “what is MOST compelling?” and “what is LEAST compelling?”.  Do that for each area of messaging that you have: pain, benefits, features, positioning paragraph, etc.  Specifically prompt for good and bad  – and always ask WHY.  The “why” helps you know if you need to throw out some messages, use different words to express the same idea, re-order the importance, or segment your messaging.

What is interesting when you do this is how small changes can make a huge, measurable difference to your programs.  On the first message validation project I did in my career, we tested four main messages that the marketing team had been using for outbound emails and direct marketing with uninspiring results. The client knew that if they could get that 1-hour meeting, the clients were hooked, but the programs just weren’t getting the same level of response.

We tested the messages as part of an overall research project, and found that 3 out of the 4 messages resonated extremely well with everybody.  But that fourth message was different. Two out of our three target segments not only didn’t find it compelling, they actually found it insulting and frustrating. 

That said, it wasn’t that the fourth message was bad across the board.  The final target segment really liked the fourth message, even though it was actually alienating the majority of the market!  The client reserved that fourth message for a campaign targeted to that specific audience.

Solid, compelling messages about your product are the basis for a successful marketing program. Message validation helps you fine-tune your messaging to ensure maximum effectiveness of your communications with customers and prospects – a great cost saver.  

Bonus Tip:  If you’re looking for budget to fund a market research project, consider adding a message validation component and using marketing program budget!

Six guidelines for compensating research participants

by

There is usually some form of compensation for participating in a market research project. For focus groups and in-depth interviews, research participants are usually offered some kind of stipend in appreciation of their time.  The question I get asked frequently is – how much?

I’ll start by saying that the stipend is usually NOT the primary motivation for participating in technology market research.  I find people are genuinely interested in expressing their opinion. They like to be heard; they like to hear about new ideas coming down the pipe; they like thinking they are influencing the market or the product; and – in the case of focus groups and customer advisory boards – they like to hear what is happening with their peers.  And let’s face it, corporate IT employees are typically paid reasonably well, so a hundred bucks isn’t going to really compensate them for trucking across town to a focus group facility, spending 2 hours talking to you, then trucking back home.

That said, the stipend is key in attracting the right audience, and is perceived as an important added benefit. So back to the question of how much.  The answer depends on a variety of factors:

1. What are you asking people to do?  Focus groups typically take about 2 hours of participants’ time. In-depth interviews usually take from thirty minutes to one hour. In addition, for in-person focus groups, the participants need to get to a specific location.  As a result, focus group participants are usually compensated more generously.

2. How big is the target pool of participants?  The harder they are to find, the more you should sweeten the pot to simplify the recruiting.  If you have a straightforward recruit like “application developers” or “network administrators”, there are plenty of those and you don’t need to have a particularly large stipend to entice them to join. 

However, if you need something really specific like “DBAs responsible for MATISSE databases”, or customers of a competitor that only has a few hundred users worldwide, you want to have a large stipend to make sure that you’re doing everything possible to attract the few people out there that match the recruiting profile.

3. What level of participants are you looking for?  CIOs and other IT executives make more money than sys admins or developers, so the stipends need to be higher to match their expectations for the value of their time.

4. Customers or prospects?  Depending on the context, offering a customer money to give you feedback may be tacky.  Particularly if it’s a customer advisory board meeting, the real value is the chance to be heard.  In this case a nice gift with the corporate logo on it is much more appropriate.  

5.  Give the philathropic option.  More and more corporations have implemented strict policies that employees CANNOT receive any gifts of any kind.  Dimensional Research always offers our research participants the option of donating their stipends to the charity of their choice. 

6.  Web surveys are different.  When you’re looking for hundreds or even thousands of respondents, any stipend, no matter how small, can quickly blow up your research budget. There are certainly options like being entered into a drawing for a gift certificate.  Or the ever popular drawing for the hot electronic item of the day – the iPod Touch has been particularly common in survey drawings in the past year. 

One thing to consider, if the findings are not confidential, is to simply offer a copy of the final report to participants.  This is a very high-value offer to people who care about the topic – probably even more than a gadget – and has the added benefit of not attracting participants who don’t care about the topic and might not give the most insightful or informed answers.

Three Tips for Crafting Better Online Surveys

by

There are many ways to make sure your online survey is efficient and effective. One of them, of course, is to avoid asking bad questions.  

Tip 1: Craft your questions carefully to avoid unwanted results 

This tip was inspire d by Seth Godin. He explains that “Every question you ask changes the way your users think. If you ask, ‘which did you hate more…’ then you’ve planted a seed.” 

Mr. Godin makes a great point. I recently booked a trip with Travelocity. It was a great trip. I was happy.  I did have a small issue with the airport transfer getting home, so I filed a complaint to see if I could get a refund.  It was a small matter – about $30 credit – so I wasn’t too worried about it. Travelocity’s reply email was pretty typical, asking me for more information (which I had to get by opening up the email THEY sent me, so a bit annoying). 

BUT… Then they sent me a survey asking about my experience with Travelocity.  One of the questions on the survey was “Do you know about Travelocity’s guarantee that your booking will be right, or we’ll work with our partners to make it right, right away?” I actually didn’t know, but this question clearly did NOT describe the experience I just had. 

Travelocity did make it right – it took them about 30 days to do so – but that wasn’t what their guarantee said and their survey pointed that out to me.  By including that question in the survey, they planted a seed that they didn’t want to plant and I ended up being less happy overall than I was before the survey.

Tip 2: Ask at least one question that participants actually WANT to answer

It’s important to ask the questions your customers are actually interested in answering.  Too often, marketing departments are so focused on the company’s newest offerings that they ignore the products their customers have come to depend on.

I use QuickBooks for my business. I have LOTS of feedback for Intuit on the core QuickBooks product, but they never ask me about that.  They constantly survey me, asking if I want to buy checks or do payroll or take credit cards, but there is no “Thanks for your time, is there anything else you’d like to tell us?” that would enable me to give them the feedback I WANT to give them.  

Another example from my own professional life: I use Zoomerang for my surveys. It is a GREAT product – with a few caveats.  One of the problems I have with the product is that they have a horrible interface for “choose the answer that most closely applies.”  It’s a small button that’s almost impossible to see. On the other hand, their interface for “choose all that apply” is great – a nice square with a big check mark.  I want to tell them about this issue, but they keep sending me web surveys about other things and I’ve never had a chance to give them important feedback that I really want to give them. Maybe they’ll read my blog and I’ll get to them that way?  <CORRECTION:  They did in fact just send me a survey this week that allowed me to give that feedback.  I’ll wait and see if they act on the feedback.>

Tip 3: Reward your participants, wisely

This tip is inspired by Patricio Robles: “offer users who respond to a survey a discount, an entry in a drawing for a prize, something of value. It will boost response rates and make them feel like they’re investing their time wisely.”

While I generally agree, I would add that when offering a reward to participants, it’s important to consider your target market. A gift that is too nice motivates people who aren’t qualified to complete the survey. Then you have to wade through junk or set up lots of qualifying questions to weed them out. If the reward is nice enough, some people will game the system and try to guess what you’re looking for, so the filtering questions won’t always work. 

When you’re doing a technology web survey, this is especially important, since people who don’t know your topic can really throw off the results, as they are not informed.  If possible, giving a copy of the final report on a topic is a great incentive, since only people who know and care about the topic will respond to this type of reward, keeping your input very clean.

“Technology Expert”, Defined

by

Dimensional Research clearly stands apart from other market research firms because we are technology experts, performing market research for technology companies.

But what exactly does it mean to be a “technology expert” when it comes to research? Obviously it doesn’t mean we’re deep technical experts in every possible area of corporate IT. We don’t write code or manage developers. We don’t manage data centers (although we do our own IT and can commiserate with some of the pain!). We don’t personally install patches or upgrades or test new releases of enterprise applications.

What we’re experts on is Corporate IT and how it works. We are very familiar with Corporate IT and its processes and can talk comfortably and knowledgeably about:

1. Technology. Not the actual coding and implementation, but the types of technology and the tradeoffs.  We understand the perceived benefits and potential downsides of a service-oriented architecture.  We know what cloud computing is (or at least, know all the ways the industry uses the word) and what is real and what is vision.  We understand the challenges with data and the effort that is needed to generate simple graphs and reports.  We know the TLAs (Three letter acronyms) – from IDE to BSM to SDL to SAN (Both the storage one and the security one!)

2. Processes. We know that ISVs and SAAS companies purchase differently than financial service organizations and that those are different than government and non-profits.  We know the software development lifecycle, the people involved, and the products used. We are familiar with ITIL and know how some organizations treat it as a guiding principle, others as one of many guidelines, and some just ignore it.  We know the new focus on costs and services in ITIL v3.  We also know IT has other systems – Six Sigma, COBIT, and more that need to be considered

3. People. We know the common organizational interactions that take place between IT and the business owners; between applications and operations groups; between development and testing; and between server and network teams.  We love talking to engineers and technologists about their challenges, and appreciate the creative side of a job that is often considered to be purely technical.

Most importantly, because we have this foundation, we can quickly come up to speed on any new technology. 

  • On a recent security project we had to learn new vocabulary like “pen testing” and “cross-site scripting”. It didn’t take us long, because we already knew the development and testing processes and quickly understood where security fit into those.
  • On our first data center automation project, we had to learn about different types of automation.  But the products automated processes we were already familiar with, so it was a very straightforward learning curve. 
  • When we did our first cloud computing project, our foundational understanding of virtualization, systems management, and application management meant we were able to be productive right away.

Being technology experts means we “get it.” Whatever technology our clients are developing, whatever processes they use, we are very comfortable talking with IT using professional terminology, asking the right follow-on questions, and gaining the respect of the participants. The result: an efficient and focused market research process that generates a highly effective analysis.

Market Research During Recession

by

Is the market research industry recession-proof? Ray Poynter thinks it’s not. He says, “I have been blogging, reading Tweeting, and engaging in online discussions to try and estimate where research is heading, and I think the story is depressing.”

Mr. Poynter goes on to say that the only reason Market Research has been doing OK so far despite the recession is that companies are still using what’s left of their budget, but this will dry up in April; and that companies are cutting the heaviest expenses first and have not yet reached the stage of cutting smaller expenses such as market research.

I disagree, especially when it comes to Market Research for technology companies. In the sub-market of companies who sell to corporate IT, we’re seeing a very different trend:

  1. Market Research can be a huge cost saver. Instead of spending $250K on a marketing program, our clients see the value of spending $20K on message validation so they can spend $150K on a marketing program that delivers four times the results of the more expensive one – because the messages are hitting exactly the right buttons. I just completed a message test that included evaluating four possible lead benefits.  We found that three of the benefits were incredibly compelling to all audiences, but to 3 out of 4 of our target segments the fourth benefit was actually off-putting and offensive.  Just by deleting that one “bad” bullet from outbound programs their click-through rate on emails went from .5% to 2.0 %.
  2. It is now more important than ever to stay close to customers and build customer loyalty. A Customer Advisory Board clearly shows the participating customers that their voice is being heard – an important factor in instilling loyalty. Done right, it also gathers key information to ensure every functional area of your company is acting in a way that increases customer loyalty – from the product itself to messages from the marketing team, service offerings and ideas for improved support. In a recent customer advisory meeting we ran, the customers came up with a way to administer licenses that was more convenient for the customer and less expensive for the vendor – a win for all groups. Customer Advisory Boards can increase customer loyalty and drive more business to your company, even during the toughest times.
  3. During a recession, ROI is more important than ever. Prospective customers with tight budgets need credible and unbiased information on your products before they buy them. You will need to give your sales team reliable, third party validation of ROI. Turning that ROI into collateral to prove ROI to prospects is highly valuable. And, technology companies have the unique option of getting more for their buck by combining the market research and collateral budget items.

As Mr. Poynter says, “I have been in this industry for 30 years, and I have seen plenty of people lose their jobs and companies go bust. But I have also seen some companies do well, under any circumstances. Some research companies will grow, despite the next two years.” I believe that technology market research companies are in an excellent position to not only survive the recession, but to grow and flourish despite the rough economy. 

Tips For Engaging Customers In An Ongoing Customer Advisory Board

by

Getting your customers to participate in a customer advisory board isn’t too difficult: customers love to sit on advisory boards.  They appreciate not only the opportunity to be heard and to have a relationship with a key vendor, but also the opportunity to hear what’s happening with their peers and to make important industry connections with your other users. 

However, ongoing advisory board programs do require a level of commitment in order to develop a great group dynamic. Typically we ask for a one-year commitment which includes four quarterly meetings.  My personal favorite is to kick the year off with an in-person customer advisory board meeting and then conduct three online meetings.

The best way to keep customers engaged with such an ongoing advisory board program is to show them that their feedback is actually impacting the company and its products. One way to do that is to start all customer advisory board meetings, except for the first one, with a summary of how the feedback from the previous meeting has impacted the company and its products.

But you can go even further and engage your customers even more. When I ran marketing at Freshwater Software, we had a product-focused advisory board (the “ThinkTank”) that was held bi-annually.  Participants were asked to come prepared with their top three product requests. During the meetings, we spent time reviewing those requests and prioritizing them. 

We used to end the meetings by voting on the top three features the group wanted.  We committed to customers that if they took the time to participate, we would guarantee that at least one of their top three features would get into the product. We were often able to get more than just one of their top features into the product. But the important point is that this kind of commitment drove active and passionate participation in the advisory board meetings.

In-Depth Interviews, Focus Groups, or Both?

by

When doing qualitative research, we need to decide which is right for the client: in-depth interviews, focus groups, or maybe a combination of both. This depends on the client’s goals.

When are focus groups better?

Dimensional Research recommends focus groups when the client wants to gain multiple perspectives in an interactive group setting.

One of the main benefits of focus groups is that they get the participants brainstorming. When one participant’s comment feeds off of another comment and so on, the group can really dig deep into an issue. When trying to evaluate market acceptance, capture challenges and issues, or understand objections to new technologies or processes, the focus group dynamic is ideal.

Focus groups have another great benefit – the client can sit behind the glass or on a conference call and hear the direct, unfiltered feedback of a large number of participants with no distractions. Focus group sessions are also recorded for further observation.  If your goal is to expose the maximum number of your team to direct input from the market, this is a very efficient way to do it.

When are interviews better?

In-depth, one-on-one interviews with technology professionals can be conducted in person or over the phone. These are appropriate when the client wants to identify detailed perceptions, opinions, beliefs, and attitudes.

In-depth interviews are particularly effective in the following scenarios:

  1. When the client’s goal is to capture feedback on experiences that occur in an isolated way, such as product messaging or product usability testing.
  2. When there is anything sensitive about the feedback that participants may not feel comfortable sharing in front of other people. They may be concerned a competitor is also be attending the focus group. Or in a win/loss analysis, a customer may not feel comfortable sharing details of poor account manager performance if they suspect the rep might be “behind the glass” and could hear him. The customer would feel much more comfortable sharing this information confidentially talking only with the researcher.

Two things that should NOT drive this decision are:

  • Travel – Sometimes the decision is driven by the geographical locations of participants. If you’re doing customer research with customers located in diverse locations or if you want global representation in the groups without the cost of travel, you still have a choice. The one-on-one nature of a phone interviews is an easy option, or for the group dynamic choose online focus groups.
  • Cost – An important consideration, of course, is cost. As a rule of thumb focus groups and interviews cost about the same per participant, so with the exception of travel – not a consideration for phone interviews or online focus groups – cost should not be the driving consideration in choosing the research approach.