Dimensional Research

Technology Market Research

Market Research: The Importance of Vocabulary

by

Model Metrics, a provider of services at the cutting edge of cloud computing, recently sponsored a Dimensional Research survey on iPad and tablet adoption in the Enterprise. It was a good project with a fun topic and the chance to work with great client. It also demonstrated very clearly the importance of knowing how your audience uses vocabulary.

Consider the following two findings from the survey:

  • 22% of IT professionals reported having officially deployed tablets
  • 72% have tablets in use for business purposes in the enterprise

If you don’t know the language of corporate IT, this probably doesn’t make sense. How can there be iPads in use for business purposes when they haven’t been deployed yet?

The answer is in the word “officially.” One of the most interesting findings of the study is that employees are purchasing iPads or other tablets on their own, then bringing them to work, and IT has to support them. IT hasn’t deployed the iPads “officially,” but they have to support them anyway.

A great example of how words matter in market research. You can read more about the Model Metrics study here, or download the full report here.

Research Bias: It Can Happen to Anyone

by

We are nearing the end of our year living in Europe.  One of our goals for this trip was to experience the rich Roman and Medieval history in the area – most of which requires driving south. It is very quick to get around France once you are on the freeway, but freeway entrances are fairly far apart.

From our house, you can get on the freeway by driving north or south. The south entrance, we will call this Route A, is further away, but is in the right direction once you are on the freeway. Here is the Google Map from close to our house to the town of Orange.

The north freeway entrance, we will call this Route B, is closer, but is north from where we live so takes you out of the way. However, you spend more of the time driving freeway speeds.  Here it is on Google Maps.

Now here’s the thing – my husband and I do NOT agree on which route to take, and driving in the car offers plenty of time to discuss why our particular point of view is the correct one.

My husband started with the stats from Google Maps to support his point of view:

  • Route A: 46.3 km,  47 minutes  (shorter and faster)
  • Route B: 64.3 km, 51 minutes (longer and slower)

I pointed out that Google Maps is based on algorithms and not reality, so suggested we do some real-life measurements. The next few trips south I wrote down our departure times and arrival times at a set point on the freeway both coming and going (yes, we are that kind of couple!) using both routes:

  • Route A: 46 minutes, 58 minutes, 53 minutes, 52 minutes  (faster 1 out of 4 times)
  • Route B: 50 minutes, 52 minutes, 51 minutes, 49 minutes  (faster 3 out of 4 times)

Now, it should be immediately obvious that my husband is an advocate of Route A, and I prefer Route B. And we both have data to prove that we are right. My husband would point out that my measurements were invalid since on the trips where Route A was slower we ran into unusual traffic. I would point out that if we ran into unusual traffic 3 out of 4 times it probably means that the traffic is not that unusual.

Of course, we should know better.  He is an engineer and I am a mathematician. Before we started all our “fact finding,” we should have just put on the table that we are biased. I don’t like roundabouts, so getting on the freeway sooner is a way to avoid them.  My husband finds the freeway boring and prefers going through the towns.

So back to the market research topic. In this case the final result doesn’t really matter. But in any market research project, it’s important to know and confront biases at the beginning of a project, so you can ensure they are addressed. Findings must be strong enough that exceptions that support existing biases are identified and understood.

Using Your (Web) Analytics for Good

by

We love analytics. Of course as soon as I started blogging we installed Google Analytics. We love watching the numbers go up and down and seeing what topics are most interesting to our audience (Our #1 post on this blog is Focus Groups vs. In-depth Interviews, by the way), whether external events, such as being named a Top 10 Research Blog by Quark Magazine, drive traffic more than specific content, and so on.

But analytics – especially web analytics – can be very misleading, because it typically misses the demographics. WHO is doing that behavior and HOW are specific parts of your market behaving differently.

We recently blogged about a client who thought that videos were their most compelling web content because the web analytics showed a lot of video activity. But it turned out that the technical buyers – arguably the most important audience, but a small portion of web site visitors – didn’t use videos, a detail that was completely lost in the overall analytics data.

So here are two very simple tips for using Web analytics for good:

  1. Assume analytics don’t tell the whole story – There is a lot of useful information in the analytics, and it should absolutely be used. The danger occurs when analytics become undiscussable facts. Start with the assumption that even with the data you don’t know everything, and you’ll be able to ask the right questions to use the best analytics and look for more information when needed.
  2. Don’t stop talking to customers because you have data – The video example mentioned above popped up after talking to just a handful of people. It doesn’t take many live conversations to figure out what is actually happening with the people who are spending money on your products. Talk to some of them!

Market Research: Can you trust it?

by

Hal Varian, chief economist at Google recently said that the “the sexy job in the next 10 years will be statisticians”.

I think he’s right.  Every participant in a first-world economy should have a solid understanding of stats.  Not that I’m recommending that everyone become a statistician, but with the volume of information out there, it’s important to understand how data can be used to sway you.

Here are two “statistics” about my own life:

i) I was the only female graduate in Pure Mathematics at my university in my year  (true).

ii) 50% of my graduating class who majored in Pure Mathematics were women (also true).

One of those statements paints a picture of a mathematics education that is oppressive to women, somehow subtly driving females away.  The other paints a picture of a very progressive math department that gave women the same opportunities as men.

As I’m sure you’ve figured out, there were only two people in my graduating class that majored in Pure Mathematics.  But without that information, I could have easily mislead you. Clearly the statistical significance is not there.

When someone presents information, whether a market researcher or anyone else, always think about the reliability. What’s the methodology?  How big is the data set? Is the audience who completed the research the right audience to comment on the topic?  Is it representative?

Research bias is very real. You need to be aware of this and always ask the right questions in order to determine whether information presented to you is valid – whether selling enterprise software or listening to the media.