Value of Research

Finding the Story in the Research

On a big reading “catch up” over the long weekend, I read two blogs posts that caught my attention:

The first was Seth Godin’s Are you doing math or arithmetic? This blog post resonated because my degree is in mathematics. [Side comment: The best way to horrify your math professor is to refer to basic arithmetic activities like addition or fractions as “mathematics.” Mathematics has nothing to do with computation at a certain level. It’s about taking your facts (called axioms), and applying pure logic to come up with a proof. Knowing that something is unarguably true or false – that is the real beauty of mathematics!]
I also read Ivana Taylor’s challenging blog post about the future of market research firms. Ivana beautifully outlined the decreasing value that exists in collecting information – it’s simply too easy to do that these days.

So the question: where is the value – or the “mathematics” as Seth Godin would say – that clients pay research firms good money for.

The massive report that shows everything is not it. It’s not that difficult to spew the data from the information gathering phase into a document. It does provide value to the internal people who take the time to read and absorb the details and is not a step that should be skipped. But let’s face it, that part is the “arithmetic.”

There is a point in every research project where you have to take those massive amounts of information, gathered over a huge number of interactions, and figure out how to communicate what matters in a one-hour, or a 30- minute or even a 10-minute meeting.

This is where the value is found – in using the information to build a compelling story. This is the “mathematics” of research. Creating the story of reality in a way that is understood and actionable by a wide audience.

Very easy to say. Very, very difficult to do. Extremely high value when you do it.

Innovation vs. Optimization

by Diane Hagglund

Seth Godin wrote an interesting blog post about Netflix. In the post he says that “Netflix tests everything. They’re very proud that they A/B test interactions, offerings, pricing, everything.” But then he notes that “The three biggest assets of the company weren’t tested, because they couldn’t be.”

It was (as expected from Seth) an intriguing and thought provoking post, and made us think about the role of research and where it fits.

We’ll confess: in the Dimensional Research client base – mostly B2B technology companies – we don’t spend much time with the true technology innovators. Every successful technology company has a few amazing minds that come up with the technology ideas that change the face of Silicon Valley – the mouse, the browser, virtualization, cloud computing. I’ve worked with some of them and met others socially and I’ve never heard of a really breakthrough technology idea that came from a focus groups. The innovator brains are visionary and are so far ahead that only a small subset of their future customers can even understand the idea when it is explained, let alone articulate the need for innovation.

But those amazing technology innovators are usually only a tiny part of the technology company machine. There are much larger teams of people who need to take those ideas and develop, market, sell, support, collect payment, and so on. Those operations don’t require INNOVATION, they need OPTIMIZATION. How can they do what they’re doing more effectively? How can they generate better business returns for lower cost? It’s here where there ARE testable ideas, where customer feedback makes a huge impact on business results, and market research truly shines.

Yes, innovation is absolutely necessary and companies need those breakthrough ideas. But optimization is also necessary if you’re going to build a great business.

As always, we welcome contradictions so we can evolve. Have you used market research to come up with a truly innovative idea? How?

Controlling the Impact of Observation

by Diane Hagglund

This tweet made me smile:

It was very funny, and anyone who has sat on the other side of a customer service call probably wishes for more “big bosses” to observe. But as a researcher I also found this to be a classic example of how you have to be very careful, when conducting market research, that you don’t influence behaviors just by being there. Clearly this boss still doesn’t really know what his staff is doing.

Of course researchers have all sorts of tricks for managing this dynamic, depending on the scenario. In my experience, the most important thing in preventing the impact of observation is to be aware that it happens. Simply asking, “What exists in this research environment that may influence behavior?” goes a long way.

I don’t know the background of @BAbackpacker’s situation, but if the “big boss” really wanted to understand the customer service dynamic, he could have recorded a wide range of phone conversations without letting the staffer know which specific ones would be reviewed or listened in, using a technology that was less obvious. Of course he could also hire a professional researcher who would be able to analyze the situation and create a more natural environment.

Do-It-Yourself Research

by Diane Hagglund

As a technology-centric market research firm, our competitors aren’t usually another market research firm – very few have the experience with corporate IT that we do. Our #1 competitor is “We know our customers, we don’t need a research firm.”

We’ll be the first to say that a company does not need to hire an external agency to do market research. (I’ll pause for a moment so all the professional market researchers can catch their breath at the blasphemy.)

Back when I had my career on the “other side of the glass,” I made it a point to talk to my customers regularly. I’m a big fan of the Crossing the Chasm model of having the product marketing manager run the beta. I always found that while my R&D counterpart did his magic to get a new software product to actually work, I could chat up the user and find out a lot about their pain, and why they were bothering to spend the time to deal with a product that was so buggy it needed a PhD to install it.

There are many benefits to “internal research.” It doesn’t have to be formal. Go to a trade show and give your elevator pitch to 100 people in one day and see how many get it – you will learn a LOT from the questions they ask in response to the pitch. Invite 5 users to a 1-hour phone call and ask them a few questions and really listen to what they have to say to each other. Take a customer out for dinner and hear what they have to say outside of the confines of the office. You will learn an incredible amount of important information from any of these activities. And in fact, because of your expertise, you may get even more from these exchanges than an external researcher would.

The trick is to figure out when an external perspective is going to add additional benefit. There are three areas where external resources can add real value:

1. Just Listening

I was presenting a customer report on product ROI and the client – a really talented product marketing manager – asked, “how do you get them to tell you that stuff?” The problem with being an internal stakeholder is that you are expected to have the answers. Which means that any customer interaction will be an exchange of them giving you information and you dealing with their concerns. As researchers, we can gather input, but since we don’t really know the answers to the questions, we don’t have to defend or position anything. We have the luxury of just listening to the feedback and drilling in to really understand it, without needing to come up with the answer on the spot like an internal stakeholder is expected to.

2. Neutrality

As researchers, we have no vested interest in the outcome of a research. We have never fought for a particular point of view. Good researchers never develop preconceived ideas before the research is executed. External research (conducted well of course) is absolutely the most neutral form of feedback you can get.

3. Finding Non-Customers

Researchers have access to all kinds of people that you may not – people who have never heard of your company, competitor’s customers, prospects in a new vertical industry. If it can be profiled and there are enough people in the world that match that profile, external researchers can find them and get them to talk to us when you can’t yourself. (Actually sometimes it’s even easier for us to find customers, depending on how bad the client’s customer database is – an ongoing problem with enterprise technology companies in spite of the improvements in CRM in the last several years.)

Can Research Do More Harm than Good?

by Diane Hagglund

First, an obvious point. We here at Dimensional Research truly believe in market research. We wouldn’t do it for a living or spend time blogging about it if we didn’t believe there was something fundamentally useful about going into the marketplace and getting feedback.

But a couple of things we saw and read this week made us pause and reflect on the value of what we do:

A regular feature in a popular comedy show is “New Rules.” In a recent show, one of the new rules offered was “Online shopping sites must stop asking for my feedback.” This was the biggest applause line of the entire show. The audience loved it!
We came across an interesting blog post from Tim Berry on Bad Research. Tim argues that it’s “Better to know what you don’t know than to make business decisions based on false information and false conclusions. If the focus group said red is better than green, nobody dares to argue for green. Even if green is really better, and the focus group was off, distorted by one very articulate and engaging green hater. Red it is.”

I understood the audience’s reaction to the “give me feedback” line. It’s frustrating to be asked to give feedback when you don’t believe it matters or don’t believe it will change anything. And I get where Tim’s angst about research comes from, especially the scenario he talks about which looks like a clear case of a “Fascinating Outlier“.

Does the research industry need their own version of the Hippocratic Oath where we swear to “first do no harm?”: We solemnly swear that we will not frustrate customers while getting their input and we will not use research to mis-inform.

We think that the learning here isn’t not to do research, it’s this: Do GOOD research.