Focus Groups

Three Tips for Online Focus Groups

We are doing more and more online focus groups. We’ve always done some, but have noticed in the past year that we have definitely “crossed a chasm” and there is now heavy interest in online focus groups.

My take from conversations with clients is that this interest in online focus groups is caused by a mix of the technology getting better and people just plain getting used to the idea. It’s a typical technology adoption curve – in the beginning it is different and you’ve never done it before so it feels risky. Then you do it once, realize it can work, and from then on you’re a convert.

I’ll admit – we’re converts. I’ve heard people online saying that online focus groups are frequently pushed because researchers are too lazy to travel. That’s not us. We LOVE to travel. I look forward to those crazy New York/Chicago/Denver/San Francisco weeks. Throw in some Europe and Asia and I’m in heaven. So we’re converts in spite of the fact that we have to stay home to do research. You can reach so many more people and include participants from the non-major cities that are an important part of the target audience – all for the same budget.

That said, online focus groups are different. It is challenging to keep people focused, especially our IT participants that have email and IM and texting all distracting them during the group. Here are three of our best tips that we’ve learned over the years:

1. Have a Smaller Group

We’ll usually recruit eight IT professionals for an in-person group. For online groups, we recruit six. It’s hard enough keeping these IT guys engaged and off their blackberries and iPhones when you’re in the same room, it is MUCH harder when you can’t see that they’re doing email instead of talking to you. Smaller groups mean the participants each have more active time and are less likely to tune out.

2. Have a Strategy to Keep Intros Short

When possible have the moderator touch base briefly with each participant the day before – this works best when doing customer research. Get the basics on what they do and put that on a slide to share with the group. Cut short the “round the table” introductions and move to the more interesting and engaging topics. We usually use a slide to share first name, industry, and major infrastructure or application responsibilities related to the study. Then in the “round the table” intros we try to do something more interesting like where they went on their last vacation or their best “silly question a non-geek asked them” story.

3. Use the Tools

Every time you call on a participant you grab their attention. You can also grab their attention by switching the visual or adding a poll. Find ways to use the online meeting tools in an interesting way. Use them as a whiteboard for capturing notes, list talking points to enforce the spoken word, use declarative statements for the discussion and write those down in a prepared slide rather than just reading from the guide.

Online focus groups definitely require planning and preparation to keep participants engaged, but with a bit of effort, the results are fantastic.

In-depth Interviews or Focus Groups: A Reprise

by Diane Hagglund

Ever since I published In-Depth Interviews vs. Focus Groups, almost two years ago, it has been our most visited blog post. Not only is it the most visited post of all time, it has also been the most visited article on this blog every month since it was published.

It seems to be pretty obvious that people are confused about the difference between the two most basic methodologies for conducting qualitative market research. As researchers, we continuously execute both types of research approaches and instinctively understand which will be best to meet the goals of a particular project. But we need to be very clear with clients who are not research savvy.

I frequently find myself having these types of conversations with clients. I often get requests asking for a quote for a certain number of focus groups, but when we review the goals of the project, it may turn out there is an opportunity for much better results given the same budget if we were to do interviews instead – especially with competitive research or message validation.

I’ve learned a couple of things:

1. Don’t be afraid to suggest an alternative to a client if it’s right for the project.

Your job is to be an expert and provide the right research solution, not just deliver against the tactics of a client’s request.

2. Be clear and use simple terms when explaining research methodologies.

It’s easy to lapse into “researcher lingo,” but clients will resist alternates if they don’t understand them.

3. Use your best researcher listening skills to figure out why the client is asking for a specific approach.

Clients frequently don’t know alternatives exist. They may have observed a focus group once and saw good results and are asking for that because they don’t realize there are other approaches. It’s also typical that there is an unstated goal to expose a particularly difficult stakeholder to direct market feedback, in which case a focus group may be the right choice, even though other indicators would suggest in-depth interviews.

A Researcher’s Confession

by Diane Hagglund

I admit it. Although web surveys are one of our most popular research offerings, I strongly prefer qualitative research (focus groups, in-depth interviews).

I can’t help looking at the findings in web surveys and feel like I want to dig in and ask more questions. The participants say things that don’t make sense to me and I want to know why. Or they make short comments responding to open-ended questions that leave me with a dozen follow-on questions I don’t have the opportunity to ask.

When I present qualitative findings, I know I can answer any question that comes up with complete confidence. With quantitative findings, I always know that there will be questions where the answer is, “We can’t draw any conclusions based on this survey.”

This came up again last earlier this month: We just did a series of in-depth interviews where we wanted to understand perceptions about the cost of various alternatives. All the participants in this study had identified themselves as product decision makers who had full visibility into costs – a requirement for the study. If you had just looked at the first level of answers people gave, you would have thought that one of the tools we were looking at was very inexpensive compared to other options.

But because this was an in-depth interview, I got to ask that all-important “why” question. I quickly realized that while all the participants had been educated on the “line 3” costs that were billed directly to their organization, not everyone was aware of the additional “line 10” costs that had to be added to support this different approach. When you added both of those up, the tool that originally appeared less expensive turned out to have a similar TCO to other options.

Now, it’s true that we could have found this out by writing a good web survey, but one of the secrets to writing great web surveys is to know the answers to all the questions first. We continue to recommend web surveys as good vehicles for quantifying concepts that you know well, but want to put an accurate % by each of the options. This is a valuable thing to do, especially for market sizing, external marketing and PR purpose.

But for finding out the answers that you don’t really know, start with qualitative research – and by all means do a web survey next to put those %s in place once you know the statements to put the %s with.

Market Research: Listen Live or Wait For the Report?

by Diane Hagglund

One of the real strengths of focus groups – in person or online – is the opportunity for a bunch of people to see a live discussion, and even ask a few additional questions. (Yes, Dimensional Research always leaves a few minutes at the end of a focus group session for the observers to ask a follow-on question or three.)

It can be extremely powerful to expose people who work in corporate roles and don’t get out into the field – marcom managers, R&D, developers, etc. – to direct customer and prospect feedback. Often operational people (finance, legal) pick up something important by watching their target market discuss their jobs.

Dimensional Research always encourages as many people as possible to listen into focus groups, or to watch the videos that we record when they’re done.

However, sometimes “listening in” is a bad idea. It basically boils down to this: If you only see part of a project, don’t assume that’s all there is. There is a reason why you conduct 8 focus groups, not just one. Or why you conduct 25 interviews, not just 3 or 4.

Don’t let these scenarios happen to you:

Attend two focus groups in New York and project that experience onto Chicago, Paris, Singapore, and Tokyo.
Listen in on only one call of a 20-call interview project.

If you decide to listen in on market research, I strongly recommend the following:

Do read the final report and attend the presentation of the report. You might as well enhance your limited experience with the full power of the overall project.
Don’t attend just one focus group or listen in on just one call!

Real life example

We recently conducted a series of 15 customer interviews about a client’s new initiative. It has been progressing for about a year and they wanted to know what messages their customers had absorbed. We spoke to 15 of their very best customers – the kind who spend lots of money every quarter, attend the user groups, and give references. It was a good study, and very helpful in finding out what parts of the new initiative were gaining traction and what parts needed even more evangelism.

There was one of the 15 interviews where the participant absolutely “got” it. He could have given the company’s pitch, including now and vision, with no problem at all. It was delightful. However, he was the ONLY one of the 15 participants who did that. The rest of the participants clearly struggled with some of the visionary aspects of the messaging. As luck would have it, that was the only interview that one of the project stakeholders listened in on. Unfortunately, during the report presentation he kept interrupting to talk about how the market “really got it.” We had to very strongly emphasize that the whole project needed to be considered – not just this one guy. The company had plenty of work to do to reach their entire customer base. They were not done.

Dealing with “Group Think”

by Diane Hagglund

Focus groups are ideal when you want to gain multiple perspectives in an interactive group setting.

However, one of the things that can happen in focus groups is “group think.” The first person starts off on a tangent – good or bad – and then the entire room goes along with that first opinion. This is a dynamic that every good researcher is aware of.

So does groupthink invalidate a focus group? Of course not, and in fact it can be very useful as long as you know it’s happening and stop it when necessary. Here are a few tips:

First, establish if group think is a good or a bad thing for your project. Depending on your goals, group think can be a very bad thing. For example, in message testing you’re usually evaluating the kinds of pain/benefit statements that someone experiences in isolation. Groupthink is very damaging in that kind of environment because you end up seeing only one participant’s uneducated, unguided reaction to messages. In this case, maybe focus groups are not the right medium – in-depth interviews or a Web survey may give you better results.
Remember that group think can be very good. If you’re trying to brainstorm an idea, having somebody to kick off the conversation and then see where the group goes with it is a very good thing. A smart moderator will encourage this kind of thinking for a while and see where it goes. The most creative ideas from focus groups I’ve been in did not come from one single individual. They came from the group members inspiring each other to build off of each other’s experiences and ideas and come up with something more powerful than any of the individual input. When one participants says “what if …”, and then the next participant says “cool idea, but how about this…” and then yet another participant says “or even better, you could take that idea and try…”. Now you’re cooking!
Be prepared to funnel groupthink. When you see it happening, a good moderator will challenge the group with the opposite assumption. Prompt some “group think” around the positive ideas, and then redirect the group to talk about the negative ideas. Insist that the group gives you multiple perspectives.

Above all – don’t be scared of negative group think! It’s very good to hear every possible objection your target audience can make about your solution BEFORE you start selling, so you can be prepared with the products, messages, and objection handling you need to be successful.

Dealing with Negative Feedback

by Diane Hagglund

Negative feedback is good for you.

Seriously.

While it’s very human to only want to hear how wonderful your solution is, you need to learn about any objections to your product NOW. Not later, when you have to deal with an entire slipped pipeline, because you didn’t realize there was a deal-breaker.

You need honest feedback, and you need it now.

But how do you ensure feedback that is real? Very few people, no matter how stupid they think your idea is, will actually tell you that. You have to create an environment that allows people to tell you about the downside.

One of the more painful exercises you can go through is to put a group of people together in a focus group and ask “what DON’T you like about this solution?” Once you ask them this question, and give them permission to be brutally honest, they WILL tell you about everything they don’t like. And the list of things they don’t like will be very long. And their list will inspire the other participants to think about even more things that they don’t like.

But as I said, this is GOOD FOR YOU! If you know about the objections, you can prepare for them in advance. In the long run, this will save you significant amounts of time and money.

How Many Market Research Participants Do I Need?

by Diane Hagglund

The answer is “enough to represent your market.” This number varies significantly according to the type of market research you’re conducting.

Web Surveys

For Web surveys this is a pretty straightforward question to answer. You will use quantitative methods to determine a sample size.

There are standard ways to calculate statistical validity, and a very easy-to-use calculator and descrition of the underlying statistics can be found here. You can use it to determine how many people you need to respond to a quantitative study in order to get results that reflect your target population.

You will need to know the population size of your audience – the total number of people in the group your sample represents. Even if you don’t know the exact population size, this is not really a problem. The mathematics of probability allows you to make a pretty good guess as long as you have some kind of basic idea. The number of participants needed is not linear. As your audience gets much larger, your sample size doesn’t increase very much.

For example, to get a result that is accurate 95% of the time within 5%, you need:

Population	Sample
100	80
1000	278
10,000	370

Qualitative Studies – In-depth Interviews and Focus Groups

But enough statistics homework for now. 🙂 The question that is less straightforward to answer, and so is asked all the time: “how many participants do I need for a qualitative study?” such as focus groups or a series of in-depth interviews.

The first thing to do when answering this question is to figure out segmentation. How many types of participants do you need to represent? This can include verticals, countries, roles, years of experience, customers vs. prospects vs. competitor customers vs. partners, and so on. The most common segmentation when working with Corporate IT is to have two groups: “technology decision-makers” and “economic buyers”.

Once you figure this out, and map any overlap between these areas (for example, partners may also be end-users), you’re ready to go.

In my experience, with corporate IT you need about 8-10 participants of each “type,” with a minimum of 10 participants, to produce a valid study. The only exception is studies with competitor’s customers, where you typically need more participants.

Note that most market research companies, including Dimensional Research, base pricing for qualitative work on the number of participants, so doing a good job right-sizing your project will give you the most bang for your buck.

Online Focus Groups: How Do They Work?

by Diane Hagglund

Online focus groups are a highly effective market research tool. My clients love them. They save the time and cost of travel while making it easier for participants and observers.

Most importantly, these benefits don’t require you to tradeoff for lower quality research. Online focus groups still facilitate the “brainstorming” dynamic that you get at an in-person focus group, where one participant’s comment sparks a comment by another participant and so on. The brainstorming dynamic enables the group to dig deep into an issue, so that the client can get a very clear picture of market acceptance of their product and of any issues they need to deal with.

Participants love online focus groups too, because they give them an opportunity to listen to the feedback of others in the industry. This is an important benefit for participants, especially in customer advisory boards, which encourages them to join.

Online focus groups are conducted using conference calls and Web-based meeting technology such as WebEx, GotoMeeting, etc. In many ways they’re similar to traditional face-to-face focus groups, except you’re not in one room. We recruit participants in the same way, set a time for the meeting, and prepare a moderator’s guide. If relevant, we can review a presentation with the participants. Just as in a traditional focus group, where the client can listen in through a one-way window, the client can listen to an online focus group on a muted phone line. And we can use Web technology to poll participants similarly to a visual “show of hands” or “nod of heads” in an in-person group.

Despite the many similarities between in-person focus groups and online focus groups, there are also a few differences. The most obvious one is that you don’t get the non-verbal clues from the participants. Because of this, the moderator needs to make an extra effort to draw out all the participants and balance the input. To facilitate this, we typically have a smaller group (4-6 participants per group rather than 6-8 for an in-person group) for the same amount of time. We may do more groups as a result – 5 instead of 4 for example.

Of course, there is also the option of conducting video online focus groups. As technology evolves, these are becoming more common and are easier to set up.

Another major benefit of online focus groups is that they reflect the majority of real-life interactions with customers. When selling to Corporate IT buyers, more and more meetings are being conducted online rather than in person, so online focus groups reflect that same scenario. If your messages work in an online focus group, they will translate well into an online sales call or a Webinar.

In-Depth Interviews, Focus Groups, or Both?

by Diane Hagglund

When doing qualitative research, we need to decide which is right for the client: in-depth interviews, focus groups, or maybe a combination of both. This depends on the client’s goals.

When are focus groups better?

Dimensional Research recommends focus groups when the client wants to gain multiple perspectives in an interactive group setting.

One of the main benefits of focus groups is that they get the participants brainstorming. When one participant’s comment feeds off of another comment and so on, the group can really dig deep into an issue. When trying to evaluate market acceptance, capture challenges and issues, or understand objections to new technologies or processes, the focus group dynamic is ideal.

Focus groups have another great benefit – the client can sit behind the glass or on a conference call and hear the direct, unfiltered feedback of a large number of participants with no distractions. Focus group sessions are also recorded for further observation. If your goal is to expose the maximum number of your team to direct input from the market, this is a very efficient way to do it.

When are interviews better?

In-depth, one-on-one interviews with technology professionals can be conducted in person or over the phone. These are appropriate when the client wants to identify detailed perceptions, opinions, beliefs, and attitudes.

In-depth interviews are particularly effective in the following scenarios:

When the client’s goal is to capture feedback on experiences that occur in an isolated way, such as product messaging or product usability testing.
When there is anything sensitive about the feedback that participants may not feel comfortable sharing in front of other people. They may be concerned a competitor is also be attending the focus group. Or in a win/loss analysis, a customer may not feel comfortable sharing details of poor account manager performance if they suspect the rep might be “behind the glass” and could hear him. The customer would feel much more comfortable sharing this information confidentially talking only with the researcher.

Two things that should NOT drive this decision are:

Travel – Sometimes the decision is driven by the geographical locations of participants. If you’re doing customer research with customers located in diverse locations or if you want global representation in the groups without the cost of travel, you still have a choice. The one-on-one nature of a phone interviews is an easy option, or for the group dynamic choose online focus groups.
Cost – An important consideration, of course, is cost. As a rule of thumb focus groups and interviews cost about the same per participant, so with the exception of travel – not a consideration for phone interviews or online focus groups – cost should not be the driving consideration in choosing the research approach.