« Comic-Con, Anyone? | Main | One sexy brew. »

Sunday, January 14, 2007

Do you do what you say you do? (psst: I don't)

Voicein_3Over at Group Review --yep, the Book Club discussions are still alive and kicking folks--one of the areas we're chatting is research, specifically the flaws of focus groups and how social media can help in "keeping it real".

In Citizen Marketers, Huba & McConnell refer to the "artificial reality" of focus groups based on a representative sample of the imagined target audience that give their opinions, get paid, go home...and move on with their lives.  Along the same lines in The Origin of Brands, Al & Laura Ries urge marketers to ask not "what will you do?" but only "what have you done?"

Armed with this advice I'm urging clients to move their research out of 2-way mirrored conference rooms and into the market. So I wanted to share some of the sound bites that have emerged from the likes of Gavin Heaton, Stephen Denny, Gianandrea Facchini, Valeria Maltoni and Laura Ries. Here's what they had to say on the flaws of focus groups...

Gavin Heaton points out the vast difference between Say and Do: "Focus groups are cut off from the real world that they claim to "represent" ... the environment is artificial, the questions posed are open to interpretation and manipulation, the the participants have to respond within very loose or imaginary contexts. And just because you SAY you would do something doesn't mean you will actually DO it."

Valeria Maltoni advocates conversations not surveys: "Conversation will allow us to develop a stronger bond with our customers; observation will keep everyone honest. I find that we (all of us) often say one thing, and do another."

Gianandrea Facchini urges us to go into the (super) market: "I believe that focus groups are limited by the human nature. I mean when ask for a question, every human being is trying to look better, smarter, cooler. What I used to do, in a very empiric way was to go into supermarket or selling point and look for people's attitudes. On field activity such this gives you more hints than an unreal situation as a focus group."

Stephen Denny encourages observing rather than asking: "Focus groups are excellent forums to raise questions but they are lousy at answering them. Observational research, anthropological research, anything where you don't ask, you watch -- this is where I'd spend more of my time and money if I could."

Laura Ries advises us to stop justifying: "Many times companies mainly use research to try and justify line-extensions. 'Oh, well our research shows that consumers will accept our spam brand expanding into the gum market. We make a great tasting spam, now they can have it in a convenient gum!' That said, it is very helpful to use research to take a reading of what your brand and your competitor's brand own in the mind of the consumer."

Are you marketers still using focus groups, or are you relying more on feedback channels like social media and observational/ethnographic studies? Do you do what you say you do? And, more to the point, do your million-dollar budgets rest on others doing what they say they will do...well, what say you?


Feed You can follow this conversation by subscribing to the comment feed for this post.

I participated in some focus groups, including some for General Motors, and derived a great deal of insight from them.

To overly depend on focus groups is to be avoided, but to dismiss them entirely is also a mistake.

I favor non-invasive, uncoached user observation testing for all products, but in a focus group, you can discover thoughts that are not expressed otherwise than in a relaxed, non-test, goal-free environment.

To ask people what they think of a color, an ad campaign, a headline, a slogan, and direct mail package, or a web site is a valid research technique.

Anonymous sponsor focus groups are where the subjects don't know who is conducting the research, just "a major player in the ______ industry" or whatever.

You can increase spontaneity and free association in focus groups in a variety of manners.

A major factor is how they are run, who oversees them, what is hoped to be achieved.

I don't see this as "are you using this OR that?"

Why can't we use a whole range of techniques, especially the ethnomethodlogist, deconstructive, and phenomenological?

*ethnomethodologist* (spelling correction)

Hiya Vaspers: A couple things to your good input (thanks):

-- I do like using groups to voice-in on areas such as logos/features/taglines and can see the value in that--so long as they're provided an array of choices. But I don't see the value in getting people to answer whether or not they would buy A, B or C as that's asking them what they would do.

-- Most of the co's I've been working with are still using the age-old focus group that does look ahead, not at history, and they use only that "research" to base multi-million dollar flops on. The issue is they're using an "either/or" approach as you aptly say.

--What I suggest is more co-creation (social media) and observational studies. Even these can be skewed but I believe far less so than asking people to look ahead. In the observational studies we're looking at the "now" and in social media feedback loops there's no pressure of looking/acting/sounding cool in front of a group.

At some point I'd love to hear more about your GM focus groups and how they were run. This area of research interests me a great deal.

Focus groups defeinitely have some major weaknesses, but they can still serve as one of many tools that should be used to get feedback and guide decisions. Online and social media feedback can be very valuable, although there can be limitations and biases that could color the accuracy of this feedback.

But I've seen how bogus focus groups can be. I have a friend who is a recruiter for focus group facilities. They are all about filling seats so they can collect their finders fee. Many of the people in focus groups lie outright -- or encouraged by recruiters to do so about the qualifying factors to be in the group. So if a marketer is looking for beer drinkers, or people who drive a Japanese car, business owners who use a certain bank, etc., there's a very good chance the people in your focus group don't fit the profile and are faking it. I tell you -- it's more commonplace than you;d like to think.

Similarly, feedback from social media might be colored by the fact that responders could be people who have a gripe or are more passionate than the masses about what you're researching.

It's tricky and decisions shouldn't be made solely on focus groups or input from online. Throw it into the mix, yes. But be sure there's a mix.

These are excellent observations. Both IDEO and Toyota advocate a "go and see" philosophy to discover user preferences. Focus groups aren't without their uses (and charms), but nothing substitutes for observing the consumers in their natural habitats.

Honestly, focus groups are great (really, some of my best friends are focus group people) at what they do. They get a small group of people together to talk about something moderated. They NEVER answer questions, though. Ever.

They are often used, as Laura says above, to validate a preconception, which is dangerous and self-serving. And they are always terribly biased. Once you see the standard eight-person play unfolding you can predict who's going to say what next with stunning regularity.

Please -- go out and read Jerry Zaltman's "What Customers Think". This and Paco Underhill's "Why We Buy" probably did more to change my research thinking than any other influences.

By the way, CK and Vaspers, I also did about 20 focus groups for GM (when I was at Onstar back ten years or so ago), all in about five days. Great insight and great feedback because we were raising quesitons and having people build on each other.

We conducted several focus groups a couple of years back before launching a new product brand. Our aim was to discover some of the thinking about that marketplace as viewed from a customer perspective.

The conversations were facilitated by a renowned researcher who conducted a very relaxed and apparently unstructured dialogue with the participants.

We remained anonymous and invited our sales and field teams for those areas to join us behind the scenes so we could extend our conversation about the groups' comments and feedback after each session. The reps knew some of the people invited to the focus groups -- and could validate that they were talking spontaneously, etc. So some observational feedback was built in the experience as well.

The knowledge and insights we developed from those sessions served us well throughout launch and even in the years following the launch.

What persuaded us to engage this tactic was the fact that we were being new entrants in an established marketplace for that specific line of products and going after some slice of the market held by an older yet loved product.

focus groups still do have to me some of an artificial flavour. i'm not saying "focus groups suck" nor that you don't have to run them.
but my point is that preparing for a focus group, are we sure that questions are neutral and not based on what we think about a product?
and could be this the reason why today we saw a lot of new product launch failing miserably and some other going in an unexpected way?
and again, could be the reason why companies are reluctant to take risk in approaching new products?
it seems to me that sitting in our comfortable office, we do miss the everyday day life of consumers.
and then when we are around as consumers, we discover the naked truth: do you remember about kohls?

Another tip/quote from me!
"Don't try to understand the consumer. BE the consumer".

Many of the top companies hire people who are in the target audience. This gives you a gut-feel insight, not a rational one. I call this "Consumer Empathy".

e.g. Nike hire active sports fans, and so they don't have to do consumer focus groups to test advertising. They do a lot of "immersive research" to understad trends, but not focus groups.

Another one is Harley Davidson riding with the Harley Owners Group:


I outsource my clients' research, and I look for research firms that understand a variety of ways and tools to get the information. There just isn't one right way, and I know you agree. Research is just another product/service that needs to understand the audience(s) and reach that audience(s) using every appropriate means.

S. Denny and Valeria nail it pretty good. I like how Valeria explained the proper functioning of a focus group.

Focus group abuses may be rampant, but so are web site, email, and telephone abuses.

A relaxed research technique, asking people what they think about a direct mail package, print ad, web design, etc. is a very fruitful avenue of investigation, when it's done correctly.

Focus subject selection is critical, as others have pointed out.

The comments to this entry are closed.