|
|
MORE ON MARKETING RESEARCH BEST PRACTICES AS SEEN IN MARKETING RESEARCH
Surviving
Death Wish Research
Sometimes marketing research is its own worst enemy
By Kevin J. Clancy and Peter C. Krieg, Winter 2002
Executive Briefing
Marketing research firms are in a slump these days, but that doesn't
mean research isn't important. It just means researchers have to step
away from dangerous "death wish" techniques—over-reliance
on focus groups, poorly planned segmentation studies, and quick but often
useless phone and Web surveys. Business as usual won't cut it anymore.
Instead, researchers need to adopt a counterintuitive approach to get
around some of the common but perilous traps that can sabotage marketing
efforts.
At its annual conference
in 1999, the Advertising Research Foundation (ARF) announced that corporate
management was not satisfied with current research. A study reported that
CEOs were insisting on more accountability, but put less faith in marketing
research than in most of their other sources of information. Management
speakers from Sony, General Motors, AT&T, and Young & Rubicam
said that research had been most disappointing in predicting new product
success and in measuring advertising's true effectiveness.
These insights into executive opinion came during the go-go days of the
late '90s, when we were swept up in the momentum of unprecedented economic
growth. We didn't have time to concern ourselves with dissatisfaction
and distrust of the industry—we had project deadlines to meet! So
long as companies—in particular dot-coms—continued to buy projects
at the extraordinary pace they were at the time, we could afford to continue
on with business as usual.
These days, as the world economy continues to slow, we're confronted with
a much different situation. As recently reported in Marketing News,
marketing research firms are facing the worst industry slump since the
early '90s. According to a study conducted by Inside Research,
a Barrington, Ill.-based industry newsletter, at the end of the first
quarter in 2001, two-thirds of research firms reported flat or declining
revenues compared to the first quarter in 2000, with field service organizations
also showing dramatic declines. By the start of the fourth quarter of
2001, the slump had turned into the unspeakable—a recession.
The Proof Is in
the Pudding
Conventional wisdom would say the practice of marketing is doing just
fine; yet the performance of marketing programs would indicate quite the
opposite is true. For more than a decade, we have collected data on the
performance of marketing programs for consumer and B2B products and services
across a broad range of product categories and types of marketing programs.
Our experience is not unlike that described by Andrew Ehrenberg in his
article, "Marketing: Romantic or Realist," in the Summer 2001
issue of Marketing Research. Whether we look at market share growth,
new product success rates, advertising ROI, customer satisfaction, or
brand equity, we see little to feel good about.
The majority of companies suffer from the following problems:
- Market share growth
is in a modest decline.
- Ninety percent
of new products fail.
- Advertising ROI
hovers at 1% to 4%.
- Customer satisfaction
drifts between 70% and 79%—a "C" grade.
- Brand equity—the
perceived value of the brand—is in decline.
A vast majority(more
than 80% of marketing programs) just barely break even and do little to
improve either the financial or competitive position of companies and
their brands.
As a direct result of poor marketing performance, more brands have become
commodities than commodities have become brands. In
The
Commoditization of Brands and Its Implication for Marketers, a recent
study done by Copernicus and Market Facts, in only four of the 51 product
and service categories studied were leading brands becoming more differentiated
over time. Ninety percent of categories were declining in differentiation,
with banks, bookstores, bottled water, catalogue clothiers, credit cards,
discount stores, and fast food restaurants leading the pack in becoming
more similar and having the least brand differentiation.
The study also found that consumers view low price as more important than
brand name in 28 out of 37 product categories, particularly when selecting
bookstores, bottled water, gas stations, office supply stores, pet supply
stores, and rental cars.
Others inside and outside of marketing have taken notice of marketing's
performance and its subsequent effects on brands as well. During the recession
of the early '90s, McKinsey & Co. wrote, "Doubts are surfacing
about the very basis of contemporary marketing: the value of ever more
costly brand advertising, which often dwells on seemingly irrelevant points
of difference; of promotions, which are often just a fancy name for price
cutting; and of large marketing departments, which, far from being an
asset, are often a millstone around an organization's neck."
More recently, in an issue dedicated to the subject of brands, The
Economist asserted, "While consumers have changed beyond recognition,
marketing has not," and quoted the CEO of Customer Strategies Worldwide,
Elliott Ettenberg, who stated, "Everything else has been reinvented—distribution,
new product development, the supply chain. But marketing is stuck in the
past."
In the face of increasing competition and financial pressures to perform,
marketing executives for the most part continue to rely on a seditious
combination of outdated beliefs and convictions—what we call marketing
mythology. This includes instinct and observations about competitors to
make mission-critical, multimillion-dollar decisions in a hurry. Senior
management hands marketing managers a set of revenue objectives and deadlines—usually
driven more by senior management's perception of what the company needs
to do rather than what's realistic or even possible—and marketers
are forced to work with the objectives and timeline they've been given.
Feeling pressed for time, these managers typically ask researchers to
run a few focus groups, make 100 telephone calls to test a concept, or
undertake one of the many other popular conventional techniques we refer
to as "death wish" research. These techniques seem reasonable
to the time-challenged because they're quick, low-cost, and often corroborate
what the marketer already thought.
They may take less time and cost less money, but death wish research techniques
offer little in the way of value. What companies usually get is more misinformation
than information, which then contributes to the failure of marketing programs.
As a result, not surprisingly, the confidence executives have in marketing
research has declined.
We as marketing researchers are by no means innocent in this situation.
After all, many of us are offering and conducting death wish research
to win and satisfy clients, thereby hammering yet another nail into our
own coffins. In order to rebuild confidence and the perceived value of
marketing research, we must acknowledge the reckless use of bad research
and put an end to it once and for all!
Obsessing Over
Focus Groups
Focus groups remain the most popular tool for supporting practically every
marketing decision, and companies seem completely fixated on using them.
They use them to understand the problems of their target, to explore alternative
positionings, to get reactions to advertising campaigns and new product
concepts, even to chart alternative pricing strategies. Groups consisting
of 18 to 49-year-olds, physicians, retail store managers, corporate purchasing
agents, college professors, technology mavens, "cool" kids,
and even animals are formed. One of the hot new "innovations"
in focus groups is to hypnotize respondents so we can find out what they're
really thinking. Focus groups are often a helpful first step in a serious
research process, but when the first step is the only step, marketing
is in trouble.
John F. Sherry and Robert V. Kozinets, both of the Kellogg School of Management
at Northwestern University, recently wrote that focus groups are "the
most overused and misused arrow in the qualitative quiver. Focus groups
often provide the illusion of human contact and the occasion of pyrotechnics
that efficiently satisfy the prematurely narrowed imagination of clients
and researchers behind the one-way glass." While focus groups reveal
subjective information about a brand and its potential, they don't provide
much beyond a superficial understanding.
Dozens of irrational dynamics skew what participants say. One man likes
the sound of his voice and pontificates. A women wonders whether she's
giving the "right" answer. Another feels uncomfortable revealing
her true feelings (if only about orange juice) or lack of knowledge (about
personal digital assistants) in front of strangers. Still another has
no opinion about credit cards, but feels obligated to contribute something.
About focus groups, Richard Grinchunas, president, and Tony Sicilian,
vice president, of A&G Research Inc., wrote in Marketing News,
"There are no facts. There are only verbatims." Focus groups
are to serious research what bumper stickers are to philosophy—they're
poor substitutes for more rigorous methods.
The Three-Minute
Segmentation Study
An over-reliance on qualitative research is not the only research culprit
leading marketing astray. One tool we see emerging is something we call
the "three-minute segmentation study" and will explain in it
in the tradition of the popular children's book, What Good Luck, What
Bad Luck.
What good luck! For decades, most marketers realized that focusing on
subsets of buyers is the most efficient way to develop a marketing program.
But what bad luck, there are literally hundreds of thousands of possible
ways to segment a market.
What good luck, a company's top management realizes they need to focus
their resources…. But what bad luck. They don't know the best way
to divide their market into segments or which target group is the best.
What good luck. A constellation of marketing research firms offer segmentation
studies. Top management contacts these firms and learns that—given
the size of the category, the size of the corporation's business, and
the importance of the targeting decision—it's critically important
they do the segmentation right.
But what bad luck. The company finds out, to do a market segmentation
correctly—in other words, to find segments competitors don't even
know exist and the most profitable target group—requires a large-scale
investigation. It may involve 1,000 or more personal interviews, 45 to
90 minutes long. It could require measuring psychographics, sociographics,
demographics, and lifestyles. To do it right will take a variety of statistical
and neural network algorithms to analyze the data. It needs three to five
months from start to finish and costs $200,000 to $1.2 million, depending
on the consulting firm and the scope of the project.
Keep in mind that most marketers feel they absolutely must make a targeting
decision as soon as possible. In fact, they needed to decide yesterday.
And anyway, if there's a spare half-million in the budget, they aren't
going to spend it on marketing research. They need to spend it to send
the management team to the Super Bowl to see the company's promotion first
hand. But that's another story.
What good luck. They can do a three-minute segmentation study! The three-minute
study consists of hypothesizing which three or four variables out of the
thousands possible might be best to segment the market and interviewing
150 to 300 buyers in the category, almost always by telephone or over
the Internet. They like it because it's fast and cheap.
But what bad luck. The odds that the segments produced from a three-minute
study actually reflect the company's best opportunities are virtually
zero. The analysis done is simple—primarily cross tabulations—to
say something as basic as "15% of the buyers in this category account
for 85% of the volume" or "25 to 54 year-old-women account for
78% of the sales in this category."
And more bad luck. The company uses the results of the three-minute segmentation
study, the dot-com crashes, and the product tanks.
Unlike the children's book, this story typically doesn't have a happy
ending. If there's a happy ending—the marketing programs hit the
mark—it's only by luck; being in the proverbial right place at the
right time. The three-minute segmentation study simply does not represent
a robust enough data set or rigorous enough analysis to provide the kind
of guidance marketers need to most efficiently and effectively spend their
marketing dollars.
Getting the Wrong
Answers
New product testing over the telephone and Web surveys have become increasingly
popular forms of death wish research. They're simple, they're cheap, and
they're very dangerous.
Ordinarily in concept testing, a respondent gets to see a written concept,
a one- or two-paragraph description of the concept idea that typically
contains the product promise (e.g., "tastes better") and the
reason (e.g., "made with all fresh, natural ingredients"). Sometimes
the concept comes in the form of an advertisement—a television commercial
people can see and hear or a print ad they can see and read.
Following the presentation of the concept in whatever form, the researchers
will ask people a series of questions that predict buyer behavior. These
questions include 5-point, 7-point, and sometimes 11-point rating scales;
the researcher gives a respondent the rating scale and a written label
or description of each point on a card to read before they answer.
Meanwhile in death wish concept testing, researchers read a description
of the concept to people over the telephone. Because they're reading and
because respondents forget the first sentence by the time they hear the
second, researchers must shorten the concept and distill it down to its
bare bones. The bare bones sometimes can be as short as a sentence or
even a phrase. They do that even though there is no clear analogue to
this telephone reading in the real world-except perhaps a 10-second radio
spot. The distilled concept is then coupled with rating scales with typically
few points (i.e., less discrimination) than would be employed in a personal
interview or Web-based survey.
In the real world, the customer watches television or reads a print ad
or looks at the package and digests the words, sounds, shapes, colors,
and whatever descriptive information is offered. Somebody hearing the
concept over the telephone loses almost everything. In a personal interview,
the respondent can actually look at the ratings scale and a visual description
of the product. He or she can contemplate, think about it, and decide
what to do. On the phone the respondent has to remember the rating scale
and give this stranger a number.
The entertainment industry is infamous for using this technique. The most
popular way for studios to test a new movie concept is to read a two-sentence
description to people over the phone and then ask whether they would be
likely to see the movie. This method is far removed from deciding to see
a movie based on watching a trailer, reading a review, or seeing an ad.
That's why it can't possibly work! For many movie studios, the decision
to green light a new movie is akin to a night in Las Vegas. They roll
the dice, take a chance, and watch nine out of 10 films flop at the box
office.
Though quick, inexpensive, and more visual than phone interviews, the
results of Web-based surveys can be unreliable. First of all, Internet
users tend to be younger and better educated than the population at large.
Great if you are a sophisticated, tech-based business looking to solve
a problem. But if you need to reach a different type of target market,
the Web won't necessarily help you. You might as well interview aliens.
Response rates to Web surveys also tend to be very low. Relying on information
provided by a Web survey with a 1% response rate is absolute lunacy and
gets you about as close to a real, actionable answer as reading fortune
cookies gets you to spiritual enlightenment.
Commonsensical
Research Can Be Dangerous
Not surprisingly, senior executives and marketing managers often don't
understand what research people are talking about. They hear the words,
"standard deviation," "logit regression," "neural
network," "statistically significant," or some of the other
jargon all researchers—including us—are guilty of using, and
they think, "I have no idea what these guys are saying, but it sure
sounds good." Very few managers—particularly in front of their
bosses—will ask for a definition or clarification. It's just human
nature to hide what we don't know.
Under these circumstances, it's easy to see why using death wish research
tools has become so prevalent. When a marketing manager has only a vague
comprehension of what a researcher will do, but it's fast, cheap, and
they hear their competitors have done it or will do it, they'll move ahead
with the project. Or maybe it's the company's policy to do research, so
they move ahead out of practice rather than in the pursuit of knowledge.
Whatever the motivation, the result is often the same. Senior managers
might feel happy and satisfied with the research at first. "We've
got the numbers. We've got the research!" But a few months later
they wonder what they got, and about a year later they realize, "Gee,
I got nothing from that one," and the fat research report becomes
a doorstop and, as is becoming increasingly clear, the research budget
becomes vulnerable.
Surviving Death
Wish Research
William Jennings Bryan once said, "Destiny is not a matter of chance,
it is a matter of choice; it is not a thing to be waited for, but a thing
to be achieved." Marketing researchers need to take control of our
own destiny and our place in the marketing organizations of companies
around the world. Business as usual isn't going to work this time, and
we need to do our part in moving the practice of marketing out of the
past and into the present.
What can individual researchers do to survive conventional research and
rebuild the value of their profession?
Focus on targeting and positioning. Philip Kotler says,
"If you nail targeting and positioning, everything else will follow."
Don't fall into the trap of picking a target in nanoseconds (e.g., as
with 93% of American brands) with no discernible positioning at all. "Rigorous
analysis of unimpeachable data" should be your mantra as you work
hard to be sure you've found the financially optimal target and a uniquely
compelling positioning.
Open the windows and get out of the box-design research creatively.
Make sure that it covers "out-of-the-box" concepts, product/service
attributes and benefits, and eventually analysis-stuff that's different
than anything currently used in category. As my mom used to say, "If
all you do is what you've done, all you'll get is what you got."
And that's not good enough!
Take the time to get it right. Rarely is speed the most
important concern for marketers, even though they may think and act as
if it is. Yes, there are some technology businesses that change at warp
speed, so speed of marketing research is of the essence. But in most industries
and for most decision areas, things change very slowly. It's more important
to do it right the first time than to keep doing it over and over again.
Drop the jargon. While it may impress our friends and colleagues,
research jargon confuses those not "in the know" and leads to
questions about what exactly the research is providing. Define terms for
both the technically and non-technically inclined, not only in terms of
the process (i.e., data collection techniques, formulas, modeling), but
also in terms of the type of information the analysis will provide.
Quantify the ROI of different research approaches. Take
a typical $20 million TV campaign, for instance. The average cost to produce
one finished 30-second commercial is $320,000, but only about $25,000
apiece to produce an animatic or photomatic—a rough version of a
commercial—and $20,000 for a research firm to test it. Two commercials
cost $90,000 in creative and research; four commercials, $180,000. Rather
than risking $320,000 on one execution that will most likely return 1%
to 4% (the ROI of most advertising campaigns), why not spend $500,000
($320,000 + $180,000) to improve the probability of choosing the execution
that will return 20% ROI, or $4 million? Presenting research choices in
terms of the greater profit potential gives marketers quantified information
they can use to justify a decision to senior management.
Focus on research innovations that truly save time rather than cut
corners. Many researchers have focused R&D efforts on developing
faster data collection techniques, often through the Internet. On the
surface, some new techniques appear faster, but a deeper look reveals
the increase in speed is the result of cutting a few corners. The result
is less representativeness and lower response rates. While the Internet
and other technologies certainly offer opportunities for overcoming many
of the impediments to quick data collection, such as distance, incidence,
and cost constraints, true innovations should preserve the integrity of
data rather than sacrifice it for speed.
Marketing research poses many dangers and even more opportunities. Bad
research can, and often does, lead companies in the wrong direction. Coca-Cola,
for instance, researched Surge in the '90s and New Coke in the '80s using
death wish techniques, and look what happened! Good research, on the other
hand, is the sine qua non of a counterintuitive approach to great
marketing.
Back
to News Room Index
|
