The Value of Knowing

One of the most common questions we get on marketing measurement begins with "How would you measure ...?"

When I hear those trigger words, my mind immediately goes to the response question: "What would you do with the information if you had it?"

There are two reasons the response question is so important prior to answering the original question.

First, the answer to the response question will help me understand the extent to which your organization has developed a thoughtful (if executionally challenged) perspective on critical metrics, or if you're still in the mode of identifying the superset of all possible things that might be measured.

Second, the answer will tell me the relative importance of the particular piece of information you're looking for and give me some sense of the economic value of having better certainty of knowledge. If having the knowledge will improve your business outcomes just marginally or not at all, we can pretty quickly agree that it doesn't really matter how we'd measure it. Conversely, if the expected economic value of knowing is significant, many possible doors open in terms of collection avenues, since we can presumably allocate a fair amount of resources to acquiring the knowledge and still show a very positive return on that investment.

Third (yes, I know I said there were only two, but it's a blog so cut me some slack), if your answer to the response question doesn't properly anticipate how having the knowledge would impact current decision processes, it's a sign that we need to lay some organizational groundwork before we even ask for the resources to go get the knowledge.

As you might imagine, most people are initially stumped by the response question. But if you think about your hairiest, most formidable measurement challenges in the context of the economic value of knowing, you really begin to define your priorities for knowledge aggregation.

Everything can be reliably measured -- somehow. The critical parameters (as in most business pursuits) are how much time, money, and political capital you're prepared to spend to acquire the knowledge. You can't possibly know what your tolerances are until you have some clarity on the value of knowing.

Measure What You SHOULD, Not Just What You CAN

In the midst of a dashboard planning session last week, the VP Marketing Intelligence brought the meeting to an abrupt halt by saying, "This is all nice in theory, but we don't have the data to measure half of these things."

Hmmm. Good point. Very pragmatic. Or at least that was the initial reaction of most of his teammates in the room.

But let's think for a minute about the implication of only measuring what we have data for.

1. In all likelihood, we don't have much insight into the data streams we have today, or we wouldn't be talking about assembling a dashboard in the first place.
2. The "spotty" data we have today leaves significant gaps between what we know, what we don't know, and most importantly, what we don't know we don't know.
3. Keeping our dashboard framed within the parameters of what we already have data for is a sure fire way to reinforce every preconceived notion we have about the business.

I'm all for pragmatism. Nobody is helped by a theoretical marketing dashboard. But the very process of planning a dashboard is intended to draw out all of our structured knowledge, scientific hypotheses, and experiential best-guesses about what happens to sales or profits when we add/change/delete marketing investments. Only by looking at the business from the perspective of "what should we be measuring" and setting the framework for a truly comprehensive view of effectiveness and efficiency can we really assess what we know and where we should prioritize our search for more knowledge.

Fact: Most marketing organizations spend far too much time and precious resources answering questions that don't generate any significant insights into the business. Laying out the complete picture of what you think you need to know first is the best way to keep your marketing measurement efforts from returning the same old knowledge with the same critical insight gaps.

It makes better sense to start with what you want to know, prioritize the pursuit of the unknowns on the basis of expected insight value, and fill in the gaps in your dashboard over time. But let everyone see the gaps as a reminder of how little we actually do know and a reassurance that we, the marketers, are diligently working to try to close those gaps. It will make them feel better about our search for objective insight.

Peter Drucker – Our Thanks

Peter Drucker, perhaps the greatest business academic ever, died on November 11 at age 95 in Claremont, California. Many know him for developing concepts like knowledge workers and decentralized management. We at MarketingNPV thank him for two insights that have an impact on every minute of our working days.

The first is management by objectives. The concept of setting objectives and allowing teams to work toward them is now commonplace, but before Drucker, command and control reigned. We at MarketingNPV cannot imagine doing our jobs without the discipline of MBO already in place. In fact, measurement of progress towards marketing objectives is much of what we do.

The second great insight we use every day to help clarify marketing’s role within a company. Drucker wrote, "Business, because its function is to create and sustain a customer, has only two purposes: Marketing and Innovation. Everything else is an expense." Many well-established companies undervalue both elements because they are living of the franchises created by earlier marketing and innovation. Success often boils down to how well the company tends its brands and customer franchise. And without measurement, marketers are hobbled in their ability to make the most of the assets under their care.

Pete Drucker may no longer be with us, but his work lives on in almost every business person every day.

5 Magic Metrics

I recently had a conversation with the CMO of a leading global technology company in which they described for me their desire to construct a marketing dashboard focused on the “5 most important metrics” for their business. When I asked her what those might be, she quickly began to list the possibilities – around 10 of them as I recall – before she stopped and admitted that the task might not be so easy.

The more we talked, the more clear it became to her that getting down to the 5 Magic Metrics would take a diligent effort of experimentation and elimination, perhaps starting with 40 or 50. The appropriate metaphor was the old story of “I’m sorry this letter is so long, I didn’t have time to write a shorter one.” The risk of jumping too fast to the logical 5 is that you might select the wrong ones and achieve the wrong goals in a very efficient and effective way.

If you want to get down to the right 5 (or 4 or 6 or however many) metrics that really forecast success, you owe it to yourself (and your CEO) to undertake a thorough exploration of the 30 or 40 hypotheses that would emerge from a cross-functional assessment of “what really drives the business.” You’d probably not be surprised at the lack of consensus within even the best-managed companies on which 30 to even start with.

From there, it takes a bit of effort to acquire the data to test each of them for diagnostic and predictive ability, or to develop a proxy approach for the inevitable majority of metrics where the data doesn’t exist. Not that it can’t be done quickly (read: a few months), but it does require a deliberate effort.

So whenever I get the CMO request for “the 5 magic metrics,” I agree with them that it’s a great idea to strive for simplicity and to align your marketing measurement framework or dashboard to reinforce the company’s specific goals. But I also advise them to be careful about how they issue that direction to their teams, lest they create the impression that they’re only interested in simplicity (which might be interpreted as superficiality), or they send the message that speed is more important than accuracy. They’re both important.

Start with a hypothesis on what the 5 key metrics might be on the highest level on your dashboard, but don’t sacrifice the real insight derived from exploring the broader spectrum of options and validating your hypotheses. The difference will be measured in the credibility and longevity of your measurement plan.

Marketers Turning to MOM

There’s a new community emerging within the world of marketing called marketing operations management, “markops” for short, MOM for shorter. Most of the markops people I’ve spoken with so far describe their roles as identifying opportunities for process improvement, and generally getting data and information from point A to point B in a marketing context. There’s great emphasis on improving the customer experience, capitalizing on lead generation, and generally promoting efficiencies.

Importantly, there seems to be a strong understanding amongst markops types that technology is best applied to automate sound business processes and to improve the suboptimal ones constrained by the limits of human processing speed, volume, or accuracy. It’s refreshing to note that this new breed of marketer is pushing automation and technology NOT for the sake of technology or job security (although they do seem to take the measure of one another through subtle clues inherent in the answer to “Which MOM platform are you running?”), but rather in the context of process improvement.

While a few of the markops folks I’ve met have been imported into the marketing department from IT or operations, most seem to share marketing or brand management DNA -- which makes them uniquely capable of envisioning the desirable outcomes of process improvement, not just the process of improvement itself. A high percentage of them are Six-Sigma-trained -- even if their current employer isn’t a Six-Sigma company. Many of the initiatives they’re undertaking are targeted at goals like more efficient e-mail marketing, Web site customization, customer datamart assembly, and integrated campaign optimization. There’s even some discussion of ROI -- albeit mostly still in the context of paying for the investments in the technology.

I think this is a very positive trend for marketing measurement and accountability. Focusing on process and information flows will accelerate the appetite for reliable measurement structures. My hypothesis is that the more tactical focus of the markops function of today will evolve into a more strategic one as the low-hanging fruit of process improvement is picked and the organizational confidence in them grows. In the future, I would expect their unique perspective within the department to translate into leading roles in architecting marketing measurement platforms. Provided, that is, that they can maintain direct access to the CMO, and that they are appropriately skilled in continuously reinventing their job descriptions to consolidate past successes and build bridges across functional groups within the marketing department. But that’s a message that the CMO needs to hear too.

If you’re in marketing operations today, I’d like to hear your perspective on the challenges and opportunities.

Brand Measurement Mayhem

Why are there so many ways to measure brand equity? Probably because there is no “right” way to do it.

Many billions of dollars are spent in this country researching and tracking brand equity, most of it through approaches that attempt to carefully dissect the individual image attributes, emotional connections, and perceptions of our companies, brands, or sub-brands. But so little of it is done in a way that inspires confidence amongst CEOs or CFOs that if we increased our ratings on “trustworthy” or “innovative,” we’d see significant improvements in financial results.

Perhaps because there are so many opinions and methodologies about how to correlate changes in key brand equity components to financial outcomes, the lack of consensus, epitomized by multiple vendors extolling their unique, proprietary systems, is possibly ENCOURAGING CFOs to believe that it's all just marketing babble and underscoring the soft, unpredictable nature of it.

Here’s a thought … What if we invited all purveyors of brand equity measurement processes to present their approach and case studies to an independent panel of financial executives, Wall Street analysts, and academics? The panel would then judge the merits of each approach in a fully informed context and propose standards that incorporated best-of-breed methods in a variety of “classes” aligned to the needs of different industry group dynamics — retail, financial services, packaged goods, electronics, automotive, etc.

This approach might actually help close the gap between the marketers and the financial community, moving one or the other towards a better understanding of the inherent challenges of the task and building a better framework for measurement evolution.

I’m not sure the research companies would line up to participate. Ad agencies might hate the idea too.

What do you think?

The Mother of All Models

No, we’re not talking about a high-cheekboned, exceptionally fertile female.

What I’m referring to is a tendency I see quite often in corporate America to “crack the code” on marketing measurement by building the world’s biggest regression model.

While admirable in their pursuit, companies that seek to answer the question “What are we getting in return for our marketing investment?” with a number, i.e., “41% ROI,” are headed into a long, dark alley with a penlight. Their chances of getting to a numerical answer with any high degree of confidence is about the same as their chances of finding a specific grain of sand on the ground in that alley.

Along the way, wonderful things are learned about correlations between marketing stimulus programs and business outcomes, most of it negative. In fact, the real value that model-seekers derive is a much higher level of clarity on what doesn’t work. So there is some value to pursuing it. But outside of some packaged goods categories with clearly defined and mature purchase patterns and competitive environments, this approach rarely results in anything close to a perfect prediction of economic outcome in relationship to changes in marketing investments.

The real question is, what is the real question? You see, if you’re trying to ascertain the optimal level of advertising spend to maximize either short-term profitability or ROI, you certainly can build some effective analytical approaches to get to a reasonably small range of uncertainty (a.k.a. high degree of confidence) in an answer. Marketing-mix models can be quite helpful in answering this and similar questions.

But if you’re trying to answer the more common CEO question, “What would happen if I spent twice as much on marketing or half as much?” the answer tends to elude the power of pure analytics absent years of detailed transactional data and previously determined influences of external variables like interest rates, housing starts, demographic mobility, etc.

The bigger the question, the more likely you are to need to a comprehensive combination of marketing metrics to assemble the preponderance of evidence, like a marketing dashboard. Using insights you gain from analytics plus test/control experiments plus research plus some structured forecasting techniques, a marketing dashboard helps focus the company on what it knows and, by definition, what it doesn’t know. Over time, the key questions are identified, researched, and answered.

In short, if your inclination is to try to tackle broad questions of marketing measurement through advanced modeling techniques, you’ve only got part of the solution. Using the full set of tools at your disposal to complement your analytics will enhance your overall ability to answer the really hard measurement questions with greater credibility.

If you’ve had any experience with a modeling-centric approach to marketing measurement, please share it with the rest of us.

Never Again

With many of you in the throes of 2006 planning and others just emerging from it, now might be a good time to take the pledge of planning sanity for next year. Repeat after me …

“I will NEVER AGAIN go through such a convoluted, sloppy, opinion-driven process of requesting marketing resources and subjecting myself to the eye-rolling, sighing, smug resistance of finance. Next year, I WILL have more factual insight and disciplined measurement structure on my side.”

Great! Now it’s time to get started. Right now.

Not because the pain is still fresh and the motivation high (although that might actually help), but because it will take you a year to develop, validate, and properly stakeholder an effective marketing measurement process. So if you start now, you might just have something in place that makes the process much more collaborative and rational in the fall of ’06.

Some of you might be asking, “Why does it take so long?” The answer is that no effective marketing measurement system is a simple mathematical exercise. The issues are complex; the necessary information and data flows are rarely in place at the start; and there WILL be a number of assumptions that need to be made in the absence of a perfectly reliable algorithm. Not to mention that there may not even be any clear alignment across the organization on what the role of marketing is and what would constitute the definition of “success” in measuring marketing effectiveness.

On the other hand, others out there may be thinking that it’s ambitious to expect to build a comprehensive measurement structure or marketing dashboard in just nine months or so. Perhaps. But if you start with alignment amongst key decision makers on exactly what you’re trying to measure, you can begin to build and deploy your measurement process in stages — bringing modules into play as they are developed and instilling a greater sense of confidence and anticipation as you go. Don’t underestimate the credibility to be gained from just scoping the task and beginning to show progress.

So if the planning process has gotten the best of you this year, start now to ensure you get on a more level playing field next year. Once you’ve sized and scoped what it will take to really create a credible measurement process, you’ll be glad you started early.

Anyone have any experience to share on how long it took them to get a workable measurement framework in place, or how long they’ve been working on it so far?