Mutli-Channel Attribution and Understanding Interaction

I'm no cosmologist, but this post is going to rely on a concept well known to astrophysicists, who often have something in common with today's marketers (as much as they might be loathe to admit it). So what is it that links marketing analytics to one of the coolest and most 'pure' sciences known to man?

I'll give you a hint: it has to do with such awesome topics as black holes, distant planets, and dark matter

The answer? It has to do with measuring the impacts of things that we can't actually see directly, but still make their presence felt. This is common practice for scientists who study the universe, and yet not nearly common enough among marketers and people who evaluate media spend and results. Like physicists, marketing analysis has progressed in stages, but we have the advantage of coming into a much more mature field, and thus avoiding the mistakes of earlier times.

Marketing analytics over the years and the assumptions created :

• Overall Business Results (i.e. revenue) : if good, marketing is working!
• Reach/Audience Measures (i.e. GRPs/TRPs) : more eyeballs = better marketing!
• Last-click Attribution (i.e. click conversions) : put more money into paid search!
• Path-based Attribution (i.e. weighted conversions) : I can track a linear path to purchase!
• Model-based Attribution (i.e. beta coefficients) : marketing is a complex web of influences!

So what does this last one mean, and how does it relate to space? When trying to find objects in the distant regions of the cosmos, scientists often rely on indirect means of locating and measuring their targets, because they can't be observed normally. For instance, we can't see planets orbiting distant stars even with our best telescopes. However, based on things like the bend in light emitted from a star, and the composition of gases detected, we can 'know' that there is a planet in orbit of a certain size and density, that is affecting the measurements that we would expect to get from that star in the absence of such a hypothetical planet. Similarly we don't see black holes, but we can detect a certain radiation signature that is created when gases under the immense gravitational force of the black hole give off x-rays.

This is basically what a good media mix/attribution model is attempting to do, and it's why regression models can work so well. You are trying to isolate the effect of a particular marketing channel or effort, not in a vacuum, but in the overall context of the consumer environment. I first remember seeing white papers about this mainly about measuring brand lift due to exposure to TV or display ads, but those were usually simple linear regression problems, connecting a single predictor variable to a response, or done as a chi-square style hypothesis test. But outside of a controlled experiment, this method simply won't give you an accurate picture of your marketing ecosystem that takes into account the whole customer journey.

As a marketer, you've surely been asked at some point "what's the ROI of x channel?" or "How many sales did x advertisement drive?" And perhaps, once upon a time, you would have been content to pull a quick conversion number out of your web analytics platform and call it a day. However, any company that does things this way isn't only going to get a completely incorrect (and therefore useless) answer, but they aren't really even asking the right question.

Modern marketing models tell us that channels can't be evaluated in isolation, even if you can make a substantially accurate attempt to isolate a specific channel's contribution to overall marketing outcomes in a particular holistic context.

Why does that last part matter? Because even if you can build a great model out of clean data that is highly predictive, all of the 'contribution' measuring that you are doing is dependent on the other variables.

So for example, if you determine that PPC is responsible for 15% of all conversions, Facebook is 9%, and email is 6%, and then back into an ROI value based on the cost of each channel and the value of the conversions, you still have to be very careful with what you do with that information. The nature of many common methods for predictive modeling is such that if your boss says, "Well, based on your model PPC has the best ROI and Facebook has the worst, so take the Facebook budget and put it into PPC" you have no reason to think that your results will improve, or change the way you assume.

Why not? Because hidden interactivity between channels is built into the models, so some of the value that PPC is providing in your initial model (as well as any error term), is based on the levels of Facebook activity that were measured during your sample period.

It's a subtle distinction, but an important one. If you truly want to have an accurate understanding of the real world that your marketing takes place in, be ready to do a few things:

1. Ask slightly different questions; look at overall marketing ROI with the current channel mix, and how each channel contributes, taking into account interaction
2. Use that information to make incremental changes to your budget allocations and marketing strategies, while continuously updating your models to make sure they still predict out-of-sample data accurately
3. If you are testing something across channels or running a new campaign, try adding it as a binary categorical variable to your model, or a split in your decision tree

Just remember, ROI is a top-level metric, and shouldn't necessarily be applied at the channel level the way that people are used to. Say this to your boss "The marketing ROI, given our current/recent marketing mix, is xxxxxxx, with relative attribution between the channels being yyyyyyy. Knowing that, I would recommend increasing/decreasing investment in channel (variable) A for a few weeks, which according to the model would increase conversions by Z, and then see if that prediction is accurate." Re-run the model, check assumptions, rinse, repeat.

Party System Has Outlived Its Usefulness

In sports, we are often taught that true fans root for (or against) the laundry, not the players.  A roster turns over, players get traded, retire, or leave through free agency.  It doesn’t matter who plays for the Red Sox, I am always going to want them to beat the Yankees.  I would be downright gleeful if they didn’t lose a game to New York for the next five years.  I have trouble understanding how Yankees fans date Red Sox fans, but I have seen it happen.

The problem is that this mentality has spilled over into politics.  You root for Democrats, or you root for Republicans, and the politicians themselves act as if the two parties were adversarial, rather than collaborative.  Imagine if your company’s divisions acted like this.  What if the Display team and the Search team couldn’t agree on how to divide the Digital budget for 2011, so they simply froze it so that neither side could get anything done.  Meanwhile, the client is stuck on the sidelines watching their products not being advertised.  That would last for about 10 seconds, until your agency were fired.

In theory, both parties belong to the same company, with the same goal: to ensure peace and prosperity for the citizens of the country.  The idea of a party isn’t strictly terrible, either.  A group of individual representatives coming together to achieve common goals and push forward a collective agenda makes sense, as long as it works.  Any organizational structure is the right one if it contributes to a functioning body, be it a government or a corporation.

The issue is that it isn’t working, and it has been slowly descending into paralysis for years as allegiance to parties became more important than individual contribution.  This is ultimately the crux of the biscuit, and what brings me to the most obvious solution:

Get rid of the useless categorization.

How about instead of choosing a team, voters actually elect individuals.  Vote for an official because you think he/she has integrity, or has good ideas, or is an able administrator.  Make every candidate run on their own merits, with their own funding.  Then, you will have a collection of bright, capable people whose views are actually somewhat representative of their constituents, and can work together unrestrained by party lines.

As it stands now, politicians have no choice but to adhere to party stricture, if they want to have a chance of winning/keeping their post.  You aren’t likely to replace an incumbent without the financial and PR backing of a party.  If you won’t toe a line, they will nominate someone who will.  The laundry is what matters in the election process, not the player.  If you put the party war chest and PR machine behind basically anyone, they become a viable candidate.

Ultimately though, this is a matter of practicality.  Would I love to live in a country where we just elected the best 100 individuals to be Senators, locked them in a room and said “solve some problems?”  Hell yes.  Do I think it will happen?  Of course not.  But what they are doing right now, isn’t working.

We live in a results-driven world.  When your job is to legislate, and set budget allocations so that the government can do its job, that’s what the people expect you to do.  If the current system isn’t working, it is completely unacceptable to stubbornly keep on doing the same old thing.  When things don’t work, you have to identify the problem, and then come up with a solution that involves doing something different.  In my industry, and I suspect most industries, “test & learn” is a phrase you hear a lot of.  If your results are satisfactory, develop a theory, and run with it.  If it doesn’t work you stop, but at least you a.) gained some valuable information about what doesn’t work, and b.) YOU DID SOMETHING. 

Doing nothing isn’t a solution, and neither party can sell the current service that they are providing to the “consumer” (in this case Americans) with any degree of honesty.  I would rather see our politicians try something to fix the economy and see if fail, than see them refuse to do anything because it’s either a potential liability for them if it fails, or a potential win for the other party if it succeeds.

You (I am shifting to speaking directly to the politicians, because it adds narrative force) aren’t doing your jobs right now, period, and a lot of people depend on you.  We as a people are sick of excuses, and finger-pointing, and empty rhetoric.  Just show us that you are even trying to help.  Pretend for a moment that you aren’t athletes playing in the big Republicans-Democrats rivalry game, and that you actually care about the people you supposedly govern for.  Make sacrifices, think outside of the box, don’t factor the 2012 elections into your decision making.

Can’t figure out how to compromise on budgets?  Here’s an idea:  Every Senator takes the 2011 budget and has to make a 1% change somewhere, in any program.  He writes it on a slip of paper anonymously, and hands it in.  That’s it.  If you personally hate Medicare, you can choose to take 1% away there.  If you feel that the Pentagon budget is too high, take 1% away there.  If you think that infrastructure is a pressing need, add your 1% to the Highway Department.  By making it anonymous, everyone will feel free to actually do what they think is best for the country, not their party.  By making it an aggregate, naturally, less popular programs will lose money and more popular ones will gain.  Chances are the final result won’t make everyone happy, but no one will be able to say they didn’t have a hand in it.  As Calvin & Hobbes taught us, “A good compromise leaves everyone unhappy.”

Is that idea unreasonable?  How about this:  TAKE TURNS.  We can learn from children on this one.  You don’t think the Democratic plan is a good one?  Fine, that’s an opinion you can have.  But let it pass, give it 6-12 months, and see what happens.  If it doesn’t work, do a post-mortem and try to figure out why, take some lessons from it, and then give the Republicans a shot.  Keep taking turns trying solutions until you hit on a fix, if you aren’t able to work together on one.

We recognize that this is a tough situation, and that you guys have an uphill battle in front of you, but give us a little credit.  We can forgive mistakes, and we can forgive failures, but inaction is unforgivable.