New Mind-Reading Tools Predict Consumer Behavior

Zach Brooke
Marketing News
Current average rating    
Key Takeaways

​​W​hat? New research by Dr. Carl Marci, chief neuroscientist at Nielsen Consumer Neuroscience at Nielsen Company, combines several existing mind-tracking tools into one measure.

So what? The new measure captures up to 77% explanatory power with in-store sales, more than any other tool previously available to marketers.

Now what? Consider neuroscience tools like Video Ad Explorer to strip down and enhance your content and directedly impact in-store sales. 

Jan. 1, 2017

By stacking several mind-reading tools into a single study, researchers used neuroscience to find a predictive whole greater than the sum of its parts

One of the big takeaways from 2016 seems to be that we’re still bad at understanding one another. It’s a considerable challenge for marketers, whose livelihoods depend on being able to know and influence others. The usefulness of self-reported market research stops short of a grand theory of buying behavior. Many subjects can’t explain why they responded well to a particular ad, or they can’t articulate their connection. 

For years, marketers have been turning to neuroscientists in their quest to drill deeper. Each new tool sheds insights on what actually drives consumer response. Facial coding is able to account for 9% of explanatory power, while the Electroencephalogram (EEG) reveals as much a 62% of decision making. But when Dr. Carl Marci, chief neuroscientist at Nielsen Consumer Neuroscience, combined all the tools into a single measure, he developed a way to see further than anyone else.

The official name of the method he developed is the Video Ad Explorer. Unofficially, he calls it the “Holy Grail of marketing”—a penetrating probe of consumers’ brains that is able to measure results with up to 77% explanatory power with in-store sales. Now, Marci talks about the significance of his research and how it could reset that ad-making process.

Q: You say you’ve found the Holy Grail of marketing. What do you mean by that?

A: There are different ways to measure non-conscious processing. EEG, biometrics, facial coding, [etc.]. Then there’s a component of self-report. How do people articulate what they like or don’t like about an ad and what they will or won’t buy? We had an opportunity here to compare all of those measures in one study that had a very strong sales outcome. 

The Holy Grail would be, which measures have value and in what combination. That was the big question that couldn’t be answered. The theory was that we were measuring different things. Therefore, if you group them all together, you would be able to capture more explanatory power of advertising creative than if you only had one or two. 

Q: How did you test that theory?

A: We looked at 60 ads across a wide variety of product categories—for instance, adult beverage, soft drinks, women’s beauty products, baby products, health and beauty and a variety of others—as they were airing and then simultaneously did testing over the course of five months. We recruited participants, and we measured EEG, biometrics, facial coding and with separate samples, self-report.

Q: Aside from self-report, are all these measurements non-conscious?

A: Yes. By non-conscious, we’re talking about measures that we don’t consciously manipulate all the time. We passively measure them. Unlike self-report (which is after you’ve seen the ad you answer a bunch of questions), we collect EEG, biometrics and facial coding as you watch the ad. So it’s not looking in the mirror, it’s the actual experience moment to moment. We then analyze the data and create metrics for every second of the ad. Then, we put all those metrics into a statistical model to see how it relates to sales. 

Q: How do you measure sales?

A: Nielsen Catalina Solutions has an approach where they take home-scanned data and combine that with set-top-box data. We have data on households exposed to the ad versus houses that are not. You then follow them into the store in a four-week window. Importantly, after the exposure, you look for the lift in the sales, in the product category and in the product itself. You control for the media spends. You control for the targeting, the size of the product and the timing. When you control for all those things, you get a fairly pure measure of the creative’s effect on sales. It’s not just how much ad spin you bought or how many channels you were on. That’s what is really exciting. 

Q: With the pure measure of creative, which creative did best? Which did poorly?

A: It’s a good question. The study question was, “What technologies did the best job of capturing a creative effect?” We’re not out there saying this type of ad does better than that type of ad. It’s much more about which methodology captures or has the most explanatory power. It turns out that the highest explanatory power came from the EEG. The second was the biometrics. Third was self-report and fourth was facial coding. In that order, and the range was as low as 9% and as high as 62%.

Q: What about eye tracking?

A: We use eye tracking for diagnostics, so we can see where people are looking. Are they looking at the brand or not? We don’t use eye tracking as a performance measure. The reason is a little bit technical but I can show you ads that are very simple, where people barely move their eyes, and ads that are more complex, and people move their eyes a lot. They can be equally engaged and have equal results in market. 

Q: How was ad reactive measured before this study? If you wanted to know or try to access the impact an ad would have or the purely creative aspects of an ad, what did you have to look at? 

A: The marketplace typically has self-report, which is still by far the most commonly used measure by itself. There are a number of companies that are adding facial coding to self-report. You might get two of the measures, but as I described, that’s probably the weakest. With the acquisition of Innerscope, we were able to combine the EEG, biometrics and facial coding into one package. That had never been done before. Nobody has the combination that Nielsen has right now. 

Q: How did you pick the ads?

A: Based on availability in market. We started the test late last year, and we were looking for big brands that we knew were going to be hitting a lot of households. The major selection criteria were: “Are you a big CPG brand?” and “Could we get the ads to test it?”

Q: You had to mount a theoretical defense for putting all these together. Why would there be opposition to that if it helps marketers better understand consumer behavior?

A: There’s a natural skepticism from clients who assume we’re going to charge more for more tools. Then, I think the research skeptics come out and say, “I get facial coding with this supplier and my self-report. Why do I need anything else?” 

Someone who expresses an emotion on their face—they smile, or they look surprised or they frown—that tells us something very important about that moment and the creative in the audience at that time. The majority of the time with video advertising, people aren’t actually expressing an emotion; they’re staring at the wall. This doesn’t mean they’re not having any emotional response. It just means they’re not expressing it on their face. We need other tools, like biometrics, to [understand] that emotional journey throughout the ad. EEG gives you more coverage of the brain. You can also get memory activation, emotional motivation and attention processing. When you think about it, you really have a nice combination of tools that, in theory, should complement each other. It turns out they actually do.

Q: How so?

A: With EEG and its broad coverage, With EEG and its broad coverage, you're getting the most brain areas covered. We derive three different measures from that. That’s not something that facial coding captures, that’s not something that biometrics captures. It’s very unique to EEG. Memory activation is something that biometrics and facial coding can’t capture, but EEG can’t tell you if someone’s smiling or frowning; it can just tell you if they’re engaged. EEG can’t tell you if the energy level in the ad is high or low, and that’s where the biometrics come in.

Q: How much can an ad change on a second-by-second basis in the brain?

A: A lot. If you look at a 30-second ad, you can see anywhere between 10 and 15 peaks and valleys in the EEG trace, sometimes more than that. All within a 30-second ad. That’s a lot of journey going on within someone’s brain. It’s hard to imagine the granularity until you really see how these things shift. I’ve learned to respect the 30-second video ad. You’ve got sight and sound. You’ve got a story with a beginning, middle and end. You have to have relatable characters, bring people on some kind of a journey, integrate the brand product or service, make sure the brand’s prominent enough to engage their attention, their emotion, leave a memory trace and then motivate them at some future time in a story to make a purchase. That’s a lot to ask for a 30-second video.

Q: When you put all these measurements together, what do you know about how somebody is responding to an ad?

A: We’re able to … tell our clients whether this is an ad that has got great promise or this is an ad that needs some work. Then, because we can measure second-by-second, we can also find which areas of the ad are areas for improvement. Another thing that we do on a fairly regular basis is take that 30-second ad and help our clients turn it into a 15-second ad by taking just the most engaging parts. There’s some art as well as science to that, but we’re able to direct our clients to keep some parts, and throw others out.

Q: The results of your study showed that the integration of multiple neuroscience measures results in up to 77% explanatory power with in-store sales. Could you unpack that number?

A: That number comes from a statistical model where we’re looking at the in-store sales effects, controlling for all those variables I described (the media plan, the targeting, the size of the product). It includes inputs from the facial coding, the biometrics and the EEG. How much of the [creative effect on the consumer] does facial coding get? Between 9% and 12%, one little piece. How much does biometrics get? A bigger piece, almost a third. What does EEG alone factor? That’s the biggest chunk, upwards of 62%. Now if these tools all measured the same thing and had significant overlap, we would expect that combining them would not go above 62%. However, we actually found that putting them together brings our number up to an impressive 77%, proving the power of using all of the tools in combination.

Q: Do you have any theories as  to what the remaining 23% could be?

A: No. It’s in the ether. As good as these technologies are, we’re not capturing every aspect of people’s mind and body. If we could add, say an FMRI—which we don’t because it’s very expensive and cumbersome—I could actually focus in on a few parts of the brain that none of these technologies capture very well. That might capture that last 23%.

Q: Why does this work? Why do non-conscious responses to ads translate into predictive consumer behavior?

A: Between 50% and 99% of brain processing is occurring without our awareness. There’s a whole lot going on in our brain that we just aren’t aware of and don’t have access to.

There’re a lot of theories and evolutionary reasons for that. We can’t possibly think about every single decision we make. It would be exhausting, and we’d be paralyzed. At the Shopper Brain Conference in Chicago, we’re hearing people talk about how [consumers] walk through retail stores. When [consumers] describe how they walk through that store, and you actually measure how they walk through that store, the two don’t look anything alike. That’s just one of many examples of how our memories and our conscious awareness of our behaviors are very limited.

People aren’t very good at explaining every aspect of their life. If you only ask people questions like, “Do you like that ad?”, “Do you remember that ad?” or “Would you buy that product?”, you’re only talking to one part of the brain: the conscious part. That might be the smallest part, so you need tools that can capture non-conscious processing. We’re not making claims that we’re capturing 100% of what’s left, but we’re capturing enough to create a powerful explanatory model.

Q: Did your results vary by product category or by target? 

A: We included all of the categories in the data, and the data set’s not quite big enough to be able to say how baby care and adult beverages perform differently. All of the advertisers think their category is special, but across these measures, so far we actually haven’t seen big differences. Engagement looks like engagement across these measures. When you look inside of categories, we have found with the biometrics you actually see more variants within categories than between.

With biometrics years ago, we looked across a couple hundred ads. We had two groups: One was entertainment, movie trailers. What could be more emotional? We compared that to financial services, banks, things like that. What could be less emotional? When you compare the two, the difference on a 100-point scale was one point. That was because there were a lot of differences within each category, so I could show you movie trailers that actually weren’t very emotional at all and financial services that got people pretty lathered up. What we see is there tends to be so much variability within a category that it washes out any potential effects between categories.

Q: Do you have any follow-up studies planned based on these results?

A: The next step is to look closer at the individual moments within the ads and to build on the data set to make it bigger and also to look within the ads to see what everybody wants to know: how do you make a great ad? That’s the work of 2017.

Recommended For You:

The Neuroscience of Brand Trust 15 Ways to Use Neuroscience in Your Web Design Gain a Global Perspective at AMA Events

The Neuroscience of Brand Trust

15 Ways to Use Neuroscience in Your Web Design

Better Understand Your Customers at
AMA Events


 Sign Up For Marketing News Weekly

Get the best marketing thought leadership delivered directly to your inbox!

Author Bio:
Zach Brooke
Zach Brooke is a staff writer for the American Marketing Association. He can be reached at
Add A Comment :

Become a Member
Access our innovative members-only resources and tools to further your marketing practice.