AIMS2012 Marketing Associates Quantifying the Buzz Effect.

Create a Business Blueprint
With DataͲDriven Customer Insights

Quantifying the "Buzz" Effect: Integrating Social
Media with Loyalty & Defection Models
Marketing Associates:
Keith Shields, Director, Decision Sciences
Roni Leibovitch, Senior Consultant, Digital Intelligence
Mindy Deatrick, Senior Consultant, Quantitative Solutions

Ford Motor Company:
Margaret Kishore, Performance and Metrics Manager

Decision Sciences
Copyright ©2012 Marketing Associates LLC. All rights reserved.

About the Title…
‰ “Buzz” refers to the amount of, and sentiment of, the FordͲrelated comments
available through social media outlets.

‰ The “Buzz Effect” refers to the increase or decrease in brand loyalty / defection
(measured by repurchase) that occurs as a result of a change in the Buzz.
ƒ “Quantifying the Buzz Effect” means we want to put a number on the amount of that
increase or decrease.
ƒ The advantage of this is that we can begin to put a dollar value on salient, publiclyͲ
known events…such as refusing to take government bailouts.

‰ “Integrating Social Media With Loyalty / Defection Models” means that we will:
ƒ Extract signals of future vehicle purchase decisions from customer comments found
through social media outlets AND
ƒ Capture those signals in the form of predictive variables to put into loyalty models.
ƒ Those variables and their associated model coefficients will quantify the buzz effect.
ƒ For the purpose of this analysis we focus our efforts on Twitter.

Decision Sciences
Copyright ©2012 Marketing Associates LLC. All rights reserved. 2

Warnings and Disclaimers
‰ We will do our best to reveal trends, patterns, and findings without showing
actual numbers (but for some cases). Hiding / changing of numbers is done
to protect the innocent (Ford Motor Company especially).

‰ In the course of the presentation we will share many FordͲrelated “tweets”.
These will be actual tweets. They will not be censored because their
informal nature highlights a point we want to make about text mining.
Please try not to be offended.

‰ We use “offͲtheͲshelf” techniques when it comes to categorizing sentiment.
ƒ Our expertise is in modeling and predicting customer behavior based on all
available and relevant customer data.
ƒ We see social media as a potentially rich source of customer data, and those data
just happen to be freeͲform text.

Decision Sciences

One More Item of Note…

Decision Sciences

Background on Ford’s Social Media Efforts…
‰ Measuring the “Consumer Experience”
ƒ Alan Mulally and Apple…
ƒ The Dealership Experience: Sales and Service
ƒ The Ownership Experience
ƒ How do people share experiences? Traditionally by talking to each other. But how
much today is done through Twitter, Facebook, Blogs?

‰ By analyzing the comments and sentiment expressed through Social Media
outlets can we glean meaningful insights about the Ford Consumer
Experience?

‰ Can we make inference about a consumer’s affinity for Ford…or an existing
customer’s loyalty to Ford?
ƒ If no, then we’re probably not trying hard enough.
ƒ Examples next 2 slides.

Decision Sciences

Google Twitter Search: Ford Comments
‰ Search: “My Ford Focus is great.”

ƒ I love my Ford Focus, but not so much Ford Service in Northampton Mass.
Thieves.
ƒ Got my new computer yesterday and can't wait to get my new 2012 Ford
Focus SEL in 4Ͳ6 weeks! 23 Apr
ƒ Am test driving Hondas and Fords 7 Apr

¾ We’d like to have a mechanism for intervening here. On April 7 this person
indicated he was facing a choice between buying a Honda and buying a Ford.

¾ Does this mean we can simply scrape Twitter for the words “test drive”? Seems
like it would be predictive of future behavior…

Decision Sciences

‰ Search: “I don’t like my Ford Escort.”

ƒ The ford escort texting and driving, I really like my life and my car,
please don't try and drive into us, twice. Close call!

ƒ My old '93 ford escort is running 130k and runs like a charm.... And
my 2003 ford ranger truck has 80k without problems.

¾ Again this seems like something that, if captured and
quantified in the form of a variable, would be predictive in
the context of a loyalty model.

Decision Sciences

‰ Search: “Ford, government bailout”
ƒ This weekend my wife and I purchased a FORD. Why? Because they chose
not to accept the government funded bailout.
ƒ Ford didn't accept the government bailout Ͳ that's pretty awesome.
ƒ Wait #Ford pulled the ad that was critical of the #Obama bailout but is
now running one that jokes about drinking and driving?
ƒ GM CEO wants higher gas tax. Buy a Ford car or truck. Please RT

¾ This is an example of how capturing “influencers” could be very important. This person
happens to have 340 followers and routinely tweets about autoͲrelated topics.

¾ So the effort to mine Twitter for Ford sentiment extends beyond improving the loyalty
and defection models…but the title of this presentation does not. That said, we will
discuss how we are affecting marketing programs with our existing knowledge of
influencers.

Decision Sciences

Background on Ford’s Social Media Efforts…
‰ Measuring the “Consumer Experience”
ƒ Alan Mulally and Apple…
ƒ The Dealership Experience: Sales and Service
ƒ The Ownership Experience
ƒ How do people share experiences? Traditionally by talking to each
other. But how much today is done through Twitter, Facebook, Blogs?

‰ By analyzing the comments and sentiment expressed through Social Media
outlets can we glean meaningful insights about the Ford Consumer
Experience?

‰ Can we make inference about a consumer’s affinity for Ford…or an existing
customer’s loyalty to Ford?
ƒ Yes! So what can we do capitalize upon good sentiment and reverse bad
sentiment?

Decision Sciences

Start With the Current Infrastructure
‰ The Ford Motor Company has a customer data warehouse that collects relevant data from all customer
touchpoints, “customerizes” it, and applies a suite of predictive models that are used for targeted
campaigns.
‰ More importantly the warehouse is connected to many customerͲfacing and dealerͲfacing operational
systems, and it passes important information about customer behavior, both past behavior and
predicted behavior, to operational systems when decisions regarding the customer have to be made in
real time.

Decision Sciences

Fitting in to the Current Infrastructure…
‰ Social media is just another customer touch point.
‰ The text we mine from social media outlets is another set of data about the customer, just
like the call center, website, or the customer surveys.
ƒ We’d like to use that data just like the rest of the customer data: to help us predict customer
purchase behavior.
‰ In some sense social media provides a source of unsolicited surveys.

Decision Sciences

A Source of “Unsolicited Surveys”…
‰ Why do we survey customers? From the narrow perspective of someone who predicts
customer behavior, the graph below is a big reason why.

‰ What’s more compelling is that
relationship between a customer’s
opinion and loyalty holds up when
we control for predicted loyalty.
‰ The “Loyalty=1” group is the group
that scores in the lowest 20% of a
loyalty model…a model built without
survey data.
‰ The results remain consistent within
each loyalty tranche so much so that
customers within group 5 can have
lower repurchase rates than those in
group 3, depending on survey
response.
‰ How much do we spend on surveys?
ƒ Whatever it is, our feeling is that if we can establish the above relationship with social media
sentiment (use it as your XͲaxis), and cover more customers for less than what we currently
spend on surveys, then we have the beginning of a business case for extracting sentiment from
social media.

Decision Sciences

A Quick Digression on Business Cases…
‰ We believe that there are three “pillars” for the business case to
actively engage consumers through social media outlets (specifically
Twitter):
1. Conquesting new customers
2. Concern resolution
3. Voice of Customer

STRATEGY: Introduce Social Media as an additional
consumer touchͲpoint

SUPPORTS THE 1. Conquest 2. Concern Res 3. VOC
STRATEGY: $XX mils per year $XX mils per year $XX mils per year

Decision Sciences

How We Use Survey Data In the Models…
‰ Let P = probability a Ford customer will repurchase another Ford upon disposing of any
one of his current Fords.
‰ Logistic regression is a very popular way to model and predict P.
ƒ ln[p / (1Ͳp)] = b0 + b1*x1 + b2*x2 + … bn*xn
ƒ b0, b1, b2 are parameter estimates. They quantify the extent to which x1, x2, …, xn affect the
probability of repurchase.
ƒ x1, x2, …, xn are explanatory variables, e.g. # of previous Ford purchase, time since most
recent Ford purchase, miles from nearest Ford dealership, etc…
‰ Now let s1 = 1 if “very likely”, 0 otherwise
‰ Let s2 = 1 if “likely”, 0 otherwise …
‰ Let s5 = 1 if “not at all very unlikely”, 0 otherwise
‰ Refit the logistic regression:
ƒ ln[p / (1Ͳp)] = b0 + b1*x1 + … bn*xn + bn+1*s1 + bn+2*s2 + bn+3*s3 + bn+4*s4 + bn+5*s5

Can be thought of as the VOC (Voice of Customer) Index, but it’s based on just
survey data, which may only be available on 10% (roughly) of the customers.
This is a nice metric, because it, by design, predicts loyalty.

Decision Sciences

How We Use Twitter Data In the Models…
‰ Let’s treat the Ford customer’s “tweets” the same way we treat survey data.
‰ Go back to our logistic regression:
‰ And let t1 = 1 if we can identify a “Ford positive” tweet for the customer, 0 otherwise.
‰ Let t2 = 1 if we can identify a “Ford neutral” tweet for the customer, 0 otherwise.
‰ Let t3 = 1 if we can identify a “Ford negative” tweet for the customer, 0 otherwise.
‰ Refit the model:
+ bn+6*t1 + bn+7*t2 + bn+8*t3

Can be thought of as the BUZZ INDEX, and it comes directly from what
ford customers are saying on Twitter. This metric also, by design,
predicts loyalty. So this is a quantification of the BUZZ EFFECT.

Decision Sciences

Interpreting the “TwitterͲEnhanced” Model…
‰ ln[p / (1Ͳp)] = b0 + b1*x1 + … bn*xn + bn+1*s1 + bn+2*s2 + bn+3*s3 + bn+4*s4 + bn+5*s5
+ bn+6*t1 + bn+7*t2 + bn+8*t3

‰ When we fit this model, we get an intuitive result:
ƒ bn+6 > bn+7 > bn+8 => good tweets lead to higher loyalty than do neutral tweets,
neutral tweets lead to higher loyalty than bad tweets.

‰ Not as intuitive (but interesting nonetheless):
ƒ All three parameters are greater than 0 (implying ANY tweeting is better than no
tweeting).
ƒ bn+8 (the parameter for bad tweets) is NOT SIGNIFICANT. There is not a sufficient
volume of bad tweets to support a significant result. The large majority of FLM
tweets are good.

‰ We think that the upshot of all of this is that tweeting about Ford, irrespective of
sentiment, signifies a high level of customer engagement. This has implications
beyond our efforts to better predict loyalty.

Decision Sciences

The Buzz Variables Improve the Loyalty Model: So What?
‰ More data and better data yield models that do a better job of “separating” loyalists
and nonͲloyalists.
ƒ One way this manifests itself: ranking the population of customers with a better model will
yield higher repurchase rates in the top decile (or demiͲdeciles…depending on how many
groups you want to establish), and lower repurchase rates in the bottom decile.
ƒ So a marketing campaign that increases everyone’s likelihood of repurchase by 15% (not an
uncommon number), does so on a larger base of loyalists within the top decile, and thus
creates more incremental sales for the same amount of mailings.

¾ Say the difference between these two bars is 200 bps.
¾ Some of the incremental sales from the campaign
Repurchase Rate

Old Model
noted in the bullet above (top decile only), are
Model w/VOC & Buzz attributable to having a better model.
¾ How many? .02 * .15 * top decile population
¾ If the population of interest is 250,000 customers, then
the impact of the better model is 750 incremental
repurchases.
¾ If a repurchase is worth $5,000 profit, then the case for
1 2 3 4 5 6 7 8 9 10
the “buzz variables” is substantial: $3.75 million.
Model Decile
Low Loyalty to High

Decision Sciences

Why We Will Regularly ReͲFit the Buzz Index
‰ We have several reasons to believe that results may change when we reͲfit our
enhanced loyalty model (twitter sentiment data being the enhancement):
1. The number of Tweeters is increasing all the time. Ford’s customer email capture isn’t great
but it is improving, and there is evidence that Ford customers are, relatively speaking, very
active on Twitter.
2. Attribution of tweets to customers is difficult and unsure; finding the Twitter names of Ford
customers is difficult and painstaking.
3. Classifying the sentiment of Tweets is an imprecise exercise, especially when using offͲtheͲ
shelf tools and software.
4. The content of the “FordͲtweeting” population leads to potentially biased results; it is biased
toward a demographic that naturally tends to be less Ford loyal:
ƒ Females
ƒ Young
ƒ Used vehicle owners
ƒ Appear to be more service loyal, which is a good thing
5. The tweeting population also happens to be geographically biased, but this does not concern
us as much as #4.

Decision Sciences

ReͲFit the Buzz Index: The Increasing Number of Tweeters
‰ According to GIGAOM (http://gigaom.com/), Twitter had 175 million users in December 2010, and
was growing by 370,000 new users every day. Also as of 12/2010:
ƒ 65% of those users lived outside the US.
ƒ Roughly 6% of all Americans were active on Twitter. More recent studies indicate the number is 9%Ͳ10% (or
13% of internet users).
‰ Of the Ford customers active as of 12/2010 (who had a valid email), we were able to find 9% of them
active on Twitter.
‰ Not surprisingly we have found the number of Ford related tweets to be increasing over time.
ƒ Good and neutral tweets have increased whereas bad tweets have stayed flat. Seems like a good thing, but we
have some thoughts on comment classification.

Frequency of FordͲRelated Comments Found on Twitter FordͲRelated Twitter Comments "Binned"
60,000
6000 60000
50,000 Negative
5000 50000
Positive
40,000

Postive, Negative
4000 Neutral 40000

Neutral
30,000
3000 30000

20,000 2000 20000

10,000 1000 10000

0 0 0
AugͲ08

AugͲ09

AugͲ10
AprͲ09

AprͲ10
JunͲ08

SepͲ08
OctͲ08
NovͲ08
DecͲ08
FebͲ09

JunͲ09

SepͲ09
OctͲ09
NovͲ09
DecͲ09
FebͲ10

JunͲ10

SepͲ10
OctͲ10
NovͲ10
DecͲ10
FebͲ11
SepͲ08

NovͲ08

SepͲ09

NovͲ09

SepͲ10

NovͲ10

MayͲ08
JulͲ08

JanͲ09
MarͲ09
MayͲ09
JulͲ09

JanͲ10
MarͲ10
MayͲ10
JulͲ10

JanͲ11
MarͲ11
MayͲ08

JulͲ08

JanͲ09

MarͲ09

MayͲ09

JulͲ09

JanͲ10

MarͲ10

MayͲ10

JulͲ10

JanͲ11

MarͲ11

Decision Sciences

ReͲFit the Buzz Index: Attributing the Tweets
‰ Twitter names can be found if you can supply an email.

‰ With an email address you can find, through the Twitter API, a Twitter name, a first
name, and a last name associated with that email address. It will not return the email.

‰ Ford has 11 million boughtͲnew stillͲretained customers. We have emails on several
million of them.

‰ We cannot run several million emails through the Twitter API. Even if they could be
processed, we would not be able to get back the email. We would only get back the
thousands of Twitter names associated with, but not matched to the emails.

‰ The only sure way to attribute emails to Twitter names is to go through the API one email
at a time. Can we offͲshore this? We can, but we still will not be done until 2013 if we
go this route. So we had to be more clever…

‰ Whatever method we choose, we need to recognize the Twitter name as useful customer
data, and as such, store it in the data warehouse. Next slide…

Decision Sciences

ReͲFit the Buzz Index:
Attributing the Tweets – You Must Retain Data
‰ What comes out of the attribution process is a table that looks like this:

EMAIL TWITTER_NAME FIRST_NAME LAST_NAME
kshields@marketingassociates.com Keith Shields
rleibovitch@marketingassociates.com ronedog Roni Leibovitch
mdeatrick@marketingassociates.com Mindy Deatrick

‰ We have another process (using RADIAN6) that scours Twitter for comments
that contain words in our “start list” (e.g. Ford, Lincoln, Mercury, Taurus,
Mustang, Fusion, etc…). It produces a table that looks like this:

TWITTER_NAME COMMENT SENTIMENT
ronedog My new Ford Focus is also imported from Detroit. Good

Decision Sciences

Attributing the Tweets – You Must Retain Data
‰ We have to integrate this into existing data warehouse processes (which
should be easy enough, if we’re treating this like just another source of
customer data):

Customer Touchpoint
EMAIL TWITTER_NAME FIRST_NAME LAST_NAME
Twitter Data Creation kshields@marketingassociates.com Keith Shields
rleibovitch@marketingassociates.com ronedog Roni Leibovitch
mdeatrick@marketingassociates.com Mindy Deatrick

Customer Data
TWITTER_NAME COMMENT SENTIMENT
Warehouse ronedog My new Ford Focus is also imported from Detroit. Good

Decision Sciences

ReͲFit the Buzz Index: Classifying the Tweets…
‰ Prepackaged comment binning algorithms are not as accurate as we’d like…they result in
a high instance of inappropriate comment binning (“Positive”, “Neutral”, or “Negative”).
Here are some actual examples of inappropriately binned comments:
ƒ Positive: “Classic Car For Sale 2001 FORD EXPLORER Ͳ Mt. Royal NJ: Runs and Looks Great!!!”
ƒ Negative: “You have insulted my Ford Fiesta, shame on you.” AND “Just drove a Ford Fiesta
getting 30 mph. Not bad!” AND “Just dropped my car off at the FORD Dealership . I want a
FORD Fusion soooooo BAD.”
ƒ Neutral: “Ford Escape!!!” AND “Smart, easy, & fun ride with the new Ford Focus with Ford
Sync's help!”

‰ In order for us to get the intuitive results we showed on slide 12 we had to depart from
sentiment classification algorithms and do a “bruteͲforce” classification.
ƒ Interns, Cornerstone Schools, Detroit, Mi. (http://www.cornerstoneschools.org/)
ƒ There are other inexpensive ways…all of which we believe to be more accurate than existing
“machine intelligence”…albeit not as scalable:
• Existing call center personnel
• Mechanical Turks (https://www.mturk.com/mturk/welcome)

Decision Sciences

The Biased, But Changing, Population of Tweeters…
‰ See the graph on the right. The fastest growing population of tweeters is 18Ͳ34 year
olds. About 56% of tweeters are 34 years or younger.

‰ The challenge for Ford: the median age for Ford customers is well above 34, despite
some recent strong entries in the small car market.
‰ The most “tweeted about” Ford is, not surprisingly, the Focus.

Decision Sciences

GeographicallyͲBiased Population of Tweeters…
‰ The numbers represent how
much higher the Twitter use per
capita is in that state versus the
nation as a whole.
‰ For example: if the national
usage rate is 10%, then
Michigan is 11% lower than
that: .10 Ͳ .11(.10) = 8.9%.

Decision Sciences

Given the Worries About Attribution, Classification…
‰ The real opportunity could lie in the business experts using intuition and
common sense to tailor campaigns and programs to tweeters based on their
most recent comments.
ƒ This would only rely upon a good mechanism for scraping relative comments
from Twitter and reacting procedurally and appropriately.

‰ If we look at the comments as unsolicited survey responses we see
opportunities for customized offers and programs (no models needed – the
comment reveals the customer’s intent):
ƒ Private Sales Offer and/or PreͲapproval: “Just dropped my car off at the FORD
Dealership. I want a FORD Fusion soooooo BAD.”
ƒ Rewards Program Offer: “My moms taking me to get this Ford explorer in the
mornin tho i should have a new whip before July then im haulin ass to the A”
ƒ Offer for tradeͲin: “Ford focus sucks. Very uncomfortable vehicle.”
ƒ Offer for service discount / extended warranty: “My car is running rough and
keeps blowing the injector and on plug coil fuses, its a 2006 3.0 V6 Ford Fusion.
HELP!!”

Decision Sciences

So We Revisit Our Three Pillars…
‰ We believe that there are three “pillars” for the business case to
actively engage consumers through social media outlets (specifically
Twitter):
1. Conquesting new customers: pay attention to influencers
2. Concern resolution
3. Voice of Customer

STRATEGY: Introduce Social Media as an additional
consumer touchͲpoint

SUPPORTS THE 1. Conquest 2. Concern Res 3. VOC
STRATEGY: $XX mils per year $XX mils per year $XX mils per year

Decision Sciences

Conquest New Customers: Influencers

Fact: 93.6% of Twitter users have less than 100 followers, while 98% of users
have less than 400 followers. Meanwhile, 1.35% of users have more 500
followers, and only 0.68% of more than 1,000 followers.

Decision Sciences

Conquest New Customers: Influencers

Fact: As Twitter users attract more
followers, they tend to Tweet more
often. This is particularly evident
once someone has 1,000 followers
the average number of Tweets/day
climb from three to six. When
someone has more than 1,750
followers, the number of Tweets/day
rises to 10.

Fact: A small group of Twitter users account for the bulk of activity. Sysomos discovered
that 5% of users account for 75% of all activity, 10% account for 86% of activity, and the
top 30 account for 97.4% of activity.

Decision Sciences

Conquest New Customers: The Opportunity
‰ Find out who is expressing inͲmarket sentiment and send them a targeted offer.
ƒ We estimate that through Twitter alone, roughly 35,000 customers per year express inclination
to buy Ford.
ƒ Marketing Associates built the process to find the tweets and measure the backͲend results.
Here are some great “Focus tweets”…just from the last couple of weeks:
¾ “I think I want a 2012 Ford Focus.” 3/12/2012
¾ “I want a 2012 ford focus...just because it parks itself. :” 3/20/2012
¾ “2012 Ford Focus ST or 2013 Dodge Dart? I dunno, the Dodge Dart it is just a Neon that
mated with an Alpha Romeo but the Focus ST looks pretty promising and I always loved
my buddies SVT Focus.. Decisions.. Decisions..” 3/1/2012

‰ Applying a result from an analysis of "handraiser campaigns", we assume 15% of the
35,000 will purchase FLM. This is 35,000 * 15% = 5,250 sales.

‰ Assuming 20% lift from a targeted offer to inͲmarket customers (not an uncommon
number), we estimate that a conquesting campaign directed at inͲmarket "socialͲmedia
leads“. This is 5,250 * .2 = 1,050 incremental sales.

‰ INTEGRATION will be through the customer data warehouse and EXECUTION through
the concern resolution center.
ƒ At this point we won’t trouble you with another infrastructure diagram.

Decision Sciences

Some Recommendations…
‰ Treat social media as another source of relevant customer data.
ƒ Comments about your product are, in some sense, unsolicited surveys. They can, like surveys
do, improve your ability to predict the behavior of your own customers.

‰ Pay careful attention to the integration of social media data. Integration
requires “customerization”, so subsequent customer behavior can be tracked.
ƒ Attributing comments to customers is tricky. It can also be painstaking. The good news is that
it can be done cheaply.

‰ Correct classification of comments is essential to understanding the true signal
in the comments.
ƒ The most accurate means of classification may also be the least scientific: have an English
speaker (who preferable understands colloquialisms) read the comments, and bin them.
ƒ Categories can be “good”, “bad”, “inͲmarket”, “service issue”, or whatever aligns with the
differentiated treatments and offers.

‰ Retain data, and integrate intelligently into a data warehouse. Test and
measure several tactical approaches to customers and prospects who are
commenting about your products.

Decision Sciences

Quantifying the "Buzz" Effect: Integrating Social Media
with Loyalty & Defection Models

Thanks for your time and attention.

Questions?

Want a copy of this presentation?
Text KEITH to 30241

Marketing Associates Alerts: Receive up to 2 msgs per month. Msg&Data rates may apply. Text STOP to stop. For more info, email info@marketingassociates.com.

Decision Sciences

AIMS2012 Marketing Associates Quantifying the Buzz Effect.

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (17)

Andere mochten auch

Andere mochten auch (19)

Ähnlich wie AIMS2012 Marketing Associates Quantifying the Buzz Effect.

Ähnlich wie AIMS2012 Marketing Associates Quantifying the Buzz Effect. (20)

Mehr von dmadetroit

Mehr von dmadetroit (7)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

AIMS2012 Marketing Associates Quantifying the Buzz Effect.