Consumer awareness has been successfully measured for many years with traditional methods, such as surveys, panel studies or focus groups. Pulte Group, a leading business-to-consumer marketer and home builder, was looking for a means to gauge how often consumers would consider their brand relative to their competitors online, but with less expense than traditional survey methods and without self-reported bias. As a marketer that employs a truly cross-media strategy in various markets, Pulte turned to Organic for the measurement solution. Using website traffic tools, social media monitoring, search activity metrics, specific housing industry metrics, and SAS, Organic was able to reduce all of these online and social media vehicles into a single, trackable monthly index. On July 13, 2011, the IAB and leaders from Organic and SAS reviewed insights from this project and partnership. The speakers answered questions about challenges in understanding consumer behavior, analytics, and the wealth of data available from social media sources. The companies also shared their best practices in digital analytics in the following discussion: SAS: Digital Analytics - Best Practices 1. How can an organization bring together different forms of online data? 2. How to manage/scrub/modify the data to begin making sense of it? 3. Approaches to analyzing online data – including sentiment analysis, data mining, and forecasting Presenters: Vinicius Vivaldi Sr. Solutions Architect, SAS Institute Suneel Grover Solutions Architect, SAS Institute Jason Harper VP – Marketing Intelligence, Organic, Inc. John Bejnarowicz Senior Statistician, Organic

1. Navigating the Cross-Section of Consumer Insights,
Sentiment, & Online Behavior
Jason Harper – Organic
John Bejnarowicz – Organic
Suneel Grover – SAS
Vinicius Vivaldi- SAS
IAB Webinar hosted by Organic and SAS

2. Who is Pulte
Group?

3. The Competition

4. Is their Marketing
Working?
© & ™or ® , Organic, Inc. Confidential. All rights reserved

5. New Marketing Technique: COBI
© & ™or ® , Organic, Inc. Confidential. All rights reserved

6. Agenda
• The Customer Journey
• The Digital Landscape
• Synthesizing Data Streams
• Predicting Success
• Lessons Learned
• SAS®: Digital Analytics - Best Practices

7. The Customer Journey
Location, Location, Location?

8. Location, Location, and a Lot More
• Location
• Price Point
• School District
• Builder

9. Capturing Builder Consideration
Potential New Home Buyers
Upper Funnel
(Location)
Social Media Monitoring
Middle
Google Insights for Search (Builder
Consideration)
3rd party web activity
Lower
(Prospect)
Purchase

10. The Digital Landscape

11. Searches

12. Searches

13. Searches
Google
Insights For
Search

14. Website Traffic
• Visits
• Page Views

15. Website Traffic
• Compete
• Alexa
• comScore
• Quantcast
• Hitwise

16. Social Media
• Facebook
• Twitter
• Blogs

17. Social Media
• Meltwater
• Sysomos
• Blog Pulse

18. Synthesizing Data Streams
Organic’s Consideration of Brand Index (COBI)

19. COBI is a single measure that captures a brand’s
“share of consumer mind” online.
Are consumers
visiting your website?
Are consumers
talking about your
COBI brand/company
online or virtually?
Are consumers
searching for your
brand/company online?

20. Multiple Virtual Metrics are Combined
to Create an Index
Website Visits
Website Page Views
Google Search Index
COBI Social Mentions
Positive Social Media Mentions
New Home Orders*
Measures from a range of “virtual” touchpoints give a better measure of overall
brand share-of-mind than any single measure.
• Direct consumer brand interaction (Website visits, page views)
• Indirect consumer brand interaction (natural Google searches)
• Active consumer participation in brand reputation (social media mentions)
• New home orders were included to give the index a real-world industry metric to
combine with the virtual metrics.

21. Bringing Multiple Metrics Down to One
Data Reduction: the object is to take multiple variables and condense their
information into a lesser number of variables while still capturing the monthly
swings in the data.
• Principal Components
• Multi Dimensional Scaling
• Factor Analysis
Statistical methodology behind Organic’s COBI is Principal Factor Analysis.

22. The COBI Process
1. Determine competitor set
2. Collect data for all competitors from four sources
• Web traffic
• Social media
• Google searches
• Builder New Orders (SEC filings)
3. Create monthly shares for the data
4. Using shares figures, perform Principal Factor Analysis to generate
scoring coefficients for a single factor.
5. Scale the resulting factor scores to range from 0-100

23. How is the Score calculated? – In practice
Collect Data
Total Positive Net New
Google Social Social Sign-ups
Month Builder Site Visits Page Views Index Mentions Mentions (quarterly)
March-10 Beazer 106,893 576,766 9 1,167 196 728
March-10 Centex 61,873 554,456 18 2,491 228 1,823
March-10 DRHorton 145,173 1,580,796 19 931 121 4,037
March-10 Hovnanian 83,707 752,977 8 430 37 961
March-10 KB 110,482 1,194,538 14 6,925 458 1,446
March-10 Lennar 166,992 1,298,694 18 6,540 835 2,652
March-10 Meritage 34,406 353,341 7 232 212 621
March-10 NVR 9,678 38,113 20 348 38 2,000
March-10 Pulte 109,500 525,906 17 2,960 344 1,925
March-10 Richmond 67,327 434,888 6 327 287 637
March-10 Ryland 78,951 395,765 16 802 209 969
March-10 Shea 69,925 755,982 5 427 82 600
March-10 Std_Pacific 45,328 310,553 6 327 32 554
March-10 Taylor 343,610 1,594,489 5 382 68 1,034
March-10 Tollbros 80,750 1,055,578 14 1,551 176 526

24. How is the Score calculated? – In practice
Calculate “Monthly Shares” for:
Monthly
Monthly Share of Quarterly
• Website Visits Monthly Monthly Share of Positive Share of
Share of Share of Google Social Social Net New
• Website Page Views Month Builder Visits Views Index Mentions Mentions Sign-ups
March-10 Beazer 7% 5% 9 5% 6% 4%
• Social Mentions March-10 Centex 4% 5% 18 10% 7% 9%
March-10 DRHorton 10% 14% 19 4% 4% 20%
• Positive Social Mentions March-10 Hovnanian 6% 7% 8 2% 1% 5%
March-10 KB 7% 10% 14 27% 14% 7%
• New Orders March-10 Lennar 11% 11% 18 25% 25% 13%
March-10 Meritage 2% 3% 7 1% 6% 3%
March-10 NVR 1% 0% 20 1% 1% 10%
March-10 Pulte 7% 5% 17 11% 10% 9%
Sums to 100% March-10
March-10
Richmond
Ryland
4%
5%
4%
3%
6
16
1%
3%
9%
6%
3%
5%
each month March-10
March-10
Shea
Std_Pacific
5%
3%
7%
3%
5
6
2%
1%
2%
1%
3%
3%
March-10 Taylor 23% 14% 5 1% 2% 5%
March-10 Tollbros 5% 9% 14 6% 5% 3%

25. How is the Score calculated? – In practice
Utilize Principal Factor Analysis to create scoring coefficients, then input the
data values into the resulting equation:
COBI Score = 27.7 + 81.5 * (Share of Visits) + 69.5 * (Share of Page Views)
+0.265 * (Google Search Index) + 44.8 * (Share of Total Social Mentions)
+25.3 * (Share of Positive Social Mentions) + 47.2 * (Share of Net New Sign ups)

26. Pulte COBI

27. Predicting Success
…For the Competition Too?

28. COBI was found to closely track with
Pulte’s “Prospect” sign-ups
80 13,000
12,000
Correlation = 88%
11,000
10,000
70
Consideration Score
9,000
8,000
Pulte
Prospects
7,000
60
6,000
5,000
4,000
50 3,000
Mar-08
Mar-09
Mar-10
May-08
May-09
Dec-08
Dec-09
Oct-08
Oct-09
Jan-08
Jul-08
Apr-08
Aug-08
Jan-09
Jul-09
Jan-10
Apr-09
Aug-09
Feb-08
Feb-09
Feb-10
Sep-08
Sep-09
Jun-08
Nov-08
Jun-09
Nov-09
…and thus, we now have a way to track how our competitors are doing as well.

29. Lessons Learned
Not All Press is Good Press

30. One top competitor is increasing investments
in social media presence… how is that strategy
working for them?
Fairly well
Social media has been a strength
for one top competitor in the past
year, and other measures are
showing similar signs.

31. A competitor is running a “free home” sweepstakes…
how has that tactic increased their consideration
with consumers?
Not much
The sweepstakes created a
short-term bump in web traffic that
disappeared when the
sweepstakes did. No other digital
metrics improved.

32. A competitor has been receiving “bad press” in some
markets… does this show up in lower consideration?
Yes.
This competitors scores are
lower than their market share
might dictate, even though
there is no direct way in the
COBI score that measures
“bad press”.

33. SAS®: Digital Analytics –
Best Practices

34. Unstructured Data Sources
Data
Management

35. Capturing & Storing Unstructured Data
• How to collect the data?
• Leveraging Application Programming Interfaces (APIs) and RSS feeds
• Ability to crawl the Internet
• Storing the data
• Capability of storing the data for
both short and long term views
• Accessing databases
• Native access engines vs. ODBC
connections

36. Data Cleansing
• Unstructured data, in the form of text, when captured, presents its
own level of data management challenges
– Being able to correctly structure the data and clean it is a priority
– Technology needs to have the ability to:
» Eliminate irrelevant information
» Quantity ≠ Quality
• Miss-spelings
• Treat acronyms and abbreviations (e.g. “LOL”)
• Pr☺f@nity
• *Punctuation*

37. Sentiment Analysis
• The action of identifying the expressed sentiments by customers, partners,
suppliers and employees
• Typically categorized into three levels
– Polarity indicator: Positive, negative, neutral
• Why is it important to measure sentiment?
• Public perception of brand, product, and/or service
• Traditional Methodologies
• Statistical
• Rules-based
• Traditional methodologies typically use one or
the other
– Common issues with measuring polarity accurately
– Hybrid approach advantages
• Overall vs. granular/feature-level sentiment

38. Overall vs. Granular/Feature-level Sentiment
 Good, but a little outdated. I bought the Nikon Coolpix L10 as my first digital
compact P&S camera. I had it for a couple of weeks, until mine had a 'lens error' that
basically made the camera inoperable (it was stuck open). It might've been due to
batteries running low, but I tried another set (which I now think was also low).
 The picture quality from the L10 was very good, a bit of barrel distortion was noticed
in the wide angle and shooting tall skyscrapers (noticed by the curve along the side
of the frame where the buildings are supposed to be straight).Another gripe I had
with the camera was how slow the auto-focus was. It would basically go through the
whole range of focus every time I pressed the shutter half-way and then some. This
became more annoying the more I used it.
 Eventually a lot of my pictures came out blurry, including outdoor overcast days with
3x optical zoom. Basically anytime there's zoom & less than ideal lighting, I would
have to have rock steady hands to get non-blurry pictures. Overall it's a good
camera if you can overlook the issues I mentioned.
Product: Nikon Coolpix L10, Polarity: mixed
Feature: Picture Quality, Polarity: positive
Feature: Autofocus, Polarity: negative

39. Unstructured Data Mining & Forecasting
• How does an organization proactively identify new topics, new terms, and
new information being generated by the consumer?
• Unstructured data mining
– Let the data speak for itself
– Develop an early warning or indication system
– Raise awareness of forthcoming topic trends
• Forecasting can help
• Predict a topic reaching a significant threshold and proactively act on this
information
– Marketing applications
» Defend and manage against brand-inhibiting
events
» Understand and act with intelligence during
a new product/service launch
» Augment Net Promoter Score strategies

40. For More Information…
• Jason Harper – Organic
• jharper@organic.com
• John Bejnarowicz – Organic
• jbejnaro@organic.com
• Vinicius Vivaldi (SAS®)
• Vinicius.Vivaldi@sas.com
• Suneel Grover (SAS®)
• Suneel.Grover@sas.com

41. Thank You.

42. Questions
• Please type your questions into
the chat feature on the upper-right
corner of your screen.

43. Upcoming Member Events
• Educational Webinars
– Compliance with IAB’s New Member Code of Conduct,
July 27th @12 Noon EST
• Professional Development Classes
– Essentials of the Digital Marketing Ecosystem, August 4th, NYC
– Professional Presentations: Turn Information into a Story That
Sells, August 9th, NYC
– On-demand training classes also available @ iab.net
• Conferences
– Mobile: IAB Marketplace, July 18, NYC
– MIXX Conference, Expo, & Awards, October 3-4, NYC
– Ad Operations Summit, November 7, NYC