1. 3 Web Measurement Problems Solved!
How Critical Mass Marketing Science is leveraging innovative
thinking, modeling and technology to overcome web measurement’s
biggest challenges.
© 2011 Critical Mass, Inc. All Rights Reserved

2. We are Critical Mass
Amsterdam Calgary Chicago Toronto
London New York Costa Rica Nashville
16 Years
650 Employees
8 Offices
© 2011 Critical Mass, Inc. All Rights Reserved
2

3. We are Critical Mass
Amsterdam Calgary Chicago Toronto
London New York Costa Rica Nashville
16 Years 25+ Dedicated MS Specialists
650 Employees 150+ Years combined
8 Offices measurement expertise
© 2011 Critical Mass, Inc. All Rights Reserved
3

4. A selection of our clients
© 2011 Critical Mass, Inc. All Rights Reserved
4

5. What is
Marketing
Science?
A brief overview...
© 2011 Critical Mass, Inc. All Rights Reserved
5

6. Definition
Marketing Science is the
application of measurement
methods to ascertain, within a
quantifiable degree of
accuracy, the effectiveness of a
marketing initiative.
Or put simply it’s proof!
© 2011 Critical Mass, Inc. All Rights Reserved
6

7. Use data
To
• Measure performance • Deliver customized content
• Find opportunities for • Improve the customer
optimization experience
• Improve marketing • Reduce costs
campaigns • Improve sales
© 2011 Critical Mass, Inc. All Rights Reserved
7

8. Problem: Unreliable data
© 2011 Critical Mass, Inc. All Rights Reserved

9. Unreliable Data
What causes web analytics data to be unreliable?
• Flawed set up
· Developers didn’t understand the tagging
· Tracking of flash or AJAX not included
· Applications or cross domain tracking not tagged
· Specific metrics aren’t being captured
• Code “broke”
· Page or section released without code
· Code rollback broke tagging
· Wrong file released to production
© 2011 Critical Mass, Inc. All Rights Reserved
10

10. Quick Poll
How often in the last year has your site suffered
incorrect or broken web analytics coding?
© 2011 Critical Mass, Inc. All Rights Reserved
11

11. Unreliable Data - Solutions
Flawed set up
New Technology: Tag Management Systems / Universal Tag
WEBSITES
IT / AGENCY / DEVELOPERS
Web Voice of Pay Per
Ad Testing Targeting Affiliate
Analytics Consumer Click
Servers Solutions Tools (PPC) Tools Tracking
Vendors (VOC)
© 2011 Critical Mass, Inc. All Rights Reserved
12

12. Unreliable Data - Solutions
Flawed set up
New Technology: Tag Management Systems / Universal Tag
WEBSITES
Tag Management System Marketing
Science
Web Voice of Ad Testing Targeting Pay Per Affiliate
Analytics Consumer Click
Servers Solutions Tools Tracking
Vendors (VOC) (PPC) Tools
© 2011 Critical Mass, Inc. All Rights Reserved
13

13. Why Tag Management?
1. Single line of code
2. Any vendor tag can be added
3. Analytics resources make updates
4. Works with HTML, AJAX Flash, Flex, Air
5. Remove/pause/replace tags at any time
Tagging changes and
updates go
© 2011 Critical Mass, Inc. All Rights Reserved
14

14. Tag Management Systems – How it works
Browser Tag Rules
Data
Tag Management
System
Third Party
Web Analytics
Or other system
Page
Request
Client Web
Infrastructure
© 2011 Critical Mass, Inc. All Rights Reserved
15

15. Unreliable Data - Solutions
Web analytics code was installed but then
“broke”
New Technology: Analytics Audit and Monitoring Tools
© 2011 Critical Mass, Inc. All Rights Reserved
16

16. Why Analytics Audit and Monitoring Tools?
1. Manual verification too time intensive
2. Provides an overall view specific
attributes
3. Automated tag discovery - every page
checked
4. Scheduled reports, alerts and audits
5. No calls to web analytics systems
6. Works on Flash and secure sites
© 2011 Critical Mass, Inc. All Rights Reserved
17

17. Audit and Monitoring – How it works
© 2011 Critical Mass, Inc. All Rights Reserved
18

18. Problem: Complex reporting needs
© 2011 Critical Mass, Inc. All Rights Reserved

19. Complex Reporting Needs
• Multiple reports with rapid delivery requirements
· Deliver dozens of dashboards in hours, not weeks
· Different stakeholders, different data , different times
• Dashboards and data are not easily accessible
· Stakeholders lack system training
· Emails with dashboards get deleted or misfiled
· Executives on-the-go have no immediate access to data
© 2011 Critical Mass, Inc. All Rights Reserved
20

20. Quick Poll
How complex is your data reporting structure?
© 2011 Critical Mass, Inc. All Rights Reserved
21

21. Complex Reporting - Solutions
Multiple reports with rapid delivery
requirements
Innovative delivery: Web-based reporting and data warehousing
© 2011 Critical Mass, Inc. All Rights Reserved
22

22. Why web-based reporting and data warehousing?
1. Merges data from web analytics and
other sources
2. Doesn’t require a BI department
3. Systems connect directly via APIs
4. Automatically generates highly
customized reports
5. Stakeholders can access from
anywhere at any time
© 2011 Critical Mass, Inc. All Rights Reserved
23

23. Web-based reporting – How it works
Customized
Portal
ETL
Display
Normalization
De-duplication
Rich Media
Search
Email
Affiliate
Media
© 2011 Critical Mass, Inc. All Rights Reserved
Online and Offline 24

24. Complex Reporting - Solutions
Dashboards and data are not easily accessible
Innovative delivery: Mobile dashboard applications
© 2011 Critical Mass, Inc. All Rights Reserved
25

25. Why mobile dashboard delivery?
1. Data in a mobile format =
increased engagement
2. Connects to a variety of data
sources
3. View, analyze and interact with
data visualizations
4. Secure, hosted and scalable
5. Available from data warehousing
and web analytics vendors
© 2011 Critical Mass, Inc. All Rights Reserved
26

26. Mobile Dashboards – How it works
Data to data warehouse to phone
© 2011 Critical Mass, Inc. All Rights Reserved
27

27. Problem: All reports, no action!
© 2011 Critical Mass, Inc. All Rights Reserved

28. Organization Culture and Measurement Spectrum
High Cost
High Consideration
Culture of
Accountability
Low Cost
High Consideration
Culture of Research
Low Cost
Low Consideration
Culture of Advertising
© 2011 Critical Mass, Inc. All Rights Reserved
29

29. All data, data, no action! - Solutions
All no action!
• No clear objectives, roadmaps, or KPIs
· What is the website is supposed to “do”
· Website decisions made by gut feel or HiPPO
• No or inadequate analytics resources
· Tech manages web analytics installation
· A BI department took on web analytics
· No one watching the data
© 2011 Critical Mass, Inc. All Rights Reserved
30

30. Quick Poll
What’s your company culture now?
© 2011 Critical Mass, Inc. All Rights Reserved
31

31. All data, no action! - Solutions
Getting from data to action
Holistic analytics approach
Web analytics maturity modelling and roadmap
The Critical Mass Marketing Science site and project process model
Analytics collaboration and integration
© 2011 Critical Mass, Inc. All Rights Reserved
32

32. Holistic Analytics Approach
• Inform and strategize
Project • Implement and measure
• Optimize
Online • Ongoing measurement and
optimization program
Ecosystem
• Evaluate the business web analytics
maturity level in several key areas
Enterprise • Define a high level roadmap to a more
data-driven eBusiness culture
© 2011 Critical Mass, Inc. All Rights Reserved
33

33. Enterprise Web Analytics Maturity Model Example
6 Analytically Addicted
5 Competitor Management, Adoption
4 Integrated Current State
6
3 Operational
Target State
2 Initiated 5
1 Analytically Unengaged
4
Ideal State
Tools, Technology & Data
3 Objectives
Integration
2
1
0
Continuous Improvement
Scope
Process & Methodology
Team and Expertise
* Stéphane Hamel - http://immeria.net/oamm See Appendix
© 2011 Critical Mass, Inc. All Rights Reserved
34

34. Site and Project Marketing Science Process
2. Pre-Analysis
3. Set Targets /
1. Confirm Goals
Project /
Opportunity
Objectives
5. Record 4. Launch and
Measure
Implementation Training,
Learnings,
Test, Optimize,
Education Socialize
Reporting
& Analysis System
Data Collection Design
System, Tagging
Software
Goals, KPIs,
Strategy Optimization
© 2011 Critical Mass, Inc. All Rights Reserved
35

35. Resources and Analytics Collaboration
We never work in isolation.
Executive
Analytics
Marketing Tech
© 2011 Critical Mass, Inc. All Rights Reserved
36

36. Thank you. Future Topics?
Dan Linton
Marketing Science Director
e danl@criticalmass.com
p 416.673.5308
criticalmass.com
© 2011 Critical Mass, Inc. All Rights Reserved

37. Appendix
© 2011 Critical Mass, Inc. All Rights Reserved
38

38. Links and Resources
Ensighten Tag Management System
http://www.ensighten.com
ObservePoint Audit and Monitoring
http://www.observepoint.com
SiteCatalyst for iPhone
http://itunes.apple.com/us/app/sitecatalyst/id289760802?mt=8
© 2011 Critical Mass, Inc. All Rights Reserved
39

39. Full-Service Digital Capabilities
Insight & Planning Experience Design
Strategic roadmaps and frameworks Brand experience audits
Brand planning Brand definition
Research and usability User experience and interaction design
Persona and scenario modeling & design Design, copy and content services
Information Architecture Content development and classification
Consumer insight development Rich media production
Experience Distribution Technology Services
Social strategy and governance Technology architectures
Video syndication and aggregation Applications, web and mobile development
Community development and social activation eCRM database and profile development
Display and behavioral planning eCommerce personalization and web services
Search engine marketing and optimization Content and digital asset management
Mobile and location-based services Rich Internet application development
eMail and digital campaign management
Program Management
Marketing Science Project and program management
Business analysis and ROI modeling Staff planning and resource management
Digital program KPI development Stakeholder relationship management
Multi-channel analytics Budget and requirements facilitation
Program analysis and optimization planning
© 2011 Critical Mass, Inc. All Rights Reserved
40

40. The WAMM
THE WEB ANALYTICS MATURITY MODEL
A strategic approach based on business maturity and critical success factors - Stéphane Hamel - http://immeria.net/oamm
1. Impaired:
Characterized by the use of out of the box tools & reports, limited resources lacking formal training (hands on skills) and education (knowledge).
Web Analytics is used on an ad-hoc basis and is of limited value and scope. Some tactical objectives are defined but results are not well
communicated and there are multiple versions of the truth.
2. Initiated:
Working with metrics to optimize specific areas of the business (such as marketing or the ecommerce catalogue). Resources are still limited but
the process is getting streamlined. Results are communicated to various business stakeholders (often director level). However, web analytic might
be supporting obsolete business processes and thus, be limited in its ability to push for optimization beyond the online channel. Success is mostly
anecdotal.
3. Operational:
Key Performance Indicators and dashboards are defined and aligned with strategic business objectives. A multidisciplinary team is in place and
use various sources of information such as competitive data, voice of customer and social media or mobile analysis. Metrics are exploited and
explored through segmentation and multivariate testing. The Internet channel is being optimized; personas are being defined. Results start to
appear and be considered at the
executive level. Results are centrally driven, but broadly distributed.
4. Integrated:
Analysts can now correlate online and offline data from various sources to provide a near 360 view of the whole, value chain (see “Limits of web
analytics” note). Optimization encompasses complete processes, including back-end and front-end. Online activities are defined from the user
perspective and persuasion
scenarios are defined. A continuous improvement process and problem solving methodologies are prevalent. Insight and recommendations reach
the CXO level.
5. Competitor:
This level is characterised by several attributes of companies with a strong analytical culture (Davenport and Harris 2007):
a) One or more senior executives strongly advocate fact based decision making and analytics
b) Widespread use of not just descriptive statistics, but predictive modeling and complex optimization techniques
c) Substantial use of analytics across multiple business functions or processe
d) Movement toward an enterprise level approach to managing analytical tools, data, and organizational skills and capabilities.
6. Addicted:
This level matches Davenport’s “Analytical Competitor” characteristics: deep strategic insight, continuous improvement, integrated, skilled
resources, top management commitment, fact-based culture, continuous testing & learning and most important: far beyond the boundaries of the
online channel.
© 2011 Critical Mass, Inc. All Rights Reserved
41

41. Infiniti Global Challenge Solution Results & Insight
As the global presence of Infiniti grew, a strong Critical Mass initiated and developed a One of the key challenges faced was
need for consistent and actionable reporting unique, automated reporting dashboard with a delivering dozens of reports in a timely fashion.
became obvious. In addition to the challenge of 3rd party data warehouse supplier. This This one-of-a-kind reporting interface has
delivering a variety of customized reports to reporting solution integrated with a challenging allowed the CM Marketing Science team to
many individual country regions as well as the variety of Omniture SiteCatalyst report focus on analysis and actionable
central brand showcase in a timely manner, the suites, each requiring regional customization. recommendations. By freeing up this extra
Global team also faced language and cultural This automated solution reduced the “reporting” time, CM has developed a deeper
differences that required a unique analysis hours required for the multitude of country sites understanding not only of the metrics, but also
approach. and allowed the CM team to focus on analysis of various regional language and cultural
and recommendations. It also allows each differences. The result is more actionable
country to access their reports at any time via recommendations and more detailed analysis.
the simplified data warehouse interface.
© 2011 Critical Mass, Inc. All Rights Reserved
42

42. P&G Challenge Solution Results & Insight
P&G’s channel agencies reported results to the We self-initiated and led the collaboration with One of several key findings we found through
client individually, yet none had a six other agencies (Carat, Arc, iProspect, the monthly process, was the need for deeper
comprehensive view of their initiatives (that Merkle, Atos, CRMMetrix) to bring together analysis related to acquisition and engagement.
drove to the website), or understood how they data from banner (Doubleclick), paid search Subsequently, we found some high volume
were part of the digital ecosystem overall. (DART), organic search (Google), survey acquisition campaigns didn’t necessarily result
Critical Mass responded to this situation by (CRMMetrix), email (Epsilon) and consumer in long-term relationships with consumers. The
recommending, creating and implementing a database data (Merkle). Together the group strategy needed to be refined first for quality
measurement program focusing heavily on analyzed performance results and collectively over quantity.
Marketing Mix Analysis where, on a monthly presented to the client on a monthly basis with
basis, engagement and acquisition via paid and recommendations for improvement.
unpaid traffic sources by origination were
analyzed and optimized.
© 2011 Critical Mass, Inc. All Rights Reserved
43

43. Visit Las Vegas Mobile Site Solution Results & Insight
As part of mobile project for Visit Las Critical Mass Marketing Science developed the Critical Mass Marketing Science can now
Vegas, Critical Mass Marketing Science was measurement strategy and created the provide the client with not only how many people
tasked with designing and implementing the necessary tagging. visited the site, but can also provide:
Omniture analytics tagging.
Working with developers, and utilizing -How many visits generated phone calls
A complexity revolved around the fact that leads Omniture’s HTTP based tracking API and - Which properties were called most frequently
were measured in referrals to other sites, in DeviceAnywhere for testing and QA, a full - How many times property details were emailed
addition to all the regular metrics that the client analytics package was deployed on the site that - And of course, how many referral leads were
wished to capture. was capable of tracking lead referrals to dozens generated to the extensive list of
of other sites. accommodation providers
In addition, the mobile site had to provide both
segmented (mobile only) data, as well as roll-up
into the overall Vegas reporting.
© 2011 Critical Mass, Inc. All Rights Reserved
44