EBook On Dealing With Big Data with Esteban Kolsky, Brent Leary, Tyson Hartman and myself

1. The Broken Data Promise:
How CRM Failed, and Why Businesses Need It More Than Ever
Brought to you by
Sponsored by
© 2011, Social Media Today, LLC

2. FOREWORD 1 2 3 4 CONCLUSIONS
The Broken Data Promise:
How CRM Failed, and Why Businesses Need It More Than Ever
3 Foreword
4 Customer 360°
Esteban Kolsky
6 What Customers Want
Brent Leary
9 The Rise of Analytical CRM
Mark Tamis
12 Big Data, Big Problems
Tyson Hartman
14 Conclusions
15 Author Bios
16 About Us

3. FOREWORD 1 2 3 4 CONCLUSIONS
Foreword
During the 1990s and early 2000s, the so-called golden years We will take a two-pronged approach. First, we’ll explore
of CRM deployment, CRM vendors made a promise to their what is necessary to achieve a holistic view of our customers,
clients: If organizations bought and implemented complete what data must be collected, and how that data can be
CRM suites; they’d be rewarded with 360º portraits of their used. We’ll also examine the benefits of this approach, and
customers. These portraits would be generated by the large look at case studies to better understand why organizations
quantity of transactional and operational data that CRM must meet the data challenge.
solutions produce. Vendors claimed that by gathering all
data about all interactions between the organization and its In the second part, we’ll explain how to monitor and store
customers, companies could then leverage analytical tools the data needed to create useful customer profiles, and
within the suite to build deep, meaningful relationships leverage those profiles in different functions. Finally, we’ll
with customers. present a case study of one organization that successfully
used a CRM system to profile its customers.
This became the 360º view of the customer promise. It was
never kept. It’s not because we didn’t try. We kept detailed
records of all interactions, all transactions, everything the
customer did and said. We gathered more information on
customers during the last 10 years than we had since the
invention of data collection.
Despite all this data, we still can’t begin to understand what
our customers truly want or need. Worse, we can’t use the
data we collect to improve our customer relationships—one
of the core goals behind the CRM data promise.
How is that possible? How could we fail so completely in
the single most important promise made by the most critical
piece of front office software to be released in our lifetimes?
This eBook will explore the answer to those questions.
334

4. FOREWORD 1 2 3 4 CONCLUSIONS
1 Customer 360°
Esteban Kolsky
It may seem obvious, but still needs to be said: This is accurate predictions about behavior within our segments.
all about data. If you want to create a complete profile Nevertheless, all this operational and transactional data is
of any customer segment, you need to collect, store and still being stored today, and is still being used as a partial
analyze lots of data. The data we need falls into four predictor of future behavior.
main categories:
Attitudinal – This is the missing behavioral link.
Demographic – this is what we traditionally think of when Organizations tend to collect and store behavioral data
we talk about customer data. The core data points are name, from their perspective: What is the customer doing and
address, and phone number, but we also retain gender, age when? But these questions don’t capture the reasons
group, education status, income, race, and similar data why customers do what they do, because organizations
that will help us classify customers in different segments. don’t see the world from the customer’s perspective.
One such group might be males aged 18 to 24 years living The point of gathering attitudinal data is to close this
in New York state. Initially we believed that members of gap by asking customers why they would buy a product.
these segments would all behave in the same way, but we What circumstances determine the attitude that drives
found that isn’t always the case. Nevertheless, gathering the behavior? We collect this type of data via surveys
basic demographic data still helps us identify customers for and other feedback events, which normally include
different purposes. satisfaction questions.
Behavioral – The promise of CRM systems was that if Sentimental – Sentimental data refers to the emotions and
we retained and analyzed sufficient transactional and feelings that a customer has towards the organization, its
operational data about customers, we could determine products, and the relationship as a whole. It has traditionally
how they behaved and make predictions about their future been materialized in metrics like satisfaction, loyalty and
behavior. If certain males aged 18 to 24 living in New York advocacy. Sentimental data can only be captured by direct
state perform a specific action at a specific time, we can feedback, and can never be inferred from other metrics.
infer that the rest of the group will behave in a similar way. This was one of the biggest pitfalls of the original CRM
Thus, when a 19-year-old male New Yorker interacts with promise: The original systems tried to guess sentiment by
us, we can offer him a particular product with a certain analyzing behaviors, yielding poor or erroneous data.
degree of confidence. Later, of course, we discovered that Continued on next page
we were missing other core data points needed to make
344

5. FOREWORD 1 2 3 4 CONCLUSIONS
Customer 360° (cont’d)
In short, a complete customer portrait would tell the
organization who the customers are (demographics), what
they do (behavioral), what they want (attitudinal), and
why they want it (sentimental). Such profiles allow an
organization to tailor solutions, products, services, and
interactions to what its customers are looking for.
354

6. FOREWORD 1 2 3 4 CONCLUSIONS
2 What Customers Want
Brent Leary
About a decade ago, Mel Gibson starred as marketing their concerns, likes and dislikes freely via social channels.
executive Nick Marshall in a movie called What Women Want. We can pick up extremely useful insights by knowing where
Nick thinks he can use his charm, powers of persuasion, and to listen, what to listen to (and for), and, maybe most
especially his perception of feminine desires in order to important, how to listen.
land a major sportswear retailer as a client for his firm. But
he never tries to understand what’s important to women The tools for listening and engaging are already plentiful,
until he gets passed over for a major promotion. and will become easier and easier to use as time goes by.
But we still need a strategy for collecting and analyzing
Shocked, Nick immerses himself in trying to get inside what we hear, so that we can translate it into solutions
the minds of his customers—not because he really cares that solve the challenges our customers and prospects face.
what women want, but to prove that he shouldn’t have While listening to our customers and analyzing what they
been ignored. say, we are also creating meaningful interactions with them
that lead ultimately to stronger relationships.
After adjusting to his new powers, Nick starts exploiting
what he hears for personal gain. He eventually realizes The more active our customers are on Facebook, Twitter
that misusing his new power is an overall negative, and other social networks, the more data they create.
so he starts listening in order to really understand women, This presents an opportunity to better understand and
and not just to validate his preconceived notions. Having engage our customers. It also challenges us to integrate
changed his own thought processes, he finally learns how this information with transaction data, activity data and
to care about women’s needs and concerns, which helps other information that adds up to a layered customer
him connect with the audience that he originally took portrait. While the challenges are not trivial, the payoff
for granted. can be substantial.
While this is only a movie—and I sincerely hope there will Lisa Larson, director of customer care at online pharmacy
never come a time where people can hear what’s going Drugstore.com, recently shared her experiences integrating
on in my head—we can learn valuable lessons from Nick’s social channels in customer service. Here’s what Lisa had
transformation. First, it’s more important than ever to to say about the importance of listening to and analyzing
understand what our customers are thinking. Fortunately the social footprints that customers leave.
we don’t need Nick’s extrasensory ability. Customers share Continued on next page
364

7. FOREWORD 1 2 3 4 CONCLUSIONS
What Customers Want (cont’d)
People who aren’t looking at this are missing a key part of These are the kind of results that might make it easier
their business. You learn the most from just listening to to understand the benefits of leveraging social tools to
your customers. Years ago, you would have paid amazing listen, analyze, and engage with customers. But don’t
amounts of money to get this kind of information. Now forget Nick—you need to be genuinely interested in
it’s free and right there for all of us, we just have to go understanding your customers, and not just interested in
listen and find it. It’s amazing, the difference. These what they can do for you.
are really honest conversations that you can listen to
and learn from, and possibly join in. You have to decide
what the best opportunity is for your company. You are
missing out if you are not involved. Besides, it’s fun!
In addition to being fun, social media interaction can help
your top and bottom lines. For example, interacting with
customers via Twitter and other chat technologies yielded
the following results for Drugstore.com:
• The overall phone time that Drugstore.com devoted to
customer interaction decreased by 15 percent. E-mail
volume shrank by 30 percent. Meanwhile shopping-cart
sizes in sales facilitated by chat are now 10 percent to
20 percent larger compared to sales without chat.
• Chat sessions deliver a conversion rate of approximately
25 percent; the site’s overall conversion rate is just
6.4 percent
• Third-quarter 2010 sales grew 17 percent, compared to
2 percent growth in e-commerce overall
• Customer satisfaction scores reached 77 on ForeSee
Results’s list of the top 15 online retailers
374

8. FOREWORD 1 2 3 4 CONCLUSIONS
COMMENT BY ESTEBAN KOLSKY
Brent makes a couple of very interesting points that are 20 and 100 times larger than structured data volumes.
worth exploring in more detail. First, he talks about the need Unstructured data complements structured data; it doesn’t
to listen. Customers give organizations lots of information, replace it. So organizations must match their new data to
but it typically doesn’t take the form of perfectly structured existing data about customers and their experiences, and
data. This was the error in the previous methods that tried integrate all the data sources to obtain more comprehensive
to generate 360º customer profiles: We relied on structured views of their customers.
data provided by transactions and interactions. We assumed
that customers who behaved in a certain way once would do The problem is that these gigantic data volumes are
the same thing again in a similar situation. cumbersome to manage. Organizations have limited capacity
and will to parse data and create actionable insights from
The problem is that we never bothered to ask why our them. This was the problem that created the second evolution
customers behaved in particular ways or what their needs of CRM software: analytical CRM.
were. We simply assumed that their actions told us everything
we needed to know. This was our great error. We can only
correct that error by collecting unstructured data, analyzing
them, and creating actionable insights from them.
Listening is the first step. Organizations have always known
how to create surveys (some good, some awful, most in-
between), distribute them, and collect the data that they
produce. With the advent of social networks and social
channels, we finally found the source for the unstructured
data that would complete the thoughts started by traditional
structured feedback events such as surveys and focus groups.
For better or worse, both methods generate lots of data.
Today, unstructured data volumes collected from customer
interactions and transactions are estimated to be between
384

9. FOREWORD 1 2 3 4 CONCLUSIONS
3 The Rise of Analytical CRM
Mark Tamis
Never before in history have we been able to gather Although these tools have their merits, they don’t trace
and store so much data about customers. We start with connections between the various data points that we have
contact information, purchases, support interactions, leads about a given customer. Nor do they help us decide how
and opportunities. The social web allows us to capture we should respond to that customer, or even whether we
data about site navigation and the like. And now we’re should respond at all.
adding even more data points: Facebook Likes, Twitter
microblogging, and online customer communities. To state the obvious, business decisions should be guided
by customer data and analysis. Although the sheer volume
So how do we turn vast data volumes into actionable of social data is daunting and the tools far from perfect,
insights that are relevant both to our organizations and we should try to use these data to enhance what we’re
to the customers we are trying to serve? There’s still a already doing with transactional data. Each customer’s
disconnect between data captured by CRM systems and cross-channel activity will need to be captured and blended
behavioral data that we capture through social media into customer “snapshots,” building on the historical and
channels as well as traditional channels such as email, transactional content of the CRM system.
surveys and interaction with customer service reps.
Step one is finding identifiers that link customers to their
The data sets captured in the different channels are hardly identities on the Social Web. Ideally this would happen
ever correlated effectively with data from other sources. We through an opt-in procedure, for example when the
seem satisfied to track Facebook fans and Twitter followers customer visits your community support forum and fills out
without following through to see whether all these fans her profile, sends in her warranty card or signs up for her
and followers were already customers or if they actually loyalty card.
bought from us after stating their interest. Nor do we try
to capture how fans and followers influence others in their The next step is to mine and analyze relevant interactions
social networks and how this influence affects our brand that we can match with a persona. Micro-segmentation can
images and sales. give us insight into the experiences that our customers
expect and suggest appropriate responses.
New analytic tools such as Radian6, Attensity and Lithium
try to organize unstructured data by using clever filtering This matching can be based on personality analytics,
to automatically or manually create CRM system entries. sentiment evolution, social and interest graph
Continued on next page
394

10. FOREWORD 1 2 3 4 CONCLUSIONS
The Rise of Analytical CRM (cont’d)
segmentation, product portfolio, issue anticipation, and so
on. The analysis should help us make business decisions
regarding the desirability of the customer.
Social data also create opportunities for predictive analysis,
due to our real-time access to the customer’s voice. As they
say, the best customer service is no service!
You also need to consider where your data are stored.
Increasingly this is done via data centers managed by
third-party infrastructure providers, otherwise known as
“the cloud.” Contrary to popular belief, the real promise
of cloud computing is not the ability to outsource your
IT management or access applications and data from
anywhere. Rather, it’s the ability to quickly connect your
datasets to those of your partners, suppliers, and channels,
and mine the collective data for customer insights that you
would miss if you were only looking at your own data.
The Social Web has given us many new ways to fine-
tune customer information. Going forward, the main
challenge will be linking customer identities across
different interaction channels and blending structured and
unstructured data into clean datasets that we can mine for
actionable insights.
34
10

11. FOREWORD 1 2 3 4 CONCLUSIONS
COMMENT BY ESTEBAN KOLSKY
Mark’s vision underscores the fact that we gather social 5. Iterate. Analytics is not an end game. New technologies
and other data about our customers in order to generate and tools allow organizations to take on new challenges
actionable insights. This should be the goal of any company and gain a better understanding of what they are after.
that analyzes the data they collect from their customers Make iteration a core part of your strategy.
and their operations. It’s the main reason to invest in
feedback and data management initiatives. But today, most Unfortunately, these steps don’t guarantee success. Tool
organizations that deploy data analytics seem to believe interfaces may be getting simpler, but pretty screens also
that actionable insights arise from some magic strike, lucky hide the true complexity of analytics. Talented analysts are
guess, or black box method. still the most critical component in analytics—and they
are very hard to come by. If you want to succeed at the
The five keys to effective data analysis are: game of analytics, either hire the expertise you need or train
committed individuals to extract vital insights from the sea
1. Always know what you are seeking. Diving into a of data in which all businesses swim.
Big Data set “just to see what’s there” will only
yield frustration.
2. Understand what you have. To achieve useful results,
it is critical that you understand what the data are,
what they say, how they flow through the systems, how
they are used by the organization, and how they relate
to other data points.
3. Correlate to KPI. Data insights must be correlated
with your organization’s key performance indicators.
Analytics must articulate with past performance
issues and future needs.
4. Define Actions. You can’t have actionable insights
without actions. You must know what should happen
when data processing yields a value that falls above
or below expectations.
34
11

12. FOREWORD 1 2 3 4 CONCLUSIONS
4 Big Data, Big Problems
Tyson Hartman
In the global marketplace, businesses and employees are puts even more pressure on executives to consume even
creating and consuming more information than ever before. more information. Which begs a question: Are executives
Gartner predicts that enterprise data in all forms will grow addicted to data? The following data points would suggest
by 650 percent over the next five years, while IDC claims that the answer is yes.
that global data volumes double every 18 months.
• 70 percent of business leaders report that their current
According to “The Business Impact of Big Data,” a new IT infrastructure allows employees to get the data they
global survey of C-level executives and IT decision makers need at the speed they need it.
commissioned by Avanade, this data deluge is creating very • 61 percent of executives still want faster access.
real challenges for business leaders. • One in three say they need even more sources of data
in order to perform their job better.
Big Data – Hype or Reality?
Across industries, regions and companies, executives report For many, this data addiction is driven by the inability to
that the exponential growth in data is degrading their ability find the information they need. In fact, a recent survey
to access critical information. According to the report, 56 found that during the recent recession, more than one-
percent of business and IT executives feel overwhelmed by quarter of executives lost business because they couldn’t
the amount of data their company manages. Many report access the right information. This dearth of accurate
that important decisions are often delayed because they information pushes executives to continuously search for
have too much information. better information, creating an addictive behavior pattern.
Despite these challenges, executives see some value in the So what kind of information are executives most concerned
data deluge. For instance, 61 percent believe that the flood about? According to the survey, their top priority is the
of data entering the enterprise fundamentally changes the ability to keep up with customer-service expectations.
way their business operates. And when it comes to perceptions of the most data
categories, customer information leads the pack. This focus
Data Addiction on customers is driving technology investments in CRM
Although the onslaught of data can make it more difficult systems—67 percent of executives have already invested
for executives to make decisions, they are still asking for in CRM or are seriously considering doing so over the next
more data, and they want it even faster. This desperation 12 months.
for the right information to make business decisions Continued on next page
34
12

13. FOREWORD 1 2 3 4 CONCLUSIONS
Big Data, Big Problems (cont’d)
Executives are recognizing the opportunity to leverage
their customer data in order to create new revenue streams
and generate new business. Alarmingly, however, fewer
than half of all managers view the available sources of
data as strategic differentiators for their organizations.
They struggle to understand how Big Data can drive real
business value.
Big Data, Big Value
So how do we get from where we are today to where we
want to be? First, companies must develop a “data culture”
in which executives, employees, and strategic partners are
active participants in managing a meaningful data lifecycle.
Companies need to start educating their employees on how
to best participate in this process.
This is not just a technology challenge. It’s also a people
and process problem. It takes a culture shift among the
people who are interacting with the data—whether they
are producing or consuming—to be more accountable for
data management.
Tomorrow’s successful organizations will be equipped to
harness new sources of information and take responsibility
for accurate data creation and maintenance. This will
enable businesses to turn data into usable information first
and then ultimately into true business insights.
34
13

14. FOREWORD 1 2 3 4 CONCLUSIONS
Conclusions
“It is imperative that companies So what have we learned? Not only did we lack the necessary The original CRM vision failed due to the lack of sufficient
develop a ‘data culture’ in data to understand our customers holistically, we also lacked data and processing ability for the data that existed. We are
which executives, employees, the operational capacity to manage and learn from the Big solving those issues today, but CRM is still not an automatic
and strategic partners are active Data sets that we created. How should organizations deal solution.
participants in managing with Big Data? There’s a vast literature of attempts to answer
a meaningful data lifecycle” this question, but the following three steps are crucial. Three core areas need to be explored in more detail:
– Tyson Hartman
1. Recognize. You must recognize that your current systems, Listening. Listen to the customer’s needs and desires through
analytic engines, databases, and potentially your architecture direct, structured feedback and interactions. This is the first
are not prepared to handle the deluge of Big Data. Trying to step towards discovery of the necessary data.
accommodate an aging and inappropriate infrastructure is a
recipe for failure. Big Data. Unstructured datasets are 20 to 100 times larger
in volume than structured datasets. The new social datasets
2. Plan. To accommodate slow growth as opposed to a must be understood and then articulated with existing data
landslide of data clobbering your systems, figure out what to create blended datasets that can provide the insights we
data will be coming from what channels and create a plan need.
to accommodate the various data streams. Once you have
the first data stream under control, focus on the second Analytics and Actionable Insights. Analytics are not, as
and third, and so on. Plan for a gradual assimilation of the we used to believe, about understanding the relationship
magnitude of data you will receive. between data and data elements. We need to build an
analytical model that produces actionable insight into what
3. Learn By Doing. You are now dealing with a lot of issues our customers need and desire.
that your organization did not have to deal with before.
Is it better to store all data and eventually get around Organizations that embark on this journey will be trailblazers,
to analyzing it? Or would we be better off simply doing building a repertoire of best practices and lessons learned
real-time analytics and storing the results? How much more than relying on others. Some will reap the reward of
accuracy is required, and how can we achieve it? You can’t nearly perfect knowledge about their customers.
answer these questions until you’ve been confronted with
them. Resolve to learn as you go and constantly improve Will your company be one of them?
your implementation.
34
14

15. FOREWORD 1 2 3 4 CONCLUSIONS
Author Bios
Esteban Kolsky is the principal and founder of ThinkJar,
an advisory and research think tank focused on
customer strategies.
Brent Leary is co-founder and partner at CRM Essentials
LLC, a CRM consulting/advisory firm focused on small and
mid-size enterprises.
Mark Tamis is a noted blogger on social CRM with
considerable experience in enterprise software.
Tyson Hartman holds the title of Avanade Fellow
at Avanade, a Seattle-based technology solutions provider.
In this role, Hartman works with the senior technology
team to define the vision and road map of Avanade’s
solution development practices.
34
15

16. FOREWORD 1 2 3 4 CONCLUSIONS
About Us
SmartData Collective, an online community moderated
by Social Media Today, provides enterprise leaders access
to the latest trends in Business Intelligence and Data
Management. Our innovative model serves as a platform
for recognized, global experts to share their insights
through peer contributions, custom content publishing and
alignment with industry leaders. SmartData Collective is a
key resource for executives who need to make informed
data management decisions.
About Our Sponsor
Teradata is the world’s largest company solely focused
on creating enterprise agility through database software,
enterprise data warehousing, data warehouse appliances,
and analytics. They deliver award-winning, integrated,
purpose-built platforms based on the most powerful,
scalable, and reliable technology platform in the industry,
with assets including:
• Approximately 7,400 associates in more than 42 countries
• Strong diversified client base of over 900 customers
worldwide and companies of all sizes
• 2,000+ implementations worldwide
316