We are on the brink of true “in-memory analytics,” a technology that will allow operational data to be held in a single database that can handle all the day-to-day customer transactions and updates as well as analytical requests—in virtually real time. To drive the shift to this new technology, CIOs must make sure the business understands its advantages and devise a governance strategy to manage its rollout and monitor its use.
WordPress Websites for Engineers: Elevate Your Brand
In-Memory Analytics: Strategies for Real-Time CRM
1. Perspective Olaf Acker
Dr. Florian Gröne
Adrian Blockus
Dr. Carsten Bange
3 columns width
2 columns width
In-Memory Analytics
Strategies for
Real-Time CRM
2. Contact Information
Atlanta Canberra Frankfurt New York
Ralph Alewine David Batrouney Olaf Acker Jeffrey Tucker
Partner Principal Partner Partner
+1-404-519-0184 +61-2-6279-1235 +49-69-97167-453 +1-212-551-6653
ralph.alewine@booz.com david.batrouney@booz.com olaf.acker@booz.com jeffrey.tucker@booz.com
Beirut Chicago London São Paulo
Ramez Shehadi Eduardo Alvarez Greg Baxter Jorge Lionel
Partner Partner Partner Principal
+961-1-336433 +1-312-578-4774 +44-20-7393-3795 +55-11-5501-6200
ramez.shehadi@booz.com eduardo.alvarez@booz.com greg.baxter@booz.com jorge.lionel@booz.com
Berlin Delhi Milan Shanghai
Florian Gröne Suvojoy Sengupta Enrico Strada Andrew Cainey
Senior Associate Partner Partner Partner
+49-30-88705-844 +91-124-499-8700 +39-02-72-50-93-00 +86-21-2327-9800
florian.groene@booz.com suvojoy.sengupta@booz.com enrico.strada@booz.com andrew.cainey@booz.com
Adrian Blockus Düsseldorf Munich Sydney
Associate Dietmar Ahlemann Bernhard Rieder Peter Burns
+49-30-88705-896 Partner Partner Partner
adrian.blockus@booz.com +49-211-3890-287 +49-89-54525-670 +61-2-9321-1974
dietmar.ahlemann@booz.com bernhard.rieder@booz.com peter.burns@booz.com
Booz & Company
3. EXECUTIVE For years, the process of devising customer data queries and
creating business intelligence reports has been a lengthy one.
SUMMARY
That’s because the information needed must be pulled from
operational systems and then structured in separate analyti-
cal data warehouse systems that can accept the queries. Now,
however, we are on the brink of true “in-memory analytics,”
a technology that will allow operational data to be held in a
single database that can handle all the day-to-day customer
transactions and updates as well as analytical requests—in
virtually real time.
The advantages of in-memory analyt- and software, companies can cut the
ics are many: Performance gains will total cost of ownership of their cus-
allow business users to retrieve better tomer data efforts significantly.
queries and create more complex
models, allowing them to experiment To drive the shift to this new tech-
more fully with the data in creating nology, CIOs must make sure the
sales and marketing campaigns, and business understands its advantages
to retrieve current customer informa- in terms of better customer intel-
tion, even while on the road, through ligence and lower overall cost. To do
mobile applications. The result- so, they must make a strong business
ing boost in customer insights will case for the transformation—always
give those who move first to these a challenge with business intel-
systems a real competitive advantage. ligence systems—including ease of
Companies whose operations depend use, better analytical reports, and
on frequent data updates will be able better decision making. And they
to run more efficiently. And by merg- must devise a governance strategy to
ing operational and analytical sys- manage the technology’s rollout and
tems, with their attendant hardware monitor its use.
Booz & Company 1
4. ANALYTICS Consider the plight of an insurance
sales rep sitting down with a customer
other products that might suit the
customer’s current needs. As a result,
ON THE FLY to discuss changes to his life insurance the rep has a much better chance of
policy. The only information he has up-selling the customer on his current
about the customer may be months policy and cross-selling him on other
old, so pricing a new policy given the products.
customer’s changing circumstances
will have to wait until the rep can add Until recently, large customer relation-
the new information to the system ship management (CRM) systems
back in the office. And because he depended on two separate databases:
can’t analyze the new data immedi- The operational database maintained
ately, there’s no opportunity to offer the day-to-day, high-volume trans-
the customer specific details on new actional data, while the analytical
products he might be interested in. database took the data needed to per-
form specific customer analyses and
Now, however, a new technology stored it separately. As a result, it was
called “in-memory analytics” lets that impossible to run real-time queries
sales rep enter new data into his com- against the most up-to-date customer
pany’s database from a tablet com- data. Thanks to major advances in
puter sitting on his lap as the meeting the speed, cost, and sophistication of
is taking place. In real time, he can storage and memory technology and
analyze the customer’s new situation in the power of processors, however,
and generate current pricing informa- the promise of real-time analytics—
tion, as well as information about through which business users can
“In-memory analytics” lets a sales
rep enter new data into his company’s
database from a tablet computer on his
lap as he is meeting with a customer.
2 Booz & Company
5. access the full set of operational data formance is dramatically improved. growth through more powerful up-
when creating their reports—is finally This increase in performance allows selling and cross-selling.
becoming a reality (see Exhibit 1). end-users to run more complex que-
ries and gives them better modeling • Lower costs: Total cost of owner-
By giving business users access to truly capabilities, adding up to greater ship is significantly lower compared
live customer data, in-memory tech- business value (see “Inside the to traditional data warehouses, in
nology will transform how companies Technology,” page 4). part because all the data is now
analyze and use that data. As such, it stored in one place. And while in-
offers three significant benefits over • Customer value creation: In- memory technology allows for the
traditional data warehouses: memory analytics gives business analysis of very large data sets, it is
users instant self-service access to much simpler to set up and main-
• Performance improvements: the information they need, pro- tain. Rapid departmental deploy-
Because users can now interact viding an entirely new level of ments can free up IT resources
with and query data in memory, customer insight that has the very previously devoted to responding to
response time and calculation per- real potential to maximize revenue requests for reports.
Exhibit 1
An Integrated CRM Architecture Can Speed Up Analytics Requests
TODAY THE FUTURE
Analytical CRM Operational CRM Real-Time CRM
Transactional
data,
analytics User Interface
Mining requests Campaigns
Campaigns Sales Service
Segmentation Sales
Real time Real time
Decision Models Customer Care In-Memory Analytics
Batch
scores,
models
Source: Booz & Company analysis
Booz & Company 3
6. Inside the Technology
Until very recently, the effort to create, store, and analyze critical
transactional data related to all kinds of business activities was a
cumbersome and expensive process. Operational data—the high-
volume, transaction-heavy data generated through a variety of business
processes, including sales, order management, and customer care
operations such as call centers—was maintained in huge data
warehouses to ensure reliable performance and data integrity. And
because the sources of the operational data typically varied significantly,
maintaining it all in a single database with the homogeneous data model
that could serve as a “single point of truth” proved very beneficial.
Meanwhile, the analytical data used to gather customer and performance
insights, to segment customers, and to model and predict future behavior
through customer usage and payment history, for example, was typically
drawn periodically from the operational database and maintained
separately. As valuable as that analytical data proved in boosting
customer profitability and allowing more efficient up-selling and cross-
selling, the architecture had some very real downsides.
Because it had to be duplicated periodically, the data in the analytical data
warehouse was frequently at least a day or two—and sometimes as much
as a month—out of date. A specially designed data mart had to be built
for practically every new analysis request, which meant long deployment
cycles, low project success rates, and ever-growing data volumes at ever-
higher cost. And the process introduced an additional layer of complex
analytical software into the enterprise architecture, requiring additional
training. Typical business users could only generate pre-defined standard
analytical reports; anything more complex needed to be set up by a
handful of power users.
Now, however, this long-time separation between operational and
analytical databases is finally coming to an end. With the emergence of
multicore processors, increasing clock speeds, and 64-bit technology,
combined with the rapid decrease in the price of memory, data can
be managed entirely in main memory. While the idea of managing
data in memory is not new, efforts to do so were hampered until
recently by the fact that the old 32-bit architecture could address only
4 gigabytes of memory, and processors were not fast enough to give
in-memory databases any real performance advantage. But with new
4 Booz & Company
7. ways of organizing, buffering, and accessing the data, the performance
improvements are significant (see Exhibit A).
The capacity capabilities of these systems are now equaling those
of large disk-based databases. For example, a pilot implementation
of a 40-terabyte in-memory database was recently completed, and
theoretically, databases as large as 16 exabytes (16,384 petabytes) could
be managed with in-memory technology, based on today’s architecture.
Throughput is seven times higher, and the response time is virtually Guidelines:
instantaneous.
11.0 million
aölkdfölka
Exhibit A 32.8%
In-Memory Technology Offers Vastly Superior Response Time and Throughput
30.1%
RESPONSE TIME THROUGHPUT TABLE HEADI
(IN MICROSECONDS) (IN THOUSANDS OF
TRANSACTIONS PER SECOND) A4 format:
- width for 3 colu
5x 7x
- width for 2 colu
700 140 Letter format:
- width for 3 colu
19x
600 120 - width for 2 colu
500 100 Lines: 0,5 pt
On-Disk
Lines for legend:
Database
400 80
In-Memory
300 60 Database Note:
Please always de
otherwise InDesi
200 40 file.
These colors can
100 20
Approved Colo
0 0
Update Select
Source: Booz & Company analysis
3 columns width
2 columns width
Booz & Company 5
8. DRIVING Several factors—involving improved
analytical speed and performance
mobility is becoming the norm.
In every industry, customers now
FACTORS and better analytical results—are expect instant responses to their
driving the push to in-memory requests and questions; in this
technology at the enterprise level. environment, in-memory technology
allows companies to create an
Business Demand entirely new level of customer
The delays that typically arise experience, and it gives users instant
out of the periodic extraction, access to the data they need to
transformation, and loading (ETL) provide online self-service, real-time
of data from the operational to the customer segmentation and dynamic
analytical systems may be generally pricing.
acceptable in doing trend analysis
and forecasting. But traditional data Time-sensitive industries like
warehouses simply cannot keep pace airlines and transport logistics
with today’s business requirements will now have access to real-time
for fast and accurate analytical information in running their
data, especially in situations where operations, and the resulting
Traditional data warehouses
simply cannot keep pace with
today’s business requirements for
fast and accurate analytical data.
6 Booz & Company
9. increase in efficiency will become interactive data visualization response times, as users now
a significant competitive advantage as the end-user interface, which expect instantaneous results.
(see Exhibit 2). allows many more people in Since in-memory analytics allows
the organization to make use data to be accessed directly from
Performance of Analytics of these systems. However, the memory, query results come back
Most analytical applications have new interfaces, which offer users much more quickly than they
moved beyond the spreadsheets interactive dashboards and the would from a traditional disk-
and tables offered by traditional ability to perform much more based data warehouse. The time it
reporting tools and now use intuitive tasks, demand very fast takes to update the database is also
Exhibit 2
Selected Benefits of Real-Time Analytics Across Different Industries
Finance ILLUSTRATIVE EXAMPLES
Automated trading,
online banking,
huge volumes
Defense Travel
Just-in-time Exploding
people tracking “look-to-book”
ratios
Real-Time Business
- Optimization
- Operations
Telecoms - CRM Online Gaming
- BI
Value-add services, User authentication
billing, subscriber & authorization,
database consolidation, credit check,
fraud management Retail best bets
Multichannel selling,
cross-selling/
up-selling,
personalization
Source: Booz & Company analysis
Booz & Company 7
10. significantly reduced, and the system and customer data put the onus volumes and the proliferation
can handle many more queries (see on organizations to maintain this of applications dependent
Exhibit 3). data and keep it available for years. on databases, companies are
Much of this data still resides on struggling to manage the many
Growing Data Volumes legacy systems, which are costly to business intelligence (BI) efforts
The sheer amount of transaction operate and maintain. In-memory being developed throughout their
data being digitally captured and analytics allows such data to be organizations. In many cases, for
stored is increasing exponentially, accessed rapidly on an ad hoc basis, instance, users simply want access
as are unstructured forms of data without having to build additional to their specific transactional
such as e-mail, video, and graphics. complex data marts and load data systems for reporting and analysis,
According to one estimate, 0.8 into them. Instead, these systems without the need to deploy a full
zettabyte of data was produced allow users to connect to legacy data data mart. In-memory analytics
in 2009—if a gigabyte were the stores, populate an ad hoc database, removes the need to build complex
size of a sesame seed, a zettabyte conduct the analysis, and then performance layers such as
would equal the diameter of the discard the in-memory database multidimensional cubes within the
sun—and that is expected to rise to once the analysis is complete. data warehouse; instead, users can
Guidelines:
35 zettabytes by 2020. At the same run their analytical applications
11.0 million =
time, tighter regulations involving Speed of Deployment directly against an in-memory
the tracking of financial transactions Given the rapid growth of data performance layer.
aölkdfölka =
32.8% =
30.1% =
Exhibit 3
Performance Comparison of Different Database Types
TABLE HEADINGS
RESPONSE TIME A4 format:
- width for 3 columns
- width for 2 columns
Microseconds In-Memory Databases
Letter format:
- width for 3 columns
- width for 2 columns
Milliseconds Lines: 0,5 pt
Lines for legend: 0,5
Disk-Based Databases Disk-Based, Memory-Cached Databases
Note:
Seconds Please always delete
otherwise InDesign w
100 1,000 10,000 100,000 file.
These colors can’t be
Throughput (transactions per second)
Approved Colors, T
Source: Booz & Company analysis
8 Booz & Company
11. GREATER In addition to the real gains in
performance and speed offered by
from the enterprise architecture,
reducing complexity and the infra-
INTELLIGENCE in-memory analytics, these new structure the traditional systems
systems can significantly improve the required. Furthermore, the source
quality of the business and customer data has to be created or populated
intelligence they generate. And they only once and then is immediately
can transform how that intelligence is available for any kind of analysis.
delivered, and to whom. The benefits Consequently, organizations can
include the following: operate at a higher level of perfor-
mance, deliver more reports per
• Improved decision making: The hour, and free up capacity on the
ease of use of in-memory technol- source systems for other opera-
ogy allows anyone in the organi- tional purposes.
zation, from business analysts to
managers, to build their own que- • Self-service business intelligence:
ries and dashboards with very little In-memory analytics allows any
technical expertise. Control over user to easily carve out subsets
critical data shifts away from those of the enterprise business intelli-
who manage it to the stakeholders gence environment for convenient
who own and use it, allowing them departmental usage. Work groups
to make better business decisions. can operate autonomously without
affecting the central data warehouse
• Richer insight: The significantly workload. In addition, in-memory
greater processing speed and calcu- technology enables a much greater
lation performance of in-memory degree of ad hoc analysis within the
technology lets end-users develop organization and allows users to
richer, more complex models, source data rapidly, build analytical
enabling better customer segmenta- applications, and conduct specific
tion and more powerful campaign investigations. Once the analysis
planning in the CRM space, for is no longer required, it can be
example. The result is significantly disposed of easily. Quick response
greater business value for the times and strong visual interfaces
system as a whole. also enable mobile BI applications,
which can be used by salespeople to
• Increased efficiency: Converting to gain a complete view of customers,
in-memory technology as a plat- based on real-time customer sales
form for analysis allows a whole data, while on the road.
technological layer to be removed
In-memory technology enables
a much greater degree of ad hoc
analysis within the organization.
Booz & Company 9
12. STEPS FOR The virtues of in-memory analytics are
many, but as with any new technol-
to relying entirely on the IT depart-
ment to perform its “data magic in
THE CIO ogy, it is the responsibility of the CIO the basement,” a process that can
to make these virtues clear both to top take days. Moreover, users frequently
management and to business users. avoid using traditional BI tools
Most important, in-memory analyt- because of their inherent complexity
ics must be seen as part of a broader and difficulty. With in-memory analyt-
BI strategy that takes into account ics, as we have seen, response time can
its overall business value and the be virtually instantaneous, and users
underlying technology architecture, have the ability to design their own
while remaining aware of the chal- queries. It is critical to make clear to
lenges inherent in every major new the business that a significant portion
technology. of the value of in-memory technol-
ogy lies in its ability to open up these
The top priority is to educate the busi- bottlenecks and offer users greater
ness as to the value and advantages access to fresh data and increased
of in-memory analytics, as well as to query flexibility.
the costs and risks involved. In many
organizations, business users of ana- As part of the education process, CIOs
lytical tools have grown accustomed should identify and point out particu-
A significant portion of the value of
in-memory technology lies in its ability
to open up bottlenecks and offer users
greater access to fresh data.
10 Booz & Company
13. larly valuable business opportunities old warehouses as they implement ture is critical. Don’t try to convert the
that in-memory technology offers. and test the new system, an additional entire architecture to in-memory tech-
These might include the ability to go up-front cost that must be taken into nology all at once. Instead, develop a
beyond traditional BI reports to create account. It will also be necessary to thorough investment road map that
powerful applications such as what-if conduct a thorough cleansing of all includes both a plan for incorporating
analyses, interactive filtering, and pat- customer data to avoid contaminating in-memory technology in the stan-
tern discovery, all in an easy, intuitive the new system with bad information. dard architecture when possible—in
fashion. These capabilities should be order to prevent business units from
actively promoted in order to foster Once such systems are installed, most adopting it as part of a “rogue” IT
a high-performance decision-making companies will struggle to calcu- effort—and a strategy for switching
culture. But to accomplish this, many late the tangible cost-of-ownership over to the in-memory technology on
of the organizational processes with benefits, such as overall infrastructure a department-by-department basis.
which both business users and the IT savings or lower administrative labor
department are familiar will need to costs. In order to build a better busi- Building a governance strategy that
be rethought—an effort that, like any ness case, CIOs must gain an in-depth can effectively manage the potential
major change, must be planned and understanding of the different types explosion in the number of analyti-
executed carefully. of BI applications and user segments cal applications is essential. Such a
involved, as well as the extent of the strategy should include an inven-
Calculating the business case for any administrative maintenance effort tory of analytical applications that
BI effort, traditional or otherwise, required. In-memory analytics can clearly defines owners and use cases
is difficult. The new technology will then be better integrated into the over- and that can serve as the basis for a
require a significant up-front invest- all BI tool strategy and positioned to wider rollout of in-memory analyt-
ment in new storage hardware and either replace or complement current ics throughout the organization. An
software, and in the training needed BI solutions. established “BI competence center”
for both the business and IT staffs with the authority to drive standard-
to make the best use of it. Moreover, The proper role of in-memory analyt- ization and exercise governance will
companies will need to maintain their ics in a company’s overall BI architec- be invaluable.
Booz & Company 11
14. HIGHLIGHTS
• End-users consistently rank
slow query performance
as among the top three
concerns that affect their
perception of the value
of business intelligence
systems.
• In-memory analytics offers
a vast improvement in
process speed, query quality,
and customer insight over
traditional operational/
analytical customer data
warehouse systems.
• The shift to in-memory
Conclusion By combining operational and
analytical databases into a single
technology will be driven by instantly available warehouse,
demand from the business in-memory analytics will give
for real-time customer and business users access to a whole
operational information. new realm of crucial customer
information, transform how they
• CIOs must develop a
use that information, and thus give
strong business case for
them a real competitive advantage
implementing in-memory
in the race to gain better customer
analytics, including the new
insights more quickly. As with any
business opportunities it will
business intelligence effort, however,
enable and its advantage
the new technology’s virtues must
in terms of total cost of
be sold to business users, and its use
ownership.
must be monitored and managed
carefully to ensure that all users are
getting the most out of it.
12 Booz & Company
15. Resources
“The BI Survey 9,” Business Application Research Center, 2010.
www.bi-survey.com
“Not Your Typical Marketing Campaign: The Next Wave of
Technology-Driven Marketing,” Booz & Company, 2009. www.
booz.com/media/uploads/ITFS-Not_Your_Typical_Marketing_
Campaign.pdf
“Loyalty by Numbers: An Integrated Approach for Telecom
Companies,” Booz & Company, 2010. www.booz.com/media/
uploads/Loyalty_by_Numbers.pdf
About the Authors
Olaf Acker is a partner in Adrian Blockus is an associate
Booz & Company’s Frankfurt with Booz & Company based
and Dubai offices. He focuses in Berlin. He specializes in IT,
on business technology technology, and strategy trans-
strategy and transformation formation programs for telecom
programs for global companies and technology companies. He
in the telecommunications, is an expert in business intel-
media, and high technology ligence and data warehousing.
industries.
Dr. Carsten Bange is the
Dr. Florian Gröne is a founder and managing director
senior associate with of the Business Application
Booz & Company in Berlin. He Research Center (BARC), the
works with communications, leading market analyst for busi-
media, and technology industry ness intelligence and data man-
players on defining their go-to- agement in central Europe. He
market strategies and operating consults regularly on business
models and on transforming intelligence and data manage-
customer-facing processes ment strategy, architecture, and
and enabling technologies. He technology selection.
leads the firm’s CRM Center of
Excellence in Europe.
Booz & Company 13