Splunk Business Analytics Application

1. Business
Analy,cs
Paradigm
Change
Dmitry
Anoshin

2. Target
Market
Trends
• “Feeding
transac,onal
data
into
a
tradi,onal
data
warehouse
no
longer
represents
the
extent
of
capabili,es
necessary
for
BI.”
• “The
simple
idea
of
building
a
tradi,onal
data
warehouse
to
support
a
BI
plaEorm
is
no
longer
sufficient.”
• “….require
new
informa,on
management
capabili,es
to
integrate
informa,on
from
disparate,
external
and
unstructured
informa,on
sources.”

3. Tradi,onal
Analy,cs
Types:
• Business
Intelligence
• Data
mining
• OLAP
• Plain
Analy,cs
Uses:
• Get
beNer
sense
of
their
opera,ons
• Cut
costs
• Improve
decision
making
• Iden,fy
inefficient
processes,
which
can
lead
to
iden,fy
new
business
opportuni,es
and
reengineering
their
processes
Challenges:
• Raw
informa,on
lives
are
usually
decoupled
or
spread
across
distributed
systems
• Difficult
to
consolidate
• Involves
an
effort
going
through
the
typical
SDLC,
which
takes
lots
of
,me

4. Typical
Process
for
Structured
Data
Applica,on
Applica,on
Applica,on
Connector
Data
base
ETL
Data
Warehouse
Analy,cs
Tool
Direct
Insert
Early
Structure
Binding
• Decide
what
ques,ons
to
ask
• Design
the
data
schema
• Normalize
the
data
• Write
database
inser,on
code
• Create
the
queries
• Feed
the
results
into
an
analy,cs
tool

5. Business
Analy,cs
–Before
Splunk
IT/Business
Challenges
• Most
organiza,ons
only
rely
on
structured
data
for
business
analy,cs
–
not
sufficient
today!
• New
data
sources
such
as
machine
increasingly
cri,cal
sources
of
insight
–
not
leveraged
by
organiza,ons
• Inability
to
scale
/
handle
data
volume
of
new
sources
as
data
con,nues
to
grow
Inability
to
deliver
real-­‐,me
insights
to
the
business.
• Most
today
rely
on
ETL
causing
latency
in
analy,cs
Exis,ng
solu,ons
unable
to
do
data
mash-­‐up
across
structured
and
machine
data
Business
Consequence
• Inability
to
gain
real-­‐,me
business
insights
from
new
data
sources
• Business
users
across
func,ons
(sales
ops,
product
managers,
marke,ng,
and
customer
support
users
cannot
leverage
new
data
sources
for
analy,cs
• Compe,,ve
disadvantage
as
other
companies
increasingly
leverage
machine
data
for
business
insights
• Unable
to
get
insights
from
new
data
sources
with
their
tradi,onal
structured
analy,cs
tools

6. Business
Analy,cs
–
A]er
Splunk
IT/Business
Vision
• Deliver
real-­‐,me
business
insight
from
machine
data
• Enrich
machine
data
with
structured
data
to
provide
business
context
• Complement
exis,ng
BI
technologies
for
insight
into
a
new
class
of
data
• Leverage
search,
interac,ve
dashboards
in
Splunk
or
other
3rd
party
visualiza,on
tools
• Rapid
,me
to
value
in
gaining
business
insights
from
machine
data
Business
Benefits
• Applica,on
Analy,cs
–
to
understand
how
customers
are
interac,ng
with
various
online
applica,ons.
• Content
&
Search
Analy,cs
–
to
understand
how
customers
are
accessing
and
searching
for
content
served
up
over
CDNs
• Real-­‐,me
Sales
Analy,cs
–
to
gain
real-­‐,me
visibility
into
products
and
services
that
customers
are
purchasing.
• Service
Cost
Analy,cs
–
to
gain
insight
(for
example)
into
call
detail
records
and
cost
associated
with
comple,ng
each
call.
• Online
Mone,za,on
Analy,cs
–
an
example
of
this
is
online
gaming
companies
where
they
are
introducing
virtual
goods
and
charging
for
them.
• Marke,ng
Analy,cs
–
understanding
customer
click-­‐
through
for
ads
helps
improve
placement,
pricing
and
click
through
rates.

7. Splunk
Delivers
Value
Across
IT
and
the
Business
Business
Analy,cs
Digital
Intelligence
Security
and
Compliance
IT
Opera,ons
App
Manageme
nt
Industrial
Data
Developer
PlaEorm
(REST
API,
SDKs)
>SPLUNK
Small
Data.
Big
Data.
Huge
Data.

8. Splunk
Turns
Machine
Data
into
Opera,onal
Intelligence
Customer
Facing
Data
Outside
the
Datacenter
ApplicaDons
" Web
logs
" Log4J,
JMS,
JMX
" .NET
events
" Code
and
scripts
Networking
" Configura,ons
" syslog
" SNMP
" neElow
Databases
" Configura,ons
" Audit/query
logs
" Tables
" Schemas
VirtualizaDon
&
Cloud
" Hypervisor
" Guest
OS,
Apps
" Cloud
Linux/Unix
" Configura,ons
" syslog
" File
system
" ps,
iostat,
top
Windows
" Registry
" Event
logs
" File
system
" sysinternals
Logfiles
Configs
Messages
Traps
Alerts
Metrics
Scripts
Changes
Tickets
" Click-­‐stream
data
" Shopping
cart
data
" Online
transac,on
data
" Manufacturing,
logis,cs…
" CDRs
&
IPDRs
" Power
consump,on
" RFID
data
" GPS
data

9. Early
vs.
Late
Binding
Schema
Early
Structure
Binding
-­‐
Tradi,onal
SELECT
customers.*
FROM
customers
WHERE
customers.customer_id
NOT
IN(SELECT
customer_id
FROM
Orders
WHERE
year(orders.order_date)
=
2004)
Structure
Data
• Schema
–
created
at
design
,me
• Homogeneous–
must
fit
into
tables
or
be
converted
to
fit
into
tables
• Queries
–
understood
at
design
,me
for
maximum
performance
•
Must
exactly
match
constraints

10. Early
vs.
Late
Binding
Schema
Late
Structure
Binding
-­‐
Splunk
Structure
Data
• Schema-­‐less
• Heterogeneous–
can
come
from
any
textual
source
• Created
at
search
,me
• Constantly
changing
• Queries/searches
can
be
ad-­‐hoc
• No
conversion
required,
no
constraints

11. Analy,cs
Early
Structure
Binding
Late
Binding
Schema
Decide
the
ques,on(s)
you
want
to
ask
Design
the
Schema
Normalize
the
data
and
write
DB
inser,on
code
Create
SQL
&
Feed
into
Analy,cs
Tool
Write
data
(or
events)
to
log
files
Collect
the
log
files
Create
searches,
graphs,
and
reports
using
Splunk
(Days,
Weeks
or
Months
&
Destruc,ve)
(Minutes
&
Non-­‐
Destruc,ve)

12. Example:
Business
Visibility
From
Machine
Data
Machine
Data
(from
customer
interacDon)
Product
InformaDon
Geo
locaDon
Data
Customer
interacts
with
service
online
or
from
any
device
Ac,on
Product
User
session
User
browser
informa,on
66.57.19.112 ..[05/Dec/2011 07:05:22:152]”GET /card.do?
action=addtocart&itemid=EST-17& product_id=K9-
BD-01&JSESSIONID.SD7SLSFF8ADFF8HTTP 1.1” 200 3923
AppleWebKit/535.2 (KHTML.like Gecko) Chrome/15.0.874.121
Safari535.2
Product_id=K9-BD-01
Product Name=2 TB Portable Drive
Manufacturer=iomega
Real-­‐Time
Business
Insights
from
Machine
Data
Geo location
data
Correlated
with
product
informa,on
from
database
Loca,on
data
based
on
where
the
customer
purchased
/
interacted
with
service
– What
products
are
popular
in
what
region?
– Which
product
are
customers
leaving
in
cart?
– What
are
interac,on
paths
by
devices?
– How
can
we
improve
customer
experience?

13. Gepng
Structured
Data
In
Splunk
CSV
lookup
Splunk
Connector
• Access
data
at
scale
• In
real-­‐,me
• Easy
set-­‐up
&
maintenance
Log
files
Structured
databases
Applica,ons
Web
Servers
Other
systems

14. DB
Connect:
Business
Context
to
Machine
Data
Structured
Data
>Machine
Data
>Business
AnalyDcs
Rate
plans,
customer
profile,
geo
loca,on
Customer
profile,
Service
subscrip,on
Product
descrip,ons,
Customer
profile
Device
ac,va,on,
Radius,
applica,on
logs
Applica,on,
server
and
network
logs
Applica,on
logs,
authen,ca,on
logs
Sales
Analy,cs
Customer
Analy,cs
Product
Analy,cs

15. Gepng
Business
Insights
from
Splunk
User
Interface:
Splunk
User
Interface:
Third
Party
Dashboards
Searches
Pivot
Schedule
SDK/APIs
ODBC

16. Posi,oning
Splunk
for
Business
Analy,cs
>New
class
of
data
for
business
analy,cs
>Enrich
machine
data
with
structured
data
>Real-­‐,me
business
insights
>Complement
tradi,onal
BI
Tools

17. Splunk
Complements
Exis,ng
BI
Tools
Features
Splunk
Leading
BI
Tools
Focus
PlaEorm
for
real-­‐,me
opera,onal
intelligence
Data
visualiza,on
and
business
intelligence
so]ware
Value
Collect,
index,
search,
monitor,
report
on,
analyze
massive
streams
of
machine
data
Analyze,
visualize
and
share
structured
data
Users
IT,
Opera,ons,
Security,
Developers,
Analysts,
Business
Users
(as
consumers)
Business
Users
and
Analysts
(already
using
data
discovery
tool)
Use
Cases
IT
Ops,
App
Management,
Security,
Digital
Intelligence,
Business
Analy,cs
from
machine
data,
Internet
of
Things
Marke,ng,
HR,
Sales
Repor,ng,
Supply
Chain
Analysis

18. Scales
to
TBs/day
and
Thousands
of
Users
" Automa,c
load
balancing
linearly
scales
indexing
" Distributed
search
and
MapReduce
linearly
scales
search
and
repor,ng

19. Summary
> Real
Time
Architecture
> Universal
Machine
Data
PlaWorm
> Schema
on
the
Fly
> Agile
ReporDng
and
AnalyDcs
> Scales
from
Desktop
to
Enterprise
> Fast
Time
to
Value
> Passionate
and
Vibrant
Community