Drowning in Data but Thirsty for Insights

PREPARED FOR:Connected Intelligence Platform
October 8-11, 2018
Chicago, IL
Platinum
Sponsor
Drowning
in Data,
But Thirsty
for Insights
Benjamin Nussbaum, CTO

October 10, 20182
$100+ Billion Spend On Big Data Is Not Getting Results
Siloed Legacy
Data Warehouses
Data Lakes
Expensive
Internal Projects
No Single System of Record
Isolated and Disconnected
Enterprise
Systems
Customer
Interactions
External
Data
IoT
Explosive Data Growth
Data remains largely isolated and in
silos
Hugely expensive projects + ongoing
maintenance and operations costs
Poor ROI on value of analytics due to
underwhelming and misleading
insights
High vulnerability to data security
issues
No cloud-first model - inability to easily
explore, deploy and scale with elastic
pricing
Poor Business Value
CSV
DOC
PDF
“For the first time, the majority of firms assign executive responsibilities and large corporate budgets to make data
and insights a coordinated and strategic enterprise initiative………. Bad news for many firms claiming to be insights-
driven — the term is overused.” – Forrester, November 2, 2016

October 10, 20183
Big Data (2005 - ?)
a large set of data that is almost impossible to manage and process
using traditional business intelligence tools

October 10, 20184
Big Data
• Data Warehouses
• Business Intelligence
• Data Lakes
existing data analysis solutions

October 10, 20185
Big Data
• Isolated Islands
• “Connected” in a Misleading Manner:
• Using a patchwork of overlays and joins
• Do not scale effectively
• Getting it there…Expensive, Time Consuming
• Then it has to be:
• Evolved
• Maintained
• Operationalized
• …all while producing underwhelming insights and exposing security vulnerabilities
aggregated but not properly connected

October 10, 20186
Big Data
• Expensive to Implement
• Slow to Change
• Difficult to Operationalize
• Inefficient Data Representation
• Inefficient Data Access
easily enamored by powerful compute and massive storage…but these just equate to more cost

October 10, 20187
Big Data
• Leading cloud companies (AWS, Azure, GC) don’t run
large Hadoop/Spark clusters from Cloudera and
Hortonworks
• They run distributed cloud-scale databases and
applications on top of container infrastructure.
• They use object storage solutions which are about 5x
cheaper on the storage cost alone (not including the
additional cost to operate HDFS)
the declining influence of hadoop

October 10, 20188
Big Data
• Increasingly, enterprises are shifting to similar
approaches because they want to reap the same
speed and scale benefits
• And according to Gartner only 15% of big data
projects ever get deployed to production
• Big Data vendors may tout customer growth and big
spend ($1m+) by customers on new initiatives
• But nearly every CIO survey highlights a lack of clear
value delivered by big data projects
the declining influence of hadoop

October 10, 20189
Big Data
• Depends…
• Are you doing it only for analytical value?
• Is it going to enable you to store data you can’t today?
Is it going to be the system of record?
• Can you leverage cloud storage?
• Are you just punting on what you should do with your
information and this is a catch-all?
• Are you doing this so you can connect multiple silos?
does a data warehouse or data lake initiative still make sense to start

October 10, 201810
The Cloud Example
how did cloud transform the enterprise?
• Transferred expensive and non-differentiating IT
infrastructure costs from internal purchasing and
support to a utility model
• Increased agility by removing multi-year capacity lock-
in to on-demand usage driven by business need
• Enabled enterprises to focus on transforming their
businesses at the pace of business
• Improved security and reliability

October 10, 201811
The Cloud Example
utility-based consumption enabled unprecedented
• Elasticity
• Buy what you need when you need it
• Flexibility
• Innovation
• Security

October 10, 201812
The Cloud Example
• Elasticity
• Flexibility
• Tailored to your specific requirements at the time you use it
• Innovation
• Security

October 10, 201813
The Cloud Example
• Elasticity
• Flexibility
• Innovation
• Tech future-proofing
• Easy to ramp new applications
• Security

October 10, 201814
The Cloud Example
• Elasticity
• Flexibility
• Innovation
• Tech future-proofing
• Easy to ramp new applications
• Security
• Everyone today acknowledges that cloud security is superior
to that of general IT

October 10, 201815
The Cloud Example for Big Data
big data is a perfect match with cloud
• Elasticity
• Data sources and volumes increasing exponentially
• Flexibility
• Requirements evolving rapidly as business keeps pace today
• Innovation
• New integrations with AI/ML processes
• Supporting advancements in analytics technology
• Security
• Ever increasing requirements
• GDPR made many data lakes a liability

October 10, 201816
Evolving Big Data to Smart Data
while useful, these big data initiatives do not yield smart data that is intelligently connected and easily traversed
Connected
Data
Big Data
Aggregation
Real-Time
Queries & Algorithms
Continuously-Computed
Queries & Algorithm
Discovery
+ Reasoning
API

October 10, 201817
The Future is in Connected Data
The world is more connected than ever before, and data
relationships are constantly, ceaselessly increasing.
Data in the enterprise is no different -- it is bidirectional,
always-flowing, and continuously changing.
Yet it remains largely segmented and disconnected

October 10, 201818
Smart Data
what do we want from our data?
• Intuitive
• Speed
• Agility
• Context
• Intelligence

October 10, 201819
Smart Data
data used to be stored like this and was completely disconnected

October 10, 201820
Smart Data
RDBMS promised it would connect our data, but index-based joining was too costly

October 10, 201821
Smart Data
…and NoSQL stores and big data warehouses/lakes went even further away from connection being front and center

October 10, 201822
Non-Graph Pains
Complex to model and store relationships
Use indexes for connecting data
Performance degrades with increases in data
Queries get long and complex
Maintenance is painful

October 10, 201823
Smart Data
we need a native graph databases because our data is actually a graph

October 10, 201824
Smart Data
a graph models people, places and things
A “Node”
in the graph
HOTEL
ROOM
PERSON

October 10, 201825
Smart Data
a graph models real-time relationships
An “Edge”
in the graph
HOTEL
ROOM
PERSON

October 10, 201826
Smart Data
a graph stores and updates data about each thing and its relationships
HOTEL
ROOM
PERSON
“Properties”
in the Graph
Name: Westin Chicago River North
Name: Jane Smith
Number: 2021

October 10, 201827
Property Graph Model
LOVES
AnnDan
CREATE (:Person { name:“Dan”} ) - [:LOVES]-> (:Person { name:“Ann”} )
NODE
LABEL PROPERTY
NODE
LABEL PROPERTYCONTEXT

October 10, 201828
LOVES
AnnDan
NODE
LABEL PROPERTY
NODE
LABEL PROPERTY
weight: 0.8
CONTEXT

October 10, 201829
LOVES
AnnDan
NODE
LABEL PROPERTY
NODE
LABEL PROPERTY
weight: 0.8
LOVES
weight: 0.2
CONTEXT

October 10, 201830
Native Graph Gains
Easy to model and store relationships
Uses pointers to do traversal
Performance of relationship traversal remains constant
with growth in data size
Queries are shortened and more readable
Adding additional properties and relationships can be
done on the fly

October 10, 201831
Native Graph Gains
easy to model: graphs connect not only your data but your whole organization

October 10, 201832
Native Graph Gains
relationships are first-class citizens: easy to store and traverse relationships without using indexes

October 10, 201833
Native Graph Gains
relationships are first-class citizens: performance of relationship traversals remains constant with growth in data size

October 10, 201834
Native Graph Gains
adding additional properties and relationships can be done on the fly: graphs improve data understanding and interactions
Purchase History
CategoryHome delivery
Location/Adress

October 10, 201835
Native Graph Gains
Returns
Purchase History
Location/Adress
Promotions

October 10, 201836
Native Graph Gains
Complaints
reviews
Tweets
Emails
Returns
Purchase History
Location/Adress
Promotions

October 10, 201837
Native Graph Gains
“Complex Join” in SQL opencypher.org – Native Query Language for Graphs
SQL Query vs Native Graph Query (Cypher)
Equivalent queries for finding the reporting
chain within an organization
queries are shortened and more readable: graphs improve developer productivity

October 10, 201838
Day in the Life of an RDBMS Developer

October 10, 201839

October 10, 201840

October 10, 201841

October 10, 201842

October 10, 201843

October 10, 201844

October 10, 201845

October 10, 201846
SELECT
p.name,
c.country, c.leader, p.hair,
u.name, u.pres, u.state
FROM
people p
LEFT JOIN country c ON c.ID=p.country
LEFT JOIN uni u ON p.uni=u.id
WHERE
u.state=‘CT’

October 10, 201847

October 10, 201848

October 10, 201849

October 10, 201850

October 10, 201851

October 10, 201852
Day in the Life of a Graph Developer

October 10, 201853
MATCH
(p:Person)-[:WENT_TO]->(u:Uni),
(p)-[:LIVES_IN]->(c:Country),
(u)-[:LED_BY]->(l:Leader),
(u)-[:LOCATED_IN]->(s:State)
WHERE
s.abbr = ‘CT’
RETURN
p.name,
u.name, l.name, s.abbr
SELECT
p.name,
u.name, u.pres, u.state
FROM
people p
LEFT JOIN country c ON c.ID=p.country
LEFT JOIN uni u ON p.uni=u.id
WHERE
u.state=‘CT’
Day in the Life of a Graph Developer

October 10, 201854
Recognizing Graph Problems
Social networks RetailHR &
Recruiting
Manufacturing
& Logistics
Health Care TelcoFinance
today we’re seeing graph projects across virtually every industry

October 10, 201855
traditional supply chain
End Consumers
Component
Manufacturers
Logistics
RetailersWholesalers
Assembly
Plants

October 10, 201856
connected customer experience
PAYMENTS
SALES-
CHANNELS
SUPPLY
CHAIN
PRODUCTS MARKETING
CRM
CUSTOMER
EXPERIENCEStore
MobileWebstore

October 10, 201857
PAYMENTS
SALES-
CHANNELS
SUPPLY
CHAIN
PRODUCTS MARKETING
CRM
CUSTOMER
EXPERIENCEStore
MobileWebstore
ShippingInventory
Express goods
Home delivery

October 10, 201858
PAYMENTS
SALES-
CHANNELS
SUPPLY
CHAIN
PRODUCTS MARKETING
CRM
CUSTOMER
EXPERIENCEStore
MobileWebstore
ShippingInventory
Express goods
Ratings
Category
Price-Range Returns
Home delivery

October 10, 201859
PAYMENTS
SALES-
CHANNELS
SUPPLY
CHAIN
PRODUCTS MARKETING
CRM
CUSTOMER
EXPERIENCEStore
MobileWebstore
ShippingInventory
Express goods
Ratings
Category
Price-Range Content
Promotions
Online Advertising
Returns
Home delivery

October 10, 201860
PAYMENTS
SALES-
CHANNELS
SUPPLY
CHAIN
PRODUCTS MARKETING
CRM
CUSTOMER
EXPERIENCEStore
MobileWebstore
ShippingInventory
Express goods
Ratings
Category
Price-Range Content
Promotions
Online Advertising
Returns
Home delivery
Feedback
Support
Loyalty Programs
Reviews
Emails
Tweets

October 10, 201861
PAYMENTS
SALES-
CHANNELS
SUPPLY
CHAIN
PRODUCTS MARKETING
CRM
CUSTOMER
EXPERIENCEStore
MobileWebstore
ShippingInventory
Express goods
Ratings
Category
Price-Range Content
Promotions
Online Advertising
Returns
Home delivery
Feedback
Support
Loyalty Programs
Reviews
Emails
Tweets
Purchase History
Cash
Credit Card
Mobile Pay

October 10, 201862

October 10, 201863
Use Case: Real-Time Recommendations

October 10, 201864
Use Case: MDM

October 10, 201865
Use Case: Fraud Detection

October 10, 201866
Use Case: Graph-Based Search
IN
IN

October 10, 201867
Use Case: Network Management

October 10, 201868
Use Case: IAM

October 10, 201869
Use Case: Real World, Real-Time
An example scenario of what becomes possible when matching the
shape of your data with the technology
Business: Buying and selling of online advertising
Accepted Reality: Maximum of 1hr to update bids
Original Technical: 3TB SQL RDBMS relying on distributed, federated and
highly indexed views to come close to 1hr
Challenge: Taking more than 1hr to update bids

October 10, 201870
Solution: Identified data structure as highly-connected & deep
New Reality: Search and Intelligent Bid Optimization
Solution Technical: 1TB Open Neo4j (10% of hardware), Elasticsearch
integrated on GraphGrid, writing over 2B nodes/edges per day
Result: Taking less than 300ms to update bids

October 10, 201871
Business: Selling complex content packages
Accepted Reality: Between 4-6hrs for sales rep to get answer
Original Technical: Generating 1B row hash tables (Oracle RDBMS)
w/only 1 or 2 SMEs able able to modify stored procedure
Challenge: Takes 4-6hrs to know if content package can be sold

October 10, 201872
Solution: Identified data structure as highly-connected, living
New Reality: Search and intelligent content package negotiator
Solution Technical: Open Neo4j, Elasticsearch integrated on GraphGrid,
interactive package optimizer & recommender
Result: Sub-second determination of non-conflicting package across
entire sales organization & advisory recommender system suggesting
content to include/exclude throughout deal

October 10, 201873
Business: Highly regulated global financial institution
Accepted Reality: Complex data lineages will never finish
Original Technical: Oracle SQL RDBMS
Challenge: Queries for complex lineages never finish

October 10, 201874
Solution: Identified data structure as highly-connected, deep & dense
New Reality: Complex lineages finish in under 1 minute
Solution Technical: Open Neo4j with traversal algorithms
Result: Governance and understanding of data movement throughout 3k+
enterprise applications possible

October 10, 201875
Use Case: Graph Cloud
Disparate
Data Sources
◦ RDBMS / Object Store
◦ Data Warehouses & Lakes
◦ Enterprise Apps
◦ Social Media
◦ Sensors / IoT
◦ Third Party
Intelligent
Insights
◦ Predictive analytics
◦ Real-time processing
◦ Systems of Record
◦ Recommendations
◦ Descriptive analysis
High Availability, business continuity, DR
Rapid scaling of data sources, volumes, geographies
World class security, access control, data protection
Commercial Open Native Graph DB integrated with AWS
Agile, elastic pay-as-you-use infrastructure and data services
Ease of legacy co-existence + adding new applications
Graph Stream Graph Compute Graph Search Graph Viz
Graph Secure Graph Ops Graph Manager Graph Publish
Curator UI AI / ML IoT Data Apps
Graph DB Connected Data Storage & Traversal
Managed Data Cloud Platform

October 10, 201876
Bringing Connected Data to the General Enterprise
Industry leaders have harnessed proprietary connected intelligence
platforms for market dominance. GraphGrid’s mission is to provide
general enterprises the tools to create and evolve their unique connected
intelligence platform.
Connected
Intelligence
Customers create their own
connected intelligence
about customers, devices,
operations, and
ecosystems
GraphGrid R&D built a
platform to manage
systems of intelligence at
scale
Labor
Intelligence
Search
Intelligence
Social
Intelligence
LinkedIn has its own
labor intelligence, from
professional networks
to relevant jobs &
training to resource
tracking
Proprietary R&D built
the LinkedIn
Connection Engine to
connect professionals
Google has its own
search intelligence,
from enhanced results
to knowledge backed
services
Proprietary R&D built
the Google Knowledge
Graph to connect the
world’s information
Facebook has its own
social intelligence, from
consumer interests to
evolving personal networks
and trends
Proprietary R&D built the
Facebook Social Graph to
connect society & provide
global engagement

77 October 10, 2018
Thank you!
Benjamin Nussbaum – CTO, GraphGrid, Inc.
(ben@graphgrid.com)

Drowning in Data but Thirsty for Insights

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Drowning in Data but Thirsty for Insights

Ähnlich wie Drowning in Data but Thirsty for Insights (20)

Mehr von Benjamin Nussbaum

Mehr von Benjamin Nussbaum (6)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Drowning in Data but Thirsty for Insights