SlideShare ist ein Scribd-Unternehmen logo
1 von 31
Downloaden Sie, um offline zu lesen
Cyber Pattern Discovery using Linked Data Analysis 
November 12, 2014 
Page 1 © Hortonworks Inc. 2014 
Webinar with Hortonworks and Sqrrl
Webinar with Hortonworks and Sqrrl 
Cyber Pattern Discovery using Linked Data Analysis 
November 12, 2014 
Page 2 © Hortonworks Inc. 2014 
Joe Travaglini 
Director of Products 
Sqrrl 
John Kreisa 
VP Strategic Marketing 
Hortonworks
The Modern Data Architecture 
Page 3 © Hortonworks Inc. 2014 
Hortonworks. We do Hadoop. 
John Kreisa, 
VP Strategic Marketing 
Hortonworks
Agenda 
• Apache Hadoop and a Modern Data Architecture 
• Security in a comprehensive data management platform 
• Security Analytics using (Big) Cybersecurity Data 
• Case study: Internal network breach 
Page 4 © Hortonworks Inc. 2014
Our Mission: Power your Modern Data Architecture 
with HDP and Enterprise Apache Hadoop 
Who we are 
June 2011: Original 24 architects, developers, operators of Hadoop from Yahoo! 
June 2014: An enterprise software company with 420+ Employees 
Our model 
Innovate and deliver Apache Hadoop as a complete enterprise data platform 
completely in the open, backed by a world class support organization 
Key Partners 
Page 5 © Hortonworks Inc. 2014
Why a Modern Data Architecture? 
Business 
Analytics 
LIMITATIONS 
Silos & Expensive 
Single Purpose 
DATA SYSTEM APPLICATIONS 
Custom 
Applications 
Page 6 © Hortonworks Inc. 2014 
Packaged 
Applications 
RDBMS EDW MPP 
MDA: Key Drivers 
1. Leverage new types of data 
2. IT optimization 
3. Enable a data lake 
GOALS 
• Extend new data sets across 
existing data platforms 
• Common data platform, multiple 
processing engines 
• Batch, interactive and real time 
on a single data platform 
EXISTING 
Systems 
Clickstream 
Web 
&Social 
Geoloca9on 
Sensor 
& 
Machine 
Server 
Logs 
Unstructured 
SOURCES
A Modern Data Architecture Includes Hadoop 
Page 7 © Hortonworks Inc. 2014 
Hadoop compliments and 
enhances existing technologies 
Common data set, multiple applications 
• Optionally land all data in a single cluster 
• Batch, interactive & real-time use cases 
• Support multi-tenant access, processing 
& segmentation of data 
YARN: Architectural center of Hadoop 
• Consistent security, governance & operations 
• Ecosystem applications certified " 
by Hortonworks to run natively in Hadoop 
SOURCES 
EXISTING 
Systems 
Clickstream 
Web 
&Social 
Batch Interactive Real-Time 
HDFS 
(Hadoop Distributed File System) 
Geoloca9on 
Sensor 
& 
Machine 
Server 
Logs 
Unstructured 
DATA SYSTEM APPLICATIONS 
Business 
Analytics 
Custom 
Applications 
Packaged 
Applications 
RDBMS EDW MPP YARN: Data Operating System 
1 ° ° ° ° ° ° ° ° ° 
° ° ° ° ° ° ° ° ° N
Unlock New Applications from New Types of Data 
INDUSTRY USE CASE Sentiment 
Page 8 © Hortonworks Inc. 2014 
& Web 
Clickstream 
& Behavior 
Machine 
& Sensor Geographic Server Logs Structured & 
Unstructured 
Financial Services 
New Account Risk Screens ✔ ✔ 
Trading Risk ✔ 
Insurance Underwriting ✔ ✔ ✔ 
Telecom 
Call Detail Records (CDR) ✔ ✔ 
Infrastructure Investment ✔ ✔ 
Real-time Bandwidth Allocation ✔ ✔ ✔ 
Retail 
360° View of the Customer ✔ ✔ ✔ 
Localized, Personalized Promotions ✔ 
Website Optimization ✔ 
Manufacturing 
Supply Chain and Logistics ✔ 
Assembly Line Quality Assurance ✔ 
Crowd-sourced Quality Assurance ✔ 
Healthcare 
Use Genomic Data in Medial Trials ✔ ✔ ✔ 
Monitor Patient Vitals in Real-Time ✔ ✔ 
Pharmaceuticals 
Recruit and Retain Patients for Drug Trials ✔ ✔ 
Improve Prescription Adherence ✔ ✔ ✔ ✔ 
Oil & Gas 
Unify Exploration & Production Data ✔ ✔ ✔ ✔ 
Monitor Rig Safety in Real-Time ✔ ✔ ✔ 
Government 
ETL Offload/Federal Budgetary Pressures ✔ ✔ 
Sentiment Analysis for Government 
Programs ✔
Break Down Silos with a Security Data Lake 
SCALE 
Page 9 © Hortonworks Inc. 2014 
SCOPE 
Unlocking the Data Lake 
RDBMS 
MPP 
EDW 
• Data Lake 
Enabled by YARN 
• Single data repository, 
shared infrastructure 
• Multiple security apps 
accessing all the data 
• Enable a shift from 
reactive to proactive 
interactions 
• Gain new insight across 
the entire enterprise 
New Analytic Apps 
or IT Optimization 
HDP 2.1 
Governance 
& Integration 
Security 
Operations 
Data Access 
YARN 
Data Management
Big Data is Changing Cyber Security 
“By 2016, more than 25 percent of global firms will adopt big data 
analytics for at least one security and fraud detection use case, up from 
current eight percent.” 
– Gartner Cyber Security report Feb 2014 
Gartner recommendations 
• Align security capabilities in a holistic security strategy tailored to the threats and risks 
• Target a single architecture to collect, index, normalize, analyze and share all information 
• Organizations should look for profile accounts, users or other entities, and look for 
anomalous transactions against those profiles 
Page 10 © Hortonworks Inc. 2014
How Can Big Data Analytics Help Cyber Security? 
• To prioritize threats, vulnerabilities, and attacks 
• To control endpoints and mobile connections /devices 
• To prevent insecure devices from accessing secure systems 
• To provide intelligence about the threat land- scape 
• To reduce false positives 
Page 11 © Hortonworks Inc. 2014
Page 12 © Hortonworks Inc. 2014 
Securely explore your data 
CYBER PATTERN 
DISCOVERY USING 
LINKED DATA ANALYSIS 
A Big Data Solution with Hortonworks and Sqrrl 
Joe Travaglini, Director of Products, Sqrrl 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Who We Are 
Page 13 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Agenda 
• Security Analytics using (Big) Cybersecurity Data 
• Dealing with the new security dilemma 
• Why Hadoop and HDP are the perfect fit 
• The ‘Linked Data’ Approach 
• Case study: internal network breach 
• Overview of scenario 
• Data modeling with Sqrrl 
• Visual, contextual research and analysis 
Page 14 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
The Numbers Don’t Lie 
229 87% 
90% $12.7M 
Page 15 © Hortonworks Inc. 2014 
Source: Mandiant Source: Verizon 
Source: Verizon Source: Ponemon 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Targeted Attacks Have Changed the Game 
Page 16 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
Source: 
Battery Ventures
What Does This Mean For Us? 
Dissolution of the Secure Perimeter 
• You’ve been breached. Deal with it. 
• Empower the investigator 
• Research and respond: better, faster, smarter 
• It’s all about speed to understanding 
Page 17 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
The Security Data Dilemma 
Detecting attacks requires more (i.e. BIG) data 
Page 18 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
But your tools 
can’t handle the 
big data wave 
So attackers are spilling in
A Modern Data Architecture 
Hortonworks and Sqrrl Solution 
• Hortonworks Data Platform 
at the core 
• Sqrrl Enterprise stack at the 
app layer 
Hadoop enables us to look at 
data differently 
Page 19 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Sqrrl Enterprise Architecture 
Interface 
Processing 
Data Model 
Data Storage 
Page 20 © Hortonworks Inc. 2014 
Visualization / API ML + Anomaly Detection 
Query Engine Bulk/Graph Processing 
Raw Events Linked Data Model 
HDFS + Accumulo 
Commodity Hardware 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
Physical 
Security 
Audit 
Cryptography 
Labeling + Policy
Big Data Transformed 
Data Sources Linked Contextual Knowledge Analysis 
Email 
Page 21 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
Security Data 
VPN FW 
Network Data 
Proxy NetFlow 
Application Data 
HR USB
Linked Data Analysis 
Adding structure to the noise 
Page 22 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Page 23 © Hortonworks Inc. 2014 
Case Study: 
Compromised Network 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Breach Detection Scenario 
Page 24 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Case Study Model 
Data Sources 
Page 25 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
Linked Meta Model 
Users 
Hosts 
login 
flow 
login 
DNS 
records 
Netflow 
Host logs 
Database 
logs 
External 
Alerts
Case Study Example Mapping 
Page 26 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
Netflow Records 
startTime endTime sourceIP destIP sourcePort destPort protocol tcpFlags bytesIn bytesOut 
10/22/14 8:58 
10/22/14 8:58 
10.0.2.15 
192.168.0.123 
37051 
139 
TCP 
...RS. 
100 
3355 
10/22/14 8:45 
10/22/14 8:45 
10.0.2.15 
192.168.0.6 
0 
3328 
ICMP 
...... 
40 
100 
10/22/14 8:59 
10/22/14 8:59 
192.168.0.119 
10.0.2.15 
139 
60071 
TCP 
.A..S. 
46 
351 
10.0.2.15 
192.168. 
0.123 
Class=Flow, 
totalBytes = 3455 
192.168. 
Class=Flow, 0.6 
totalBytes = 140
Case Study Example Data 
Page 27 © Hortonworks Inc. 2014 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Investigation Process 
1. Set the Stage 2. Enable Search 
Page 28 © Hortonworks Inc. 2014 
and Discovery 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
3. Automate 
Analysis 
• Define the 
security-centric 
entity/ 
relationship 
model 
• Extract and 
maintain the 
model 
• Visually 
navigate assets 
and actors in 
the network 
• Drill down to the 
raw data 
seeding the 
model 
• Use behavioral 
analytics to build 
expectations of 
‘normal’ 
• Flag entities as 
potentially 
‘abnormal’ and 
sniff them out
Page 29 © Hortonworks Inc. 2014 
Visualizing the Threat 
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Page 30 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
Page 31 © Hortonworks Inc. 2014 
Thanks! 
© 2014 Sqrrl Data, Inc. | All Rights Reserved 
Joe Travaglini 
Director of Products, Sqrrl Data, Inc. 
@joe_travaglini 
jtrav@sqrrl.com 
http://www.sqrrl.com

Más contenido relacionado

Was ist angesagt?

Hortonworks and Platfora in Financial Services - Webinar
Hortonworks and Platfora in Financial Services - WebinarHortonworks and Platfora in Financial Services - Webinar
Hortonworks and Platfora in Financial Services - WebinarHortonworks
 
Apache Hadoop on the Open Cloud
Apache Hadoop on the Open CloudApache Hadoop on the Open Cloud
Apache Hadoop on the Open CloudHortonworks
 
Data Lake for the Cloud: Extending your Hadoop Implementation
Data Lake for the Cloud: Extending your Hadoop ImplementationData Lake for the Cloud: Extending your Hadoop Implementation
Data Lake for the Cloud: Extending your Hadoop ImplementationHortonworks
 
Rescue your Big Data from Downtime with HP Operations Bridge and Apache Hadoop
Rescue your Big Data from Downtime with HP Operations Bridge and Apache HadoopRescue your Big Data from Downtime with HP Operations Bridge and Apache Hadoop
Rescue your Big Data from Downtime with HP Operations Bridge and Apache HadoopHortonworks
 
Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...
Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...
Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...Hortonworks
 
Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...
Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...
Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...Hortonworks
 
Discover.hdp2.2.storm and kafka.final
Discover.hdp2.2.storm and kafka.finalDiscover.hdp2.2.storm and kafka.final
Discover.hdp2.2.storm and kafka.finalHortonworks
 
Predicting Customer Experience through Hadoop and Customer Behavior Graphs
Predicting Customer Experience through Hadoop and Customer Behavior GraphsPredicting Customer Experience through Hadoop and Customer Behavior Graphs
Predicting Customer Experience through Hadoop and Customer Behavior GraphsHortonworks
 
Simplify and Secure your Hadoop Environment with Hortonworks and Centrify
Simplify and Secure your Hadoop Environment with Hortonworks and CentrifySimplify and Secure your Hadoop Environment with Hortonworks and Centrify
Simplify and Secure your Hadoop Environment with Hortonworks and CentrifyHortonworks
 
YARN Ready: Integrating to YARN with Tez
YARN Ready: Integrating to YARN with Tez YARN Ready: Integrating to YARN with Tez
YARN Ready: Integrating to YARN with Tez Hortonworks
 
State of the Union with Shaun Connolly
State of the Union with Shaun ConnollyState of the Union with Shaun Connolly
State of the Union with Shaun ConnollyHortonworks
 
Webinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalWebinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalHortonworks
 
Discover HDP 2.2: Apache Falcon for Hadoop Data Governance
Discover HDP 2.2: Apache Falcon for Hadoop Data GovernanceDiscover HDP 2.2: Apache Falcon for Hadoop Data Governance
Discover HDP 2.2: Apache Falcon for Hadoop Data GovernanceHortonworks
 
Yahoo! Hack Europe
Yahoo! Hack EuropeYahoo! Hack Europe
Yahoo! Hack EuropeHortonworks
 
Hortonworks and Red Hat Webinar - Part 2
Hortonworks and Red Hat Webinar - Part 2Hortonworks and Red Hat Webinar - Part 2
Hortonworks and Red Hat Webinar - Part 2Hortonworks
 
Enterprise Apache Hadoop: State of the Union
Enterprise Apache Hadoop: State of the UnionEnterprise Apache Hadoop: State of the Union
Enterprise Apache Hadoop: State of the UnionHortonworks
 
Hp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar SlidesHp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar SlidesHortonworks
 
Create a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache HadoopCreate a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache HadoopHortonworks
 
Discover HDP 2.1: Apache Solr for Hadoop Search
Discover HDP 2.1: Apache Solr for Hadoop SearchDiscover HDP 2.1: Apache Solr for Hadoop Search
Discover HDP 2.1: Apache Solr for Hadoop SearchHortonworks
 
Eliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside HadoopEliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside HadoopHortonworks
 

Was ist angesagt? (20)

Hortonworks and Platfora in Financial Services - Webinar
Hortonworks and Platfora in Financial Services - WebinarHortonworks and Platfora in Financial Services - Webinar
Hortonworks and Platfora in Financial Services - Webinar
 
Apache Hadoop on the Open Cloud
Apache Hadoop on the Open CloudApache Hadoop on the Open Cloud
Apache Hadoop on the Open Cloud
 
Data Lake for the Cloud: Extending your Hadoop Implementation
Data Lake for the Cloud: Extending your Hadoop ImplementationData Lake for the Cloud: Extending your Hadoop Implementation
Data Lake for the Cloud: Extending your Hadoop Implementation
 
Rescue your Big Data from Downtime with HP Operations Bridge and Apache Hadoop
Rescue your Big Data from Downtime with HP Operations Bridge and Apache HadoopRescue your Big Data from Downtime with HP Operations Bridge and Apache Hadoop
Rescue your Big Data from Downtime with HP Operations Bridge and Apache Hadoop
 
Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...
Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...
Optimizing your Modern Data Architecture - with Attunity, RCG Global Services...
 
Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...
Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...
Discover hdp 2.2: Data storage innovations in Hadoop Distributed Filesystem (...
 
Discover.hdp2.2.storm and kafka.final
Discover.hdp2.2.storm and kafka.finalDiscover.hdp2.2.storm and kafka.final
Discover.hdp2.2.storm and kafka.final
 
Predicting Customer Experience through Hadoop and Customer Behavior Graphs
Predicting Customer Experience through Hadoop and Customer Behavior GraphsPredicting Customer Experience through Hadoop and Customer Behavior Graphs
Predicting Customer Experience through Hadoop and Customer Behavior Graphs
 
Simplify and Secure your Hadoop Environment with Hortonworks and Centrify
Simplify and Secure your Hadoop Environment with Hortonworks and CentrifySimplify and Secure your Hadoop Environment with Hortonworks and Centrify
Simplify and Secure your Hadoop Environment with Hortonworks and Centrify
 
YARN Ready: Integrating to YARN with Tez
YARN Ready: Integrating to YARN with Tez YARN Ready: Integrating to YARN with Tez
YARN Ready: Integrating to YARN with Tez
 
State of the Union with Shaun Connolly
State of the Union with Shaun ConnollyState of the Union with Shaun Connolly
State of the Union with Shaun Connolly
 
Webinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalWebinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_final
 
Discover HDP 2.2: Apache Falcon for Hadoop Data Governance
Discover HDP 2.2: Apache Falcon for Hadoop Data GovernanceDiscover HDP 2.2: Apache Falcon for Hadoop Data Governance
Discover HDP 2.2: Apache Falcon for Hadoop Data Governance
 
Yahoo! Hack Europe
Yahoo! Hack EuropeYahoo! Hack Europe
Yahoo! Hack Europe
 
Hortonworks and Red Hat Webinar - Part 2
Hortonworks and Red Hat Webinar - Part 2Hortonworks and Red Hat Webinar - Part 2
Hortonworks and Red Hat Webinar - Part 2
 
Enterprise Apache Hadoop: State of the Union
Enterprise Apache Hadoop: State of the UnionEnterprise Apache Hadoop: State of the Union
Enterprise Apache Hadoop: State of the Union
 
Hp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar SlidesHp Converged Systems and Hortonworks - Webinar Slides
Hp Converged Systems and Hortonworks - Webinar Slides
 
Create a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache HadoopCreate a Smarter Data Lake with HP Haven and Apache Hadoop
Create a Smarter Data Lake with HP Haven and Apache Hadoop
 
Discover HDP 2.1: Apache Solr for Hadoop Search
Discover HDP 2.1: Apache Solr for Hadoop SearchDiscover HDP 2.1: Apache Solr for Hadoop Search
Discover HDP 2.1: Apache Solr for Hadoop Search
 
Eliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside HadoopEliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside Hadoop
 

Andere mochten auch

Enterprise Data Warehouse Optimization: 7 Keys to Success
Enterprise Data Warehouse Optimization: 7 Keys to SuccessEnterprise Data Warehouse Optimization: 7 Keys to Success
Enterprise Data Warehouse Optimization: 7 Keys to SuccessHortonworks
 
Eric Baldeschwieler Keynote from Storage Developers Conference
Eric Baldeschwieler Keynote from Storage Developers ConferenceEric Baldeschwieler Keynote from Storage Developers Conference
Eric Baldeschwieler Keynote from Storage Developers ConferenceHortonworks
 
Streamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSenseStreamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSenseHortonworks
 
How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...
How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...
How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...Hortonworks
 
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017 Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017 Hortonworks
 
Benefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at ScaleBenefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at ScaleHortonworks
 

Andere mochten auch (6)

Enterprise Data Warehouse Optimization: 7 Keys to Success
Enterprise Data Warehouse Optimization: 7 Keys to SuccessEnterprise Data Warehouse Optimization: 7 Keys to Success
Enterprise Data Warehouse Optimization: 7 Keys to Success
 
Eric Baldeschwieler Keynote from Storage Developers Conference
Eric Baldeschwieler Keynote from Storage Developers ConferenceEric Baldeschwieler Keynote from Storage Developers Conference
Eric Baldeschwieler Keynote from Storage Developers Conference
 
Streamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSenseStreamline Apache Hadoop Operations with Apache Ambari and SmartSense
Streamline Apache Hadoop Operations with Apache Ambari and SmartSense
 
How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...
How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...
How to Architect and Omnichannel Retail Solution to Achieve Real-Time Custome...
 
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017 Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017
 
Benefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at ScaleBenefits of Transferring Real-Time Data to Hadoop at Scale
Benefits of Transferring Real-Time Data to Hadoop at Scale
 

Ähnlich wie Hortonworks sqrrl webinar v5.pptx

Supporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big DataSupporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big DataWANdisco Plc
 
Splunk-hortonworks-risk-management-oct-2014
Splunk-hortonworks-risk-management-oct-2014Splunk-hortonworks-risk-management-oct-2014
Splunk-hortonworks-risk-management-oct-2014Hortonworks
 
Hortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with HadoopHortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with HadoopMats Johansson
 
Hortonworks Hadoop @ Oslo Hadoop User Group
Hortonworks Hadoop @ Oslo Hadoop User GroupHortonworks Hadoop @ Oslo Hadoop User Group
Hortonworks Hadoop @ Oslo Hadoop User GroupMats Johansson
 
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...Hortonworks
 
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...Hortonworks
 
Hortonworks and HP Vertica Webinar
Hortonworks and HP Vertica WebinarHortonworks and HP Vertica Webinar
Hortonworks and HP Vertica WebinarHortonworks
 
Apache Hadoop and its role in Big Data architecture - Himanshu Bari
Apache Hadoop and its role in Big Data architecture - Himanshu BariApache Hadoop and its role in Big Data architecture - Himanshu Bari
Apache Hadoop and its role in Big Data architecture - Himanshu Barijaxconf
 
IoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJIoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJDaniel Madrigal
 
Hortonworks Data In Motion Webinar Series Pt. 2
Hortonworks Data In Motion Webinar Series Pt. 2Hortonworks Data In Motion Webinar Series Pt. 2
Hortonworks Data In Motion Webinar Series Pt. 2Hortonworks
 
Introduction to Hadoop
Introduction to HadoopIntroduction to Hadoop
Introduction to HadoopPOSSCON
 
Storm Demo Talk - Colorado Springs May 2015
Storm Demo Talk - Colorado Springs May 2015Storm Demo Talk - Colorado Springs May 2015
Storm Demo Talk - Colorado Springs May 2015Mac Moore
 
S2DS London 2015 - Hadoop Real World
S2DS London 2015 - Hadoop Real WorldS2DS London 2015 - Hadoop Real World
S2DS London 2015 - Hadoop Real WorldSean Roberts
 
Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...
Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...
Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...Hortonworks
 
The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...
The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...
The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...Platfora
 
Eliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside HadoopEliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside HadoopHortonworks
 
Real-Time Processing in Hadoop for IoT Use Cases - Phoenix HUG
Real-Time Processing in Hadoop for IoT Use Cases - Phoenix HUGReal-Time Processing in Hadoop for IoT Use Cases - Phoenix HUG
Real-Time Processing in Hadoop for IoT Use Cases - Phoenix HUGskumpf
 

Ähnlich wie Hortonworks sqrrl webinar v5.pptx (20)

Supporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big DataSupporting Financial Services with a More Flexible Approach to Big Data
Supporting Financial Services with a More Flexible Approach to Big Data
 
Splunk-hortonworks-risk-management-oct-2014
Splunk-hortonworks-risk-management-oct-2014Splunk-hortonworks-risk-management-oct-2014
Splunk-hortonworks-risk-management-oct-2014
 
Hortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with HadoopHortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with Hadoop
 
Meetup oslo hortonworks HDP
Meetup oslo hortonworks HDPMeetup oslo hortonworks HDP
Meetup oslo hortonworks HDP
 
Hortonworks Hadoop @ Oslo Hadoop User Group
Hortonworks Hadoop @ Oslo Hadoop User GroupHortonworks Hadoop @ Oslo Hadoop User Group
Hortonworks Hadoop @ Oslo Hadoop User Group
 
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
Starting Small and Scaling Big with Hadoop (Talend and Hortonworks webinar)) ...
 
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
A Comprehensive Approach to Building your Big Data - with Cisco, Hortonworks ...
 
Hortonworks and HP Vertica Webinar
Hortonworks and HP Vertica WebinarHortonworks and HP Vertica Webinar
Hortonworks and HP Vertica Webinar
 
Big Data for Security
Big Data for SecurityBig Data for Security
Big Data for Security
 
Apache Hadoop and its role in Big Data architecture - Himanshu Bari
Apache Hadoop and its role in Big Data architecture - Himanshu BariApache Hadoop and its role in Big Data architecture - Himanshu Bari
Apache Hadoop and its role in Big Data architecture - Himanshu Bari
 
IoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJIoT Crash Course Hadoop Summit SJ
IoT Crash Course Hadoop Summit SJ
 
Solving Big Data Problems using Hortonworks
Solving Big Data Problems using Hortonworks Solving Big Data Problems using Hortonworks
Solving Big Data Problems using Hortonworks
 
Hortonworks Data In Motion Webinar Series Pt. 2
Hortonworks Data In Motion Webinar Series Pt. 2Hortonworks Data In Motion Webinar Series Pt. 2
Hortonworks Data In Motion Webinar Series Pt. 2
 
Introduction to Hadoop
Introduction to HadoopIntroduction to Hadoop
Introduction to Hadoop
 
Storm Demo Talk - Colorado Springs May 2015
Storm Demo Talk - Colorado Springs May 2015Storm Demo Talk - Colorado Springs May 2015
Storm Demo Talk - Colorado Springs May 2015
 
S2DS London 2015 - Hadoop Real World
S2DS London 2015 - Hadoop Real WorldS2DS London 2015 - Hadoop Real World
S2DS London 2015 - Hadoop Real World
 
Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...
Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...
Distilling Hadoop Patterns of Use and How You Can Use Them for Your Big Data ...
 
The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...
The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...
The Big Data Gusher: Big Data Analytics, the Internet of Things and the Oil B...
 
Eliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside HadoopEliminating the Challenges of Big Data Management Inside Hadoop
Eliminating the Challenges of Big Data Management Inside Hadoop
 
Real-Time Processing in Hadoop for IoT Use Cases - Phoenix HUG
Real-Time Processing in Hadoop for IoT Use Cases - Phoenix HUGReal-Time Processing in Hadoop for IoT Use Cases - Phoenix HUG
Real-Time Processing in Hadoop for IoT Use Cases - Phoenix HUG
 

Mehr von Hortonworks

Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next LevelHortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next LevelHortonworks
 
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT StrategyIoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT StrategyHortonworks
 
Getting the Most Out of Your Data in the Cloud with Cloudbreak
Getting the Most Out of Your Data in the Cloud with CloudbreakGetting the Most Out of Your Data in the Cloud with Cloudbreak
Getting the Most Out of Your Data in the Cloud with CloudbreakHortonworks
 
Johns Hopkins - Using Hadoop to Secure Access Log Events
Johns Hopkins - Using Hadoop to Secure Access Log EventsJohns Hopkins - Using Hadoop to Secure Access Log Events
Johns Hopkins - Using Hadoop to Secure Access Log EventsHortonworks
 
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad GuysCatch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad GuysHortonworks
 
HDF 3.2 - What's New
HDF 3.2 - What's NewHDF 3.2 - What's New
HDF 3.2 - What's NewHortonworks
 
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
Curing Kafka Blindness with Hortonworks Streams Messaging ManagerCuring Kafka Blindness with Hortonworks Streams Messaging Manager
Curing Kafka Blindness with Hortonworks Streams Messaging ManagerHortonworks
 
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
Interpretation Tool for Genomic Sequencing Data in Clinical EnvironmentsInterpretation Tool for Genomic Sequencing Data in Clinical Environments
Interpretation Tool for Genomic Sequencing Data in Clinical EnvironmentsHortonworks
 
IBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data LandscapeIBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data LandscapeHortonworks
 
Premier Inside-Out: Apache Druid
Premier Inside-Out: Apache DruidPremier Inside-Out: Apache Druid
Premier Inside-Out: Apache DruidHortonworks
 
Accelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at ScaleAccelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at ScaleHortonworks
 
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATATIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATAHortonworks
 
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...Hortonworks
 
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: ClearsenseDelivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: ClearsenseHortonworks
 
Making Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with EaseMaking Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with EaseHortonworks
 
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World PresentationWebinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World PresentationHortonworks
 
Driving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data ManagementDriving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data ManagementHortonworks
 
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming FeaturesHDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming FeaturesHortonworks
 
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...Hortonworks
 
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDCUnlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDCHortonworks
 

Mehr von Hortonworks (20)

Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next LevelHortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
Hortonworks DataFlow (HDF) 3.3 - Taking Stream Processing to the Next Level
 
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT StrategyIoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
IoT Predictions for 2019 and Beyond: Data at the Heart of Your IoT Strategy
 
Getting the Most Out of Your Data in the Cloud with Cloudbreak
Getting the Most Out of Your Data in the Cloud with CloudbreakGetting the Most Out of Your Data in the Cloud with Cloudbreak
Getting the Most Out of Your Data in the Cloud with Cloudbreak
 
Johns Hopkins - Using Hadoop to Secure Access Log Events
Johns Hopkins - Using Hadoop to Secure Access Log EventsJohns Hopkins - Using Hadoop to Secure Access Log Events
Johns Hopkins - Using Hadoop to Secure Access Log Events
 
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad GuysCatch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
Catch a Hacker in Real-Time: Live Visuals of Bots and Bad Guys
 
HDF 3.2 - What's New
HDF 3.2 - What's NewHDF 3.2 - What's New
HDF 3.2 - What's New
 
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
Curing Kafka Blindness with Hortonworks Streams Messaging ManagerCuring Kafka Blindness with Hortonworks Streams Messaging Manager
Curing Kafka Blindness with Hortonworks Streams Messaging Manager
 
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
Interpretation Tool for Genomic Sequencing Data in Clinical EnvironmentsInterpretation Tool for Genomic Sequencing Data in Clinical Environments
Interpretation Tool for Genomic Sequencing Data in Clinical Environments
 
IBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data LandscapeIBM+Hortonworks = Transformation of the Big Data Landscape
IBM+Hortonworks = Transformation of the Big Data Landscape
 
Premier Inside-Out: Apache Druid
Premier Inside-Out: Apache DruidPremier Inside-Out: Apache Druid
Premier Inside-Out: Apache Druid
 
Accelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at ScaleAccelerating Data Science and Real Time Analytics at Scale
Accelerating Data Science and Real Time Analytics at Scale
 
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATATIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA
 
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
Blockchain with Machine Learning Powered by Big Data: Trimble Transportation ...
 
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: ClearsenseDelivering Real-Time Streaming Data for Healthcare Customers: Clearsense
Delivering Real-Time Streaming Data for Healthcare Customers: Clearsense
 
Making Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with EaseMaking Enterprise Big Data Small with Ease
Making Enterprise Big Data Small with Ease
 
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World PresentationWebinewbie to Webinerd in 30 Days - Webinar World Presentation
Webinewbie to Webinerd in 30 Days - Webinar World Presentation
 
Driving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data ManagementDriving Digital Transformation Through Global Data Management
Driving Digital Transformation Through Global Data Management
 
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming FeaturesHDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
HDF 3.1 pt. 2: A Technical Deep-Dive on New Streaming Features
 
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
Hortonworks DataFlow (HDF) 3.1 - Redefining Data-In-Motion with Modern Data A...
 
Unlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDCUnlock Value from Big Data with Apache NiFi and Streaming CDC
Unlock Value from Big Data with Apache NiFi and Streaming CDC
 

Último

Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updateadam112203
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1DianaGray10
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud DataEric D. Schabell
 
Scenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosScenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosErol GIRAUDY
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationKnoldus Inc.
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and businessFrancesco Corti
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechProduct School
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Libraryshyamraj55
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTopCSSGallery
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameKapil Thakar
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch TuesdayIvanti
 

Último (20)

Patch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 updatePatch notes explaining DISARM Version 1.4 update
Patch notes explaining DISARM Version 1.4 update
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
UiPath Studio Web workshop series - Day 1
UiPath Studio Web workshop series  - Day 1UiPath Studio Web workshop series  - Day 1
UiPath Studio Web workshop series - Day 1
 
3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data3 Pitfalls Everyone Should Avoid with Cloud Data
3 Pitfalls Everyone Should Avoid with Cloud Data
 
Scenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenariosScenario Library et REX Discover industry- and role- based scenarios
Scenario Library et REX Discover industry- and role- based scenarios
 
Introduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its applicationIntroduction to RAG (Retrieval Augmented Generation) and its application
Introduction to RAG (Retrieval Augmented Generation) and its application
 
From the origin to the future of Open Source model and business
From the origin to the future of  Open Source model and businessFrom the origin to the future of  Open Source model and business
From the origin to the future of Open Source model and business
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
 
How to release an Open Source Dataweave Library
How to release an Open Source Dataweave LibraryHow to release an Open Source Dataweave Library
How to release an Open Source Dataweave Library
 
Top 10 Squarespace Development Companies
Top 10 Squarespace Development CompaniesTop 10 Squarespace Development Companies
Top 10 Squarespace Development Companies
 
Flow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First FrameFlow Control | Block Size | ST Min | First Frame
Flow Control | Block Size | ST Min | First Frame
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
March Patch Tuesday
March Patch TuesdayMarch Patch Tuesday
March Patch Tuesday
 

Hortonworks sqrrl webinar v5.pptx

  • 1. Cyber Pattern Discovery using Linked Data Analysis November 12, 2014 Page 1 © Hortonworks Inc. 2014 Webinar with Hortonworks and Sqrrl
  • 2. Webinar with Hortonworks and Sqrrl Cyber Pattern Discovery using Linked Data Analysis November 12, 2014 Page 2 © Hortonworks Inc. 2014 Joe Travaglini Director of Products Sqrrl John Kreisa VP Strategic Marketing Hortonworks
  • 3. The Modern Data Architecture Page 3 © Hortonworks Inc. 2014 Hortonworks. We do Hadoop. John Kreisa, VP Strategic Marketing Hortonworks
  • 4. Agenda • Apache Hadoop and a Modern Data Architecture • Security in a comprehensive data management platform • Security Analytics using (Big) Cybersecurity Data • Case study: Internal network breach Page 4 © Hortonworks Inc. 2014
  • 5. Our Mission: Power your Modern Data Architecture with HDP and Enterprise Apache Hadoop Who we are June 2011: Original 24 architects, developers, operators of Hadoop from Yahoo! June 2014: An enterprise software company with 420+ Employees Our model Innovate and deliver Apache Hadoop as a complete enterprise data platform completely in the open, backed by a world class support organization Key Partners Page 5 © Hortonworks Inc. 2014
  • 6. Why a Modern Data Architecture? Business Analytics LIMITATIONS Silos & Expensive Single Purpose DATA SYSTEM APPLICATIONS Custom Applications Page 6 © Hortonworks Inc. 2014 Packaged Applications RDBMS EDW MPP MDA: Key Drivers 1. Leverage new types of data 2. IT optimization 3. Enable a data lake GOALS • Extend new data sets across existing data platforms • Common data platform, multiple processing engines • Batch, interactive and real time on a single data platform EXISTING Systems Clickstream Web &Social Geoloca9on Sensor & Machine Server Logs Unstructured SOURCES
  • 7. A Modern Data Architecture Includes Hadoop Page 7 © Hortonworks Inc. 2014 Hadoop compliments and enhances existing technologies Common data set, multiple applications • Optionally land all data in a single cluster • Batch, interactive & real-time use cases • Support multi-tenant access, processing & segmentation of data YARN: Architectural center of Hadoop • Consistent security, governance & operations • Ecosystem applications certified " by Hortonworks to run natively in Hadoop SOURCES EXISTING Systems Clickstream Web &Social Batch Interactive Real-Time HDFS (Hadoop Distributed File System) Geoloca9on Sensor & Machine Server Logs Unstructured DATA SYSTEM APPLICATIONS Business Analytics Custom Applications Packaged Applications RDBMS EDW MPP YARN: Data Operating System 1 ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° ° N
  • 8. Unlock New Applications from New Types of Data INDUSTRY USE CASE Sentiment Page 8 © Hortonworks Inc. 2014 & Web Clickstream & Behavior Machine & Sensor Geographic Server Logs Structured & Unstructured Financial Services New Account Risk Screens ✔ ✔ Trading Risk ✔ Insurance Underwriting ✔ ✔ ✔ Telecom Call Detail Records (CDR) ✔ ✔ Infrastructure Investment ✔ ✔ Real-time Bandwidth Allocation ✔ ✔ ✔ Retail 360° View of the Customer ✔ ✔ ✔ Localized, Personalized Promotions ✔ Website Optimization ✔ Manufacturing Supply Chain and Logistics ✔ Assembly Line Quality Assurance ✔ Crowd-sourced Quality Assurance ✔ Healthcare Use Genomic Data in Medial Trials ✔ ✔ ✔ Monitor Patient Vitals in Real-Time ✔ ✔ Pharmaceuticals Recruit and Retain Patients for Drug Trials ✔ ✔ Improve Prescription Adherence ✔ ✔ ✔ ✔ Oil & Gas Unify Exploration & Production Data ✔ ✔ ✔ ✔ Monitor Rig Safety in Real-Time ✔ ✔ ✔ Government ETL Offload/Federal Budgetary Pressures ✔ ✔ Sentiment Analysis for Government Programs ✔
  • 9. Break Down Silos with a Security Data Lake SCALE Page 9 © Hortonworks Inc. 2014 SCOPE Unlocking the Data Lake RDBMS MPP EDW • Data Lake Enabled by YARN • Single data repository, shared infrastructure • Multiple security apps accessing all the data • Enable a shift from reactive to proactive interactions • Gain new insight across the entire enterprise New Analytic Apps or IT Optimization HDP 2.1 Governance & Integration Security Operations Data Access YARN Data Management
  • 10. Big Data is Changing Cyber Security “By 2016, more than 25 percent of global firms will adopt big data analytics for at least one security and fraud detection use case, up from current eight percent.” – Gartner Cyber Security report Feb 2014 Gartner recommendations • Align security capabilities in a holistic security strategy tailored to the threats and risks • Target a single architecture to collect, index, normalize, analyze and share all information • Organizations should look for profile accounts, users or other entities, and look for anomalous transactions against those profiles Page 10 © Hortonworks Inc. 2014
  • 11. How Can Big Data Analytics Help Cyber Security? • To prioritize threats, vulnerabilities, and attacks • To control endpoints and mobile connections /devices • To prevent insecure devices from accessing secure systems • To provide intelligence about the threat land- scape • To reduce false positives Page 11 © Hortonworks Inc. 2014
  • 12. Page 12 © Hortonworks Inc. 2014 Securely explore your data CYBER PATTERN DISCOVERY USING LINKED DATA ANALYSIS A Big Data Solution with Hortonworks and Sqrrl Joe Travaglini, Director of Products, Sqrrl © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 13. Who We Are Page 13 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 14. Agenda • Security Analytics using (Big) Cybersecurity Data • Dealing with the new security dilemma • Why Hadoop and HDP are the perfect fit • The ‘Linked Data’ Approach • Case study: internal network breach • Overview of scenario • Data modeling with Sqrrl • Visual, contextual research and analysis Page 14 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 15. The Numbers Don’t Lie 229 87% 90% $12.7M Page 15 © Hortonworks Inc. 2014 Source: Mandiant Source: Verizon Source: Verizon Source: Ponemon © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 16. Targeted Attacks Have Changed the Game Page 16 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved Source: Battery Ventures
  • 17. What Does This Mean For Us? Dissolution of the Secure Perimeter • You’ve been breached. Deal with it. • Empower the investigator • Research and respond: better, faster, smarter • It’s all about speed to understanding Page 17 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 18. The Security Data Dilemma Detecting attacks requires more (i.e. BIG) data Page 18 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved But your tools can’t handle the big data wave So attackers are spilling in
  • 19. A Modern Data Architecture Hortonworks and Sqrrl Solution • Hortonworks Data Platform at the core • Sqrrl Enterprise stack at the app layer Hadoop enables us to look at data differently Page 19 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 20. Sqrrl Enterprise Architecture Interface Processing Data Model Data Storage Page 20 © Hortonworks Inc. 2014 Visualization / API ML + Anomaly Detection Query Engine Bulk/Graph Processing Raw Events Linked Data Model HDFS + Accumulo Commodity Hardware © 2014 Sqrrl Data, Inc. | All Rights Reserved Physical Security Audit Cryptography Labeling + Policy
  • 21. Big Data Transformed Data Sources Linked Contextual Knowledge Analysis Email Page 21 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved Security Data VPN FW Network Data Proxy NetFlow Application Data HR USB
  • 22. Linked Data Analysis Adding structure to the noise Page 22 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 23. Page 23 © Hortonworks Inc. 2014 Case Study: Compromised Network © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 24. Breach Detection Scenario Page 24 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 25. Case Study Model Data Sources Page 25 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved Linked Meta Model Users Hosts login flow login DNS records Netflow Host logs Database logs External Alerts
  • 26. Case Study Example Mapping Page 26 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved Netflow Records startTime endTime sourceIP destIP sourcePort destPort protocol tcpFlags bytesIn bytesOut 10/22/14 8:58 10/22/14 8:58 10.0.2.15 192.168.0.123 37051 139 TCP ...RS. 100 3355 10/22/14 8:45 10/22/14 8:45 10.0.2.15 192.168.0.6 0 3328 ICMP ...... 40 100 10/22/14 8:59 10/22/14 8:59 192.168.0.119 10.0.2.15 139 60071 TCP .A..S. 46 351 10.0.2.15 192.168. 0.123 Class=Flow, totalBytes = 3455 192.168. Class=Flow, 0.6 totalBytes = 140
  • 27. Case Study Example Data Page 27 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 28. Investigation Process 1. Set the Stage 2. Enable Search Page 28 © Hortonworks Inc. 2014 and Discovery © 2014 Sqrrl Data, Inc. | All Rights Reserved 3. Automate Analysis • Define the security-centric entity/ relationship model • Extract and maintain the model • Visually navigate assets and actors in the network • Drill down to the raw data seeding the model • Use behavioral analytics to build expectations of ‘normal’ • Flag entities as potentially ‘abnormal’ and sniff them out
  • 29. Page 29 © Hortonworks Inc. 2014 Visualizing the Threat © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 30. Page 30 © Hortonworks Inc. 2014 © 2014 Sqrrl Data, Inc. | All Rights Reserved
  • 31. Page 31 © Hortonworks Inc. 2014 Thanks! © 2014 Sqrrl Data, Inc. | All Rights Reserved Joe Travaglini Director of Products, Sqrrl Data, Inc. @joe_travaglini jtrav@sqrrl.com http://www.sqrrl.com