SlideShare ist ein Scribd-Unternehmen logo
1 von 20
Page1 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Big Data in Financial Services
We Do Hadoop
Page2 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Global banking trends
Source: E&Y
Page3 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Key focus areas within the financial services industry
Internal domains..
Please note that this is not a comprehensive list of deployed use-cases
across the major domains, just the major areas in which industry shifts are
occurring and where customers are looking to deploy enterprise Big Data in.
External Customer facing domains..
Risk Mgmt
Cyber
Security
Fraud
Detection
Data
Compliance
Digital Banking
360
degree view
Capital Markets
Retail Banking
and Lending
Credit Cards;
Payment Networks
Wealth Mgmt
Corporate Banking
Asset Mgmt
(Brokerage, MF and
Stock exchanges)
Page4 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
My Experience in Risk & Compliance
Mike DeSanti
November 4th, 2015
Page5 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
• The move from proprietary to flow-based trading in Capital Markets is affecting
revenue
• Fintech companies are taking over traditional revenue sources in the lending and
payments spaces
• The millenials reliance on mobile technology and instant gratification
• Increased regulatory spending is strangling discretionary spending
• Having to deal with very dated and expensive IT infrastructure
Issues Affecting Most Banks
Page6 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
• Fragmented Book of Record Transaction systems
– Lending systems along geographic and business lines
– Trading systems along desk and geographic lines
• Fragmented enterprise systems
– Multiple general ledgers
– Multiple risk systems by risk function
• Credit limit management, traded credit, Basel capital systems, CVA, Market Risk VaR, Stress VaR, Market Risk reporting…
– Multiple compliance systems by business line by compliance initiative
• AML for Retail, AML for Commercial Lending, AML for Capital Markets…
• They are full of proprietary vendor and in-house built solutions that have been
acquired over the years
What I Have Seen at Banks
Page7 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Leads to Current State Complexity
• Thousands of point-to point feeds to each
enterprise system from each transaction
system
• Data is independently sourced leading to
timing and data lineage issues
• Close processes are complicated and
error prone
• Reconciliation requires a large effort and
has significant gaps
Book of Record Transaction Systems
Enterprise Risk, Compliance and Finance Systems
Page8 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
• Centralize business and operation functions
• Incentivize technology to shrink not grow
• Populate a data lake that with a set of canonical feeds from the transaction
systems
• Create a linearly scalable platform to host enterprise applications on top of this
data lake, including hot, warm and cold computing zones
• Develop a partnership with Hortonworks to evolve the platform to meet the
Banking Industry’s needs
How Do We Change?
Page9 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
• A free open source linearly scalable platform has only become available within
the last few years
• Due to the amount of regulation over the last 15 years all bank enterprise
compliance, risk and finance systems now function essentially the same way
• Banks partnering with an open source partner is very different from partnering
with a vendor who develops proprietary software
• Proprietary software vendors will adopt the new standards since it is in their self
interest to do so
• Regulators can now streamline their regulatory practices by adopting this
platform
Why Will This Work Now?
Page10 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Open Platform for Risk & Compliance
David Lattimore-Gay (Eikos Partners)
Page11 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
What is OPRC ?
• Leveraging Open Source
– Large Development Community
– Innovation
– Reuse not build
• Commodity hardware
– Data Storage
– In Memory Cache
– Computing Power
• Light weight way for ingesting and storing data
– Single Source Data
– Multiple views on the same data with Schema on Read capability
– Built in data lineage
• Unified development environment for analytics
– Partnership between quants and IT developers that will allow IT to package analytics for
deployment rather than recoding them
Page12 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Data Lake
OPRC Originating Source System
Source System
Canonica
l
L1
Mapped
to
Schema
Cache
Reference / Transaction Data / Market Data / Static Data / Meta Data
Reference Data
Validation
Normalization
PK/AK
What has
changed ?
Load
L0
Raw Data
Reporting
Derived
Data
Page13 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Compute Service
Grid
Web Services
UI Framework
In Memory
API
Data
Fabric
Desktop Mobile
Data Access
API
Task
Calculator Framework
(Job Create/Monitor/Control)
Task
Task
TaskTask Task
Task
Task
Compute Service Strategy Engine
Task Dependency
Scheduler
Task Dependency
Scheduler
Task
Aggregation &
Reporting Engine
Data
Lake
Standardized Book of
Record Transaction
(BORT) & Other
Systems Feeds
ETL
Framework
OPRC High Level Architecture
Page14 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Technology Architecture
Nadeem Asghar
Page15 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
OPRC Functional Requirements
LANDING DATA
ZONE
L0
STANDARDIZED
DATA ZONE
L1
CANONICAL
DATA ZONE
L2 Regulatory
Reports
Internal
Reports
External
Reports
Search
REPORTING/
ANALYTICS ZONE
L3
Golden Source &
Feeds
Master Data
Contrats
Balances
Transaction
Positions
Factors/
Scenarios
Market Data
Unstructured
Data
Original
Data
RAW Data
Standardized
Data
Materialized
View
Extract & Load
Extract & Load
Connectors/
Adaptors
Standardized
Data
DQ & Validation
Normalization
Mapping &
Schemas
Data Cleansing
Standardized
Data
Data Join/merging
Data
Transformation
Business Rules
Model Rep &
Validation
Canonical
Position
Data
Canonical
Transaction
Data
Data Aggregation
Calculators
Scenarios Run
Revision Hisotry
Scenarios
Results
Data
Aggregations
Analytics/
Reports
Revision
History
Common Repositories/Meta Data Management
Security
Connectors Schema Mapping Catalogging End to End Lineage Tracing
Authenication RBAC Authorization Encryption Audit
Page16 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Proposed Solution based on Hortonworks Stack
Hortonworks Data Platform
LANDING DATA
ZONE
L0
STANDARDIZED
DATA ZONE
L1
CANONICAL
DATA ZONE
L2 Regulatory
Reports
Internal
Reports
External
Reports
Search
REPORTING/
ANALYTICS ZONE
L3
Golden Source &
Feeds
Master Data
Contrats
Balances
Transaction
Positions
Factors/
Scenarios
Market Data
Unstructured
Data
(hdfs)
Original
Data
(hdfs)
RAW Data
hdfs)
Standardized
Data
(Hive/orc)
Materialized
View
(Hive/orc)
sqoop/hadoop fs/
nfs
Kafka/Storm
Java/Scala
Standardized
Data
(Hive/Orc)
Hive/Spark/Scala
Hive/Spark/Scala
MHive/Spark/Scal
Hive/Spark/Scala
Standardized
Data
(Hive/Orc)
Hive/Spark/Scala
Hive/Spark/Scala
Hive/Spark/Scala
TBD??
Canonical
Position
Data
(hive/orc)
Canonical
Transaction
Data
(Hive/orc)
Hive/Spark
Scala/Python/R
etc
Scala/Java
Hive/Spark
Scenarios
Results
(Hive/orc)
Data
Aggregations
(Hive/orc/
Hbase)
Analytics/
Reports
(Hive/orc/
HBase)
Revision
History
(Hive/orc)
Common Repositories/Meta Data Management
Security
Apache Atlas/Falcon/ Custom Solution
Apache Ranger/ Atlas and Custom/Partner Solution
Page17 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
OPRC Game Plan:
Solution Based on IP(Above the line)/ Open Source(Below the Line)
Page18 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Wrap up..
Page19 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Hadoop is a Platform Decision
Adoption follows a consistent journey
Data architecture efficiencies, new analytic apps, and
ultimately to a “data lake”.
HDP: A centralized architecture built on YARN
Any application, any data, anywhere.
HDP: A completely open data platform
Platforms are ultimately defined by open communities.
HDP subscription supports entire lifecycle
World class experience to ensure success from architecture to
production to expansion.
Page20 © Hortonworks Inc. 2011 – 2015. All Rights Reserved
Cautionary Statement Regarding Forward-Looking Statements
This presentation contains forward-looking statements involving risks and uncertainties. Such
forward-looking statements in this presentation generally relate to future events, our ability to
increase the number of support subscription customers, the growth in usage of the Hadoop
framework, our ability to innovate and develop the various open source projects that will enhance
the capabilities of the Hortonworks Data Platform, anticipated customer benefits and general
business outlook. In some cases, you can identify forward-looking statements because they
contain words such as “may,” “will,” “should,” “expects,” “plans,” “anticipates,” “could,” “intends,”
“target,” “projects,” “contemplates,” “believes,” “estimates,” “predicts,” “potential” or “continue” or
similar terms or expressions that concern our expectations, strategy, plans or intentions. You
should not rely upon forward-looking statements as predictions of future events. We have based
the forward-looking statements contained in this presentation primarily on our current expectations
and projections about future events and trends that we believe may affect our business, financial
condition and prospects. We cannot assure you that the results, events and circumstances
reflected in the forward-looking statements will be achieved or occur, and actual results, events, or
circumstances could differ materially from those described in the forward-looking statements.
The forward-looking statements made in this prospectus relate only to events as of the date on
which the statements are made and we undertake no obligation to update any of the information in
this presentation.
Trademarks
Hortonworks is a trademark of Hortonworks, Inc. in the United States and other jurisdictions. Other
names used herein may be trademarks of their respective owners.

Weitere ähnliche Inhalte

Was ist angesagt?

Leveraging Big Data to Drive Bank Customer Engagement and Loyalty
Leveraging Big Data to Drive Bank Customer Engagement and LoyaltyLeveraging Big Data to Drive Bank Customer Engagement and Loyalty
Leveraging Big Data to Drive Bank Customer Engagement and LoyaltyJim Marous
 
Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...
Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...
Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...Molly Alexander
 
Big Data & Analytics perspectives in Banking
Big Data & Analytics perspectives in BankingBig Data & Analytics perspectives in Banking
Big Data & Analytics perspectives in BankingGianpaolo Zampol
 
Lead to Cash: The Value of Big Data and Analytics for Telco
Lead to Cash: The Value of Big Data and Analytics for TelcoLead to Cash: The Value of Big Data and Analytics for Telco
Lead to Cash: The Value of Big Data and Analytics for TelcoSam Thomsett
 
Business Intelligence and Analytics in Banking
Business Intelligence and Analytics in BankingBusiness Intelligence and Analytics in Banking
Business Intelligence and Analytics in BankingSesameindia Pvt Ltd
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Business Over Broadway
 
Pi cube banking on predictive analytics151
Pi cube   banking on predictive analytics151Pi cube   banking on predictive analytics151
Pi cube banking on predictive analytics151Cole Capital
 
IBM Solutions Connect 2013 - Getting started with Big Data
IBM Solutions Connect 2013 - Getting started with Big DataIBM Solutions Connect 2013 - Getting started with Big Data
IBM Solutions Connect 2013 - Getting started with Big DataIBM Software India
 
201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut
201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut
201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide ShutFrancisco Calzado
 
The Journey to Big Data Analytics
The Journey to Big Data AnalyticsThe Journey to Big Data Analytics
The Journey to Big Data AnalyticsDr.Stefan Radtke
 
Building new business models through big data dec 06 2012
Building new business models through big data   dec 06 2012Building new business models through big data   dec 06 2012
Building new business models through big data dec 06 2012Aki Balogh
 
BI & Big data use case for banking - by rully feranata
BI & Big data use case for banking - by rully feranataBI & Big data use case for banking - by rully feranata
BI & Big data use case for banking - by rully feranataRully Feranata
 
Big Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT INGBig Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT INGMatt Stubbs
 
M&A Trends in Telco Analytics
M&A Trends in Telco AnalyticsM&A Trends in Telco Analytics
M&A Trends in Telco AnalyticsOpen Analytics
 
Top 10 digital transformation trends for healthcare in 2022
Top 10 digital transformation trends for healthcare in 2022Top 10 digital transformation trends for healthcare in 2022
Top 10 digital transformation trends for healthcare in 2022IndusNetMarketing
 
Fueling the future of fintech with data science and ai
Fueling the future of fintech with data science and aiFueling the future of fintech with data science and ai
Fueling the future of fintech with data science and aiIndusNetMarketing
 
7 Steps Big Data Journey for Enterprises
7 Steps Big Data Journey for Enterprises7 Steps Big Data Journey for Enterprises
7 Steps Big Data Journey for EnterprisesRaju Shreewastava
 
Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...
Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...
Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...SigortaTatbikatcilariDernegi
 

Was ist angesagt? (20)

Leveraging Big Data to Drive Bank Customer Engagement and Loyalty
Leveraging Big Data to Drive Bank Customer Engagement and LoyaltyLeveraging Big Data to Drive Bank Customer Engagement and Loyalty
Leveraging Big Data to Drive Bank Customer Engagement and Loyalty
 
Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...
Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...
Towards the Next Generation Financial Crimes Platform - How Data, Analytics, ...
 
Big Data & Analytics perspectives in Banking
Big Data & Analytics perspectives in BankingBig Data & Analytics perspectives in Banking
Big Data & Analytics perspectives in Banking
 
Lead to Cash: The Value of Big Data and Analytics for Telco
Lead to Cash: The Value of Big Data and Analytics for TelcoLead to Cash: The Value of Big Data and Analytics for Telco
Lead to Cash: The Value of Big Data and Analytics for Telco
 
Business Intelligence and Analytics in Banking
Business Intelligence and Analytics in BankingBusiness Intelligence and Analytics in Banking
Business Intelligence and Analytics in Banking
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
 
Pi cube banking on predictive analytics151
Pi cube   banking on predictive analytics151Pi cube   banking on predictive analytics151
Pi cube banking on predictive analytics151
 
IBM Solutions Connect 2013 - Getting started with Big Data
IBM Solutions Connect 2013 - Getting started with Big DataIBM Solutions Connect 2013 - Getting started with Big Data
IBM Solutions Connect 2013 - Getting started with Big Data
 
201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut
201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut
201407 Global Insights and Actions for Banks in the Digital Age - Eyes Wide Shut
 
The Journey to Big Data Analytics
The Journey to Big Data AnalyticsThe Journey to Big Data Analytics
The Journey to Big Data Analytics
 
Building new business models through big data dec 06 2012
Building new business models through big data   dec 06 2012Building new business models through big data   dec 06 2012
Building new business models through big data dec 06 2012
 
IBM Big Data Platform Nov 2012
IBM Big Data Platform Nov 2012IBM Big Data Platform Nov 2012
IBM Big Data Platform Nov 2012
 
BI & Big data use case for banking - by rully feranata
BI & Big data use case for banking - by rully feranataBI & Big data use case for banking - by rully feranata
BI & Big data use case for banking - by rully feranata
 
Big Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT INGBig Data LDN 2018: DATA SCIENCE AT ING
Big Data LDN 2018: DATA SCIENCE AT ING
 
Banking Big Data Analytics
Banking Big Data AnalyticsBanking Big Data Analytics
Banking Big Data Analytics
 
M&A Trends in Telco Analytics
M&A Trends in Telco AnalyticsM&A Trends in Telco Analytics
M&A Trends in Telco Analytics
 
Top 10 digital transformation trends for healthcare in 2022
Top 10 digital transformation trends for healthcare in 2022Top 10 digital transformation trends for healthcare in 2022
Top 10 digital transformation trends for healthcare in 2022
 
Fueling the future of fintech with data science and ai
Fueling the future of fintech with data science and aiFueling the future of fintech with data science and ai
Fueling the future of fintech with data science and ai
 
7 Steps Big Data Journey for Enterprises
7 Steps Big Data Journey for Enterprises7 Steps Big Data Journey for Enterprises
7 Steps Big Data Journey for Enterprises
 
Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...
Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...
Digital Transformation for Insurance and Underwriting Processes - Caroly Mart...
 

Andere mochten auch

IBM Banking videocast - 3/20/2013
IBM Banking videocast - 3/20/2013 IBM Banking videocast - 3/20/2013
IBM Banking videocast - 3/20/2013 Casey Lucas
 
FircoSoft Company Overview
FircoSoft Company OverviewFircoSoft Company Overview
FircoSoft Company OverviewFircoSoft
 
Banking application architecture mishra
Banking application architecture mishraBanking application architecture mishra
Banking application architecture mishraAjay Mishra
 
IBM Industry Models and Data Lake
IBM Industry Models and Data Lake IBM Industry Models and Data Lake
IBM Industry Models and Data Lake Pat O'Sullivan
 
Profiting from customer profitability + big data fitzgerald analytics
Profiting from customer profitability + big data fitzgerald analyticsProfiting from customer profitability + big data fitzgerald analytics
Profiting from customer profitability + big data fitzgerald analyticsFitzgerald Analytics, Inc.
 
The cognitive bank ibm launch deck 2016
The cognitive bank ibm launch deck 2016The cognitive bank ibm launch deck 2016
The cognitive bank ibm launch deck 2016Charlie Chan
 
Combating Constantly Evolving Advanced Threats – Solution Architecture
Combating Constantly Evolving Advanced Threats – Solution ArchitectureCombating Constantly Evolving Advanced Threats – Solution Architecture
Combating Constantly Evolving Advanced Threats – Solution ArchitectureIBM Sverige
 
Logical Data Warehouse and Data Lakes
Logical Data Warehouse and Data Lakes Logical Data Warehouse and Data Lakes
Logical Data Warehouse and Data Lakes Denodo
 
Ibm big data ibm marriage of hadoop and data warehousing
Ibm big dataibm marriage of hadoop and data warehousingIbm big dataibm marriage of hadoop and data warehousing
Ibm big data ibm marriage of hadoop and data warehousing DataWorks Summit
 
Core Banking Transformation: Solutions to Standardize Processes and Cut Costs
Core Banking Transformation: Solutions to Standardize Processes and Cut CostsCore Banking Transformation: Solutions to Standardize Processes and Cut Costs
Core Banking Transformation: Solutions to Standardize Processes and Cut CostsIBM Banking
 
Capgemini Smart Analytics Solutions Platform for Banking
Capgemini Smart Analytics Solutions Platform for BankingCapgemini Smart Analytics Solutions Platform for Banking
Capgemini Smart Analytics Solutions Platform for BankingCapgemini
 
Big Data in Banking (Data Science Thailand Meetup #2)
Big Data in Banking (Data Science Thailand Meetup #2)Big Data in Banking (Data Science Thailand Meetup #2)
Big Data in Banking (Data Science Thailand Meetup #2)Data Science Thailand
 
Big Data Analytics for Banking, a Point of View
Big Data Analytics for Banking, a Point of ViewBig Data Analytics for Banking, a Point of View
Big Data Analytics for Banking, a Point of ViewPietro Leo
 
Data Lake: A simple introduction
Data Lake: A simple introductionData Lake: A simple introduction
Data Lake: A simple introductionIBM Analytics
 
Ibm cognitive commerce for banking final
Ibm cognitive commerce for banking finalIbm cognitive commerce for banking final
Ibm cognitive commerce for banking finalChris Yaldezian
 
Top Ten Trends in Banking 2017
Top Ten Trends in Banking 2017Top Ten Trends in Banking 2017
Top Ten Trends in Banking 2017Capgemini
 

Andere mochten auch (20)

IBM Banking videocast - 3/20/2013
IBM Banking videocast - 3/20/2013 IBM Banking videocast - 3/20/2013
IBM Banking videocast - 3/20/2013
 
FircoSoft Company Overview
FircoSoft Company OverviewFircoSoft Company Overview
FircoSoft Company Overview
 
Banking application architecture mishra
Banking application architecture mishraBanking application architecture mishra
Banking application architecture mishra
 
IBM Industry Models and Data Lake
IBM Industry Models and Data Lake IBM Industry Models and Data Lake
IBM Industry Models and Data Lake
 
Profiting from customer profitability + big data fitzgerald analytics
Profiting from customer profitability + big data fitzgerald analyticsProfiting from customer profitability + big data fitzgerald analytics
Profiting from customer profitability + big data fitzgerald analytics
 
The cognitive bank ibm launch deck 2016
The cognitive bank ibm launch deck 2016The cognitive bank ibm launch deck 2016
The cognitive bank ibm launch deck 2016
 
Combating Constantly Evolving Advanced Threats – Solution Architecture
Combating Constantly Evolving Advanced Threats – Solution ArchitectureCombating Constantly Evolving Advanced Threats – Solution Architecture
Combating Constantly Evolving Advanced Threats – Solution Architecture
 
Smart data for a predictive bank
Smart data for a predictive bankSmart data for a predictive bank
Smart data for a predictive bank
 
Logical Data Warehouse and Data Lakes
Logical Data Warehouse and Data Lakes Logical Data Warehouse and Data Lakes
Logical Data Warehouse and Data Lakes
 
Ibm big data ibm marriage of hadoop and data warehousing
Ibm big dataibm marriage of hadoop and data warehousingIbm big dataibm marriage of hadoop and data warehousing
Ibm big data ibm marriage of hadoop and data warehousing
 
Core Banking Transformation: Solutions to Standardize Processes and Cut Costs
Core Banking Transformation: Solutions to Standardize Processes and Cut CostsCore Banking Transformation: Solutions to Standardize Processes and Cut Costs
Core Banking Transformation: Solutions to Standardize Processes and Cut Costs
 
Big Data Application Architectures - Fraud Detection
Big Data Application Architectures - Fraud DetectionBig Data Application Architectures - Fraud Detection
Big Data Application Architectures - Fraud Detection
 
Capgemini Smart Analytics Solutions Platform for Banking
Capgemini Smart Analytics Solutions Platform for BankingCapgemini Smart Analytics Solutions Platform for Banking
Capgemini Smart Analytics Solutions Platform for Banking
 
Big Data in Banking (Data Science Thailand Meetup #2)
Big Data in Banking (Data Science Thailand Meetup #2)Big Data in Banking (Data Science Thailand Meetup #2)
Big Data in Banking (Data Science Thailand Meetup #2)
 
Big Data Analytics for Banking, a Point of View
Big Data Analytics for Banking, a Point of ViewBig Data Analytics for Banking, a Point of View
Big Data Analytics for Banking, a Point of View
 
Data Lake: A simple introduction
Data Lake: A simple introductionData Lake: A simple introduction
Data Lake: A simple introduction
 
Ibm cognitive commerce for banking final
Ibm cognitive commerce for banking finalIbm cognitive commerce for banking final
Ibm cognitive commerce for banking final
 
Core banking
Core bankingCore banking
Core banking
 
Core banking
Core bankingCore banking
Core banking
 
Top Ten Trends in Banking 2017
Top Ten Trends in Banking 2017Top Ten Trends in Banking 2017
Top Ten Trends in Banking 2017
 

Ähnlich wie Big Data in Financial Services

The path to a Modern Data Architecture in Financial Services
The path to a Modern Data Architecture in Financial ServicesThe path to a Modern Data Architecture in Financial Services
The path to a Modern Data Architecture in Financial ServicesHortonworks
 
Hortonworks - IBM Cognitive - The Future of Data Science
Hortonworks - IBM Cognitive - The Future of Data ScienceHortonworks - IBM Cognitive - The Future of Data Science
Hortonworks - IBM Cognitive - The Future of Data ScienceThiago Santiago
 
2015 02 12 talend hortonworks webinar challenges to hadoop adoption
2015 02 12 talend hortonworks webinar challenges to hadoop adoption2015 02 12 talend hortonworks webinar challenges to hadoop adoption
2015 02 12 talend hortonworks webinar challenges to hadoop adoptionHortonworks
 
Big Data vs. Big Risk: Real-Time Trade Surveillance in Financial Markets
Big Data vs. Big Risk: Real-Time Trade Surveillance in Financial MarketsBig Data vs. Big Risk: Real-Time Trade Surveillance in Financial Markets
Big Data vs. Big Risk: Real-Time Trade Surveillance in Financial MarketsArcadia Data
 
Unleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on SalesforceUnleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on SalesforceDreamforce
 
Heroku + Salesforce = Partner Success
Heroku + Salesforce = Partner SuccessHeroku + Salesforce = Partner Success
Heroku + Salesforce = Partner SuccessAlexander Sutherland
 
How to Become an Analytics Ready Insurer - with Informatica and Hortonworks
How to Become an Analytics Ready Insurer - with Informatica and HortonworksHow to Become an Analytics Ready Insurer - with Informatica and Hortonworks
How to Become an Analytics Ready Insurer - with Informatica and HortonworksHortonworks
 
Webinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalWebinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalHortonworks
 
Webinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalWebinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalHortonworks
 
O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...
O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...
O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...DataWorks Summit
 
Hortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with HadoopHortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with HadoopMats Johansson
 
Hortonworks - How Hadoop makes the successful Retailer.
Hortonworks - How Hadoop makes the successful Retailer. Hortonworks - How Hadoop makes the successful Retailer.
Hortonworks - How Hadoop makes the successful Retailer. Mats Johansson
 
Analytics in the Cloud and the ROI for B2B
Analytics in the Cloud and the ROI for B2BAnalytics in the Cloud and the ROI for B2B
Analytics in the Cloud and the ROI for B2BVeronica Kirn
 
Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...
Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...
Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...Flink Forward
 
Processing Big Data At-Scale in the App Cloud
Processing Big Data At-Scale in the App CloudProcessing Big Data At-Scale in the App Cloud
Processing Big Data At-Scale in the App CloudSalesforce Developers
 
Insights on Procurement Data with Analytics Solutions from SAP
Insights on Procurement Data with Analytics Solutions from SAPInsights on Procurement Data with Analytics Solutions from SAP
Insights on Procurement Data with Analytics Solutions from SAPSAP Ariba
 

Ähnlich wie Big Data in Financial Services (20)

The path to a Modern Data Architecture in Financial Services
The path to a Modern Data Architecture in Financial ServicesThe path to a Modern Data Architecture in Financial Services
The path to a Modern Data Architecture in Financial Services
 
Hortonworks - IBM Cognitive - The Future of Data Science
Hortonworks - IBM Cognitive - The Future of Data ScienceHortonworks - IBM Cognitive - The Future of Data Science
Hortonworks - IBM Cognitive - The Future of Data Science
 
2015 02 12 talend hortonworks webinar challenges to hadoop adoption
2015 02 12 talend hortonworks webinar challenges to hadoop adoption2015 02 12 talend hortonworks webinar challenges to hadoop adoption
2015 02 12 talend hortonworks webinar challenges to hadoop adoption
 
Big Data vs. Big Risk: Real-Time Trade Surveillance in Financial Markets
Big Data vs. Big Risk: Real-Time Trade Surveillance in Financial MarketsBig Data vs. Big Risk: Real-Time Trade Surveillance in Financial Markets
Big Data vs. Big Risk: Real-Time Trade Surveillance in Financial Markets
 
Unleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on SalesforceUnleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on Salesforce
 
Heroku + Salesforce = Partner Success
Heroku + Salesforce = Partner SuccessHeroku + Salesforce = Partner Success
Heroku + Salesforce = Partner Success
 
How to Become an Analytics Ready Insurer - with Informatica and Hortonworks
How to Become an Analytics Ready Insurer - with Informatica and HortonworksHow to Become an Analytics Ready Insurer - with Informatica and Hortonworks
How to Become an Analytics Ready Insurer - with Informatica and Hortonworks
 
Webinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalWebinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_final
 
Webinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_finalWebinar turbo charging_data_science_hawq_on_hdp_final
Webinar turbo charging_data_science_hawq_on_hdp_final
 
O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...
O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...
O2’s Financial Data Hub: going beyond IFRS compliance to support digital tran...
 
Sandeep_Rampalle_Resume
Sandeep_Rampalle_ResumeSandeep_Rampalle_Resume
Sandeep_Rampalle_Resume
 
Hortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with HadoopHortonworks & Bilot Data Driven Transformations with Hadoop
Hortonworks & Bilot Data Driven Transformations with Hadoop
 
Hortonworks - How Hadoop makes the successful Retailer.
Hortonworks - How Hadoop makes the successful Retailer. Hortonworks - How Hadoop makes the successful Retailer.
Hortonworks - How Hadoop makes the successful Retailer.
 
Retail Banking
Retail BankingRetail Banking
Retail Banking
 
Analytics in the Cloud and the ROI for B2B
Analytics in the Cloud and the ROI for B2BAnalytics in the Cloud and the ROI for B2B
Analytics in the Cloud and the ROI for B2B
 
Greenplum User Case
Greenplum User Case Greenplum User Case
Greenplum User Case
 
Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...
Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...
Flink Forward Berlin 2017 Keynote: Ferd Scheepers - Taking away customer fric...
 
Processing Big Data At-Scale in the App Cloud
Processing Big Data At-Scale in the App CloudProcessing Big Data At-Scale in the App Cloud
Processing Big Data At-Scale in the App Cloud
 
Insights on Procurement Data with Analytics Solutions from SAP
Insights on Procurement Data with Analytics Solutions from SAPInsights on Procurement Data with Analytics Solutions from SAP
Insights on Procurement Data with Analytics Solutions from SAP
 
Org Merge Best Practices
Org Merge Best PracticesOrg Merge Best Practices
Org Merge Best Practices
 

Kürzlich hochgeladen

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProduct Anonymous
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKJago de Vreede
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native ApplicationsWSO2
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfOverkill Security
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWERMadyBayot
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Angeliki Cooney
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusZilliz
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesrafiqahmad00786416
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfsudhanshuwaghmare1
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobeapidays
 

Kürzlich hochgeladen (20)

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 

Big Data in Financial Services

  • 1. Page1 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Big Data in Financial Services We Do Hadoop
  • 2. Page2 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Global banking trends Source: E&Y
  • 3. Page3 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Key focus areas within the financial services industry Internal domains.. Please note that this is not a comprehensive list of deployed use-cases across the major domains, just the major areas in which industry shifts are occurring and where customers are looking to deploy enterprise Big Data in. External Customer facing domains.. Risk Mgmt Cyber Security Fraud Detection Data Compliance Digital Banking 360 degree view Capital Markets Retail Banking and Lending Credit Cards; Payment Networks Wealth Mgmt Corporate Banking Asset Mgmt (Brokerage, MF and Stock exchanges)
  • 4. Page4 © Hortonworks Inc. 2011 – 2015. All Rights Reserved My Experience in Risk & Compliance Mike DeSanti November 4th, 2015
  • 5. Page5 © Hortonworks Inc. 2011 – 2015. All Rights Reserved • The move from proprietary to flow-based trading in Capital Markets is affecting revenue • Fintech companies are taking over traditional revenue sources in the lending and payments spaces • The millenials reliance on mobile technology and instant gratification • Increased regulatory spending is strangling discretionary spending • Having to deal with very dated and expensive IT infrastructure Issues Affecting Most Banks
  • 6. Page6 © Hortonworks Inc. 2011 – 2015. All Rights Reserved • Fragmented Book of Record Transaction systems – Lending systems along geographic and business lines – Trading systems along desk and geographic lines • Fragmented enterprise systems – Multiple general ledgers – Multiple risk systems by risk function • Credit limit management, traded credit, Basel capital systems, CVA, Market Risk VaR, Stress VaR, Market Risk reporting… – Multiple compliance systems by business line by compliance initiative • AML for Retail, AML for Commercial Lending, AML for Capital Markets… • They are full of proprietary vendor and in-house built solutions that have been acquired over the years What I Have Seen at Banks
  • 7. Page7 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Leads to Current State Complexity • Thousands of point-to point feeds to each enterprise system from each transaction system • Data is independently sourced leading to timing and data lineage issues • Close processes are complicated and error prone • Reconciliation requires a large effort and has significant gaps Book of Record Transaction Systems Enterprise Risk, Compliance and Finance Systems
  • 8. Page8 © Hortonworks Inc. 2011 – 2015. All Rights Reserved • Centralize business and operation functions • Incentivize technology to shrink not grow • Populate a data lake that with a set of canonical feeds from the transaction systems • Create a linearly scalable platform to host enterprise applications on top of this data lake, including hot, warm and cold computing zones • Develop a partnership with Hortonworks to evolve the platform to meet the Banking Industry’s needs How Do We Change?
  • 9. Page9 © Hortonworks Inc. 2011 – 2015. All Rights Reserved • A free open source linearly scalable platform has only become available within the last few years • Due to the amount of regulation over the last 15 years all bank enterprise compliance, risk and finance systems now function essentially the same way • Banks partnering with an open source partner is very different from partnering with a vendor who develops proprietary software • Proprietary software vendors will adopt the new standards since it is in their self interest to do so • Regulators can now streamline their regulatory practices by adopting this platform Why Will This Work Now?
  • 10. Page10 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Open Platform for Risk & Compliance David Lattimore-Gay (Eikos Partners)
  • 11. Page11 © Hortonworks Inc. 2011 – 2015. All Rights Reserved What is OPRC ? • Leveraging Open Source – Large Development Community – Innovation – Reuse not build • Commodity hardware – Data Storage – In Memory Cache – Computing Power • Light weight way for ingesting and storing data – Single Source Data – Multiple views on the same data with Schema on Read capability – Built in data lineage • Unified development environment for analytics – Partnership between quants and IT developers that will allow IT to package analytics for deployment rather than recoding them
  • 12. Page12 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Data Lake OPRC Originating Source System Source System Canonica l L1 Mapped to Schema Cache Reference / Transaction Data / Market Data / Static Data / Meta Data Reference Data Validation Normalization PK/AK What has changed ? Load L0 Raw Data Reporting Derived Data
  • 13. Page13 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Compute Service Grid Web Services UI Framework In Memory API Data Fabric Desktop Mobile Data Access API Task Calculator Framework (Job Create/Monitor/Control) Task Task TaskTask Task Task Task Compute Service Strategy Engine Task Dependency Scheduler Task Dependency Scheduler Task Aggregation & Reporting Engine Data Lake Standardized Book of Record Transaction (BORT) & Other Systems Feeds ETL Framework OPRC High Level Architecture
  • 14. Page14 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Technology Architecture Nadeem Asghar
  • 15. Page15 © Hortonworks Inc. 2011 – 2015. All Rights Reserved OPRC Functional Requirements LANDING DATA ZONE L0 STANDARDIZED DATA ZONE L1 CANONICAL DATA ZONE L2 Regulatory Reports Internal Reports External Reports Search REPORTING/ ANALYTICS ZONE L3 Golden Source & Feeds Master Data Contrats Balances Transaction Positions Factors/ Scenarios Market Data Unstructured Data Original Data RAW Data Standardized Data Materialized View Extract & Load Extract & Load Connectors/ Adaptors Standardized Data DQ & Validation Normalization Mapping & Schemas Data Cleansing Standardized Data Data Join/merging Data Transformation Business Rules Model Rep & Validation Canonical Position Data Canonical Transaction Data Data Aggregation Calculators Scenarios Run Revision Hisotry Scenarios Results Data Aggregations Analytics/ Reports Revision History Common Repositories/Meta Data Management Security Connectors Schema Mapping Catalogging End to End Lineage Tracing Authenication RBAC Authorization Encryption Audit
  • 16. Page16 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Proposed Solution based on Hortonworks Stack Hortonworks Data Platform LANDING DATA ZONE L0 STANDARDIZED DATA ZONE L1 CANONICAL DATA ZONE L2 Regulatory Reports Internal Reports External Reports Search REPORTING/ ANALYTICS ZONE L3 Golden Source & Feeds Master Data Contrats Balances Transaction Positions Factors/ Scenarios Market Data Unstructured Data (hdfs) Original Data (hdfs) RAW Data hdfs) Standardized Data (Hive/orc) Materialized View (Hive/orc) sqoop/hadoop fs/ nfs Kafka/Storm Java/Scala Standardized Data (Hive/Orc) Hive/Spark/Scala Hive/Spark/Scala MHive/Spark/Scal Hive/Spark/Scala Standardized Data (Hive/Orc) Hive/Spark/Scala Hive/Spark/Scala Hive/Spark/Scala TBD?? Canonical Position Data (hive/orc) Canonical Transaction Data (Hive/orc) Hive/Spark Scala/Python/R etc Scala/Java Hive/Spark Scenarios Results (Hive/orc) Data Aggregations (Hive/orc/ Hbase) Analytics/ Reports (Hive/orc/ HBase) Revision History (Hive/orc) Common Repositories/Meta Data Management Security Apache Atlas/Falcon/ Custom Solution Apache Ranger/ Atlas and Custom/Partner Solution
  • 17. Page17 © Hortonworks Inc. 2011 – 2015. All Rights Reserved OPRC Game Plan: Solution Based on IP(Above the line)/ Open Source(Below the Line)
  • 18. Page18 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Wrap up..
  • 19. Page19 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Hadoop is a Platform Decision Adoption follows a consistent journey Data architecture efficiencies, new analytic apps, and ultimately to a “data lake”. HDP: A centralized architecture built on YARN Any application, any data, anywhere. HDP: A completely open data platform Platforms are ultimately defined by open communities. HDP subscription supports entire lifecycle World class experience to ensure success from architecture to production to expansion.
  • 20. Page20 © Hortonworks Inc. 2011 – 2015. All Rights Reserved Cautionary Statement Regarding Forward-Looking Statements This presentation contains forward-looking statements involving risks and uncertainties. Such forward-looking statements in this presentation generally relate to future events, our ability to increase the number of support subscription customers, the growth in usage of the Hadoop framework, our ability to innovate and develop the various open source projects that will enhance the capabilities of the Hortonworks Data Platform, anticipated customer benefits and general business outlook. In some cases, you can identify forward-looking statements because they contain words such as “may,” “will,” “should,” “expects,” “plans,” “anticipates,” “could,” “intends,” “target,” “projects,” “contemplates,” “believes,” “estimates,” “predicts,” “potential” or “continue” or similar terms or expressions that concern our expectations, strategy, plans or intentions. You should not rely upon forward-looking statements as predictions of future events. We have based the forward-looking statements contained in this presentation primarily on our current expectations and projections about future events and trends that we believe may affect our business, financial condition and prospects. We cannot assure you that the results, events and circumstances reflected in the forward-looking statements will be achieved or occur, and actual results, events, or circumstances could differ materially from those described in the forward-looking statements. The forward-looking statements made in this prospectus relate only to events as of the date on which the statements are made and we undertake no obligation to update any of the information in this presentation. Trademarks Hortonworks is a trademark of Hortonworks, Inc. in the United States and other jurisdictions. Other names used herein may be trademarks of their respective owners.

Hinweis der Redaktion

  1. Provides a single consolidated group of generic processors, running and open source data ingestion, persistence and parallel computing stack Eliminate computational grid licensing costs associated with Grid Computing technology Eliminate the data caching grid licensing costs associated with IMDGs Eliminate Relational Database and Data Appliance costs that are associated with Risk and Compliance applications. Eliminate the costs associated with ETL tools such as Data Stage and Ab Initio. Provides a unified development environment for analytics Use open source languages for analytic development such as R as well as supporting SAS and MatLab interfaces. Provide open source source code management and QA tools. Create a partnership between quants and IT developers that will allow IT to package analytics for deployment rather than recoding them Sharply decrease analytic development and test times by providing a calculator framework that provides data as a service Provides a light weight way for ingesting and storing data Store and record each risk and compliance groups transformations to the base data providing BCBS 239 compliance Support multiple views on the same data with Schema on Read capability Support multiple language access to the data via SQL, PIG, Scala and Java interfaces
  2. We use Cache to keep frequently access reference data for Normalization and Validation Data Green represents plugin’s to the flow, out of the box and also custom my business. Orange lines represent Connectivity back to the previous steps (Data Lineage) Is it possible that data loading and validation could be done in a MapReduce fashion ?? Need to be able to plug and play with the Source Systems, i.e. a new Source System can be added with no impact on existing feeds being consumed. Pluggable infrastructure L0 is OSS Specific, L1 defines the mandatory / optional columns needed for each entity type (trade, position, security), could be extended dependant on OSS, L2 is the final form, including any additional attributes, in a key value pair structure Entity, AttributeName, Value, DataType (+ Bi Temporal fields)
  3. High Level architecture, highlighting the components that are needed
  4. DLG: Diagram is too busy, there is a lot going on, so for me it just says busy
  5. DLG: again Diagram is too busy, there is a lot going on, so for me it just says busy
  6. Hundreds of organizations have turned to Hortonworks because Hadoop is ultimately a platform decision. It is typically the first step towards re-architecting your back end data systems. These organizations that have already been successful with Hadoop have required not just a stable, reliable and complete Hadoop solution, but more importantly a connection with the architects, builders and operators of this open source technology. They saw this in Hortonworks. And as with any platform decision, it is imperative that Hadoop integrates with the tools and systems that are already resident in your data center. We forge deep relationships with our hundreds of partners so that you can not only ensure integration but also effectively reapply existing systems and skillsets toward your big data challenges. At Hortonworks, we hold true to these foundational beliefs and have partnered with hundreds of organizations from some of the largest and earliest big data adopters to the most conservative and data rich companies on the planet. We ensure that your Hadoop journey is successful and more companies are turning to Hortonworks today than any other offering on the marketplace. We invite you to join our community.