SlideShare ist ein Scribd-Unternehmen logo
1 von 26
Downloaden Sie, um offline zu lesen
Processing Big Data At Scale
Naren Chawla
Senior Director, Product Management (nchawla@salesforce.com)
Prashant Kommireddi @prashant1784
Leverage platform-native Data Pipelines for ETL
Safe Harbor
Safe harbor statement under the Private Securities Litigation Reform Act of 1995:
This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if
any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward-
looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of
product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of
management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and
customer contracts or use of our services.
The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our
service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth,
interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any
possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and
motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non-
salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial
results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for
the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor
Information section of our Web site.
Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not
be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently
available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
Topics
Big Data Processing Problem and Proposed Solution
Data Pipeline Deep-dive
Demo
Key Use-cases
Customer Stories
Summary
Q&A
Problem
ERP
HCM SCM Logs
1. Acquire & Store Data
2. Prepare Data
(Cleanse, Augment, Transform, Join)
Data Lake /
EDW
4. Take Action
Customer Success Platform
3. Analyze
Wave
Firewall
•  Cost and complexity of
managing external data
platforms
•  Slow time-to-value, poor
support for ad-hoc
analysis
•  Inability to deliver high-
value packaged analytic
solutions
Solution
ERP
HCM SCM Logs/Machine Data
4. Take Action
Salesforce Apps
3. Analyze
Wave
Firewall
• Greater ease-of-use,
consistent end-to-end
experience
• Greater flexibility and
faster time-to-value
• Packaged Analytic
Solutions
2. Prepare Data
Data Pipelines / Async Query
1. Acquire & Store Data
BigObjects
Data Pipelines Overview
Currently in Pilot
Data Pipelines
Programmatic language based on
Apache Pig plus whitelisted UDF
libraries (Piggybank, DataFu)
Multi-tenancy resource management,
scheduling, job monitoring and
management
Data Sources Data Targets
SObjects
BigObjects
Wave Data Sets
External Objects
Files
Archive Objects
SObjects
BigObjects
Wave Data Sets
External Objects
Files
Archive Objects
Generate mapReduce Jobs
Hadoop
Big Data Processing
Architecture
Salesforce Data Center
Data Pipeline
BigObjects vs. SObjects
SObjects BigObjects
Use cases CRM transactional data Read-only immutable data
Data volumes <50m Rows Billions of Rows
Field types All Types Strings, numbers, dates, json
Query Real Time Query Response Blend of real time and asynchronous query response
determine by size of result set
Transactions ACID transactions Record Level Consistency
Access Management Full Sharing User Permissions and Field-level Security
APIs Full Support SOQL, Async Query, Data Pipelines
Triggers Full Support None
Reports Full Support Limited CRTs
Search Full Support None
DEMO
Transformations
●  JOIN
●  FILTER
●  UNION
●  MERGE
●  GROUP
●  DISTINCT
●  ORDER BY
●  RANK
●  LIMIT
●  … and many more
Key Use Cases
Big Object
Ext Object
Files
sObject
Wave
sObject
sObject
Native Big Data Processing
Data Prep for Descriptive
Analytics
Data Enrichment to turn “Insight into
Actions”
Big Object
Ext Object
Files
sObject
Wave
sObject
Handling Semi-structured Data
JSON, HTML, XML and
other complex semi-
structured data...
Customer Stories
Gamification - based on
experience points update user
levels
Computing Partner
Scorecards
Asset Management Analytics Analytics
Large volume data processing
(250M + records). Trawl the
rewards and update user-
objects. Later, will like to use
analytics.
Scorecard determines
status which in turn
determines pricing,
resources that partners
have access to assist in
sales.
Calculated multiple times
every week for Partner
Accounts (70h+).
Account assignment at account/
office/contact levels.
Will like to run daily
Correlate game-play data
with customer interaction
to improve customer
retention, loyalty, etc.
Multi-org consolidation;
White-space analysis.
Future
Roadmap Themes
1.  Resource Management/Fair Allocation
2.  Predictive Analytics
3.  Business Analyst/Salesforce Admin Interface
Summary & Next-Steps
Why Data Pipeline?
●  Massive Parallelism (10-40X performance improvement)
●  Overcome governor limits
●  Work towards Data Lake Architecture
●  Reduce complexity/cost - 100% Platform-Native
Resources
●  Implementation Guide - http://docs.releasenotes.salesforce.com/en-us/summer15/release-notes/
rn_forcecom_data_pipelines.htm
Join the Pilot Program
Any questions: nchawla@salesforce.com
Salesforce.com
Confidential
And make any adjustments needed before loading.
FUTURE
BigObjects
External
SObjects
•  New object type optimized for extremely large row-count
•  Use cases: read-only data from external systems, point-of-sale
data, connected product event data, clickstream data, etc.
•  Backed by HBase as a System of Record
•  Integrated into platform via External sObject framework,
Phoenix, Pliny
HBase
Phoenix
SQL
Pliny
SOQL
Platform
Data Pipelines Overview
Data Pipelines
Programmatic language based on
Apache Pig plus whitelisted UDF
libraries (Piggybank, DataFu)
Declarative tooling
for admins and
analysts
WaveDevConsoleSetup
Multi-tenancy Hadoop, resource
management, scheduling, job monitoring
and management
Data Sources
Data
Targets
SObjects
BigObjects
Wave Data Sets
External Objects
Files
Archive Objects
SObjects
BigObjects
Wave Data Sets
External Objects
Files
Archive Objects
Data Set
Objects
Snapshot for
provenance
tracking
Generate Data Pipelines
Generate mapReduce Jobs
Data Processing
Data Set
Objects
Snapshot for
provenance
tracking
Remove Data Sets Object
Declarative Tooling - bring it
later
Customer Name Brief Description Use-cases
Cloud App CloudApps increases
organisational performance by
enabling, encouraging,
enhancing and measuring
behavioural change using
gamification
Large volume data processing (250M + records). Trawl the rewards and update user-objects. Later,
will like to use analytics.
EMC Computing Partner Scorecards Business Partner scorecards help partners track whether they qualify for a particular Partner Tier
status (Gold, Silver, Platinum). Tier status determines pricing, resources that partners have access
to assist in sales. Scorecards are calculated multiple times every week for Partner Accounts. This
takes 70h+ to calculate. When being processed Scorecards are zero'ed out and a Partner cannot
not see the details of why they are in a certain status. In order to process them in a shorter window
(~10h), they've reduced the total number of Partner Accounts that qualify for the Business Partner
program from 22K to 780.
Legg Mason Asset Management Legg Mason has built an internal process to updates account assignment at account/office/contact
levels. They will like to do this more frequently, but async batch apex process is causing them to hit
several limits and preventing them to run this process daily.
Activision Video Game Developer Activision want’s to correlate game-play data with customer interaction to improve customer
retention, loyalty, etc. Currently, they load game-play data every 2 weeks, they will like to do that
daily. Plus, use Pipeline to join game play data with Case records and use Analytics to drive insight
(for example, impact of service issue on gaming behaviour)
Financial Force ERP on Platform FF gets files in emails and they have to do manual downstream processing to generate invoices, etc
based on this incoming files. They want to leverage Pipelines to scale and automate some steps
USPS Business Transformation USPS wants to combine CRM data with external data (from Equifax) to marry physical address with
digital identity for a user. They expect 500 million external records. And they will build
transformational applications based on this data (For ex, twitter handle on envelopes, Uber for
Customer Stories
Data Pipelines Roadmap (WORK ON THIS SLIDE)
-  Spark for internal customers
-  Wave connectors
-  Better error handling
-  Monitoring improvements
-  Basic limits
198
Winter ’16 / DF15
-  Resource management
-  Scheduler
-  Performance / optimization
-  Hardening
200
Spring 16
-  Metadata API
-  Simple Monitoring
-  Dev Console integration
-  Logging improvements
-  Deployment to HBase servers
196
Summer ‘15
Pilot II Pilot III
GA
(stretch goal)
Salesforce.com
Confidential
External
SObjects
BigObjects
•  New object type optimized for extremely large row-count
•  Targeted functionality
•  Use cases: read-only data from external systems, point-of-sale
data, connected product event data, clickstream data, etc.
•  Backed by HBase as a System of Record
•  Integrated into platform via External sObject framework,
Phoenix, Pliny
HBase
Phoenix
SQL
Pliny
SOQL
Platform
18
6
2.4
Salesforce.com
Confidential
BigObjects vs. SObjects
SObjects BigObjects
Use cases CRM transactional data Write-once / Read-only data from external systems, point-of-
sale data, connected product event data, clickstream data,
etc
Data volumes <50m Rows Billions of Rows
Filed types All Types Strings, numbers, dates
Query Realtime query response Blend of real time and asynchronous query response
determine by size of result set
Transactions ACID transactions Eventually consistent
Access Management Full Sharing Object Perm Based, Sharing Descriptors in future
APIs Full Support REST, SOQL, Bulk
Triggers Full Support None
Reports Full Support Limited CRTs
Search Full Support None

Weitere ähnliche Inhalte

Was ist angesagt?

Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...
Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...
Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...Swapna Tammishetty
 
Bringing the Power of Big Data Computation to Salesforce
Bringing the Power of Big Data Computation to SalesforceBringing the Power of Big Data Computation to Salesforce
Bringing the Power of Big Data Computation to SalesforceSalesforce Developers
 
Lane Summers - Resume 2015-01-22
Lane Summers - Resume 2015-01-22Lane Summers - Resume 2015-01-22
Lane Summers - Resume 2015-01-22Lane Summers
 
Ha100 notes units 1 and 2 sp08
Ha100 notes units 1 and 2   sp08Ha100 notes units 1 and 2   sp08
Ha100 notes units 1 and 2 sp08Duskydope Rao
 
Customer-Centric Data Management for Better Customer Experiences
Customer-Centric Data Management for Better Customer ExperiencesCustomer-Centric Data Management for Better Customer Experiences
Customer-Centric Data Management for Better Customer ExperiencesInformatica
 
20100430 introduction to business objects data services
20100430 introduction to business objects data services20100430 introduction to business objects data services
20100430 introduction to business objects data servicesJunhyun Song
 
Ha100 unit 3 hana architecture sp08
Ha100 unit 3 hana architecture sp08Ha100 unit 3 hana architecture sp08
Ha100 unit 3 hana architecture sp08Duskydope Rao
 
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & AnalyticsMDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & AnalyticsMDS ap
 
Informatica Solution for SWIFT Integration
Informatica Solution for SWIFT IntegrationInformatica Solution for SWIFT Integration
Informatica Solution for SWIFT IntegrationKim Loughead
 
Benefits of a data warehouse presentation by Being topper
Benefits of a data warehouse presentation by Being topperBenefits of a data warehouse presentation by Being topper
Benefits of a data warehouse presentation by Being topperBeing Topper
 
SAP HANA - Big Data and Fast Data
SAP HANA - Big Data and Fast DataSAP HANA - Big Data and Fast Data
SAP HANA - Big Data and Fast DataVitaliy Rudnytskiy
 
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron ApplianceIntegrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron ApplianceSandeep Chellingi
 
Adeel Khan Resume
Adeel Khan ResumeAdeel Khan Resume
Adeel Khan ResumeAdeel Khan
 
Data Wearhouse (Dw) concepts
Data Wearhouse (Dw)  conceptsData Wearhouse (Dw)  concepts
Data Wearhouse (Dw) conceptsBeing Topper
 
Data integration ppt-bhawani nandan prasad - iim calcutta
Data integration ppt-bhawani nandan prasad - iim calcuttaData integration ppt-bhawani nandan prasad - iim calcutta
Data integration ppt-bhawani nandan prasad - iim calcuttaBhawani N Prasad
 
Redefining Data Analytics Through Search
Redefining Data Analytics Through SearchRedefining Data Analytics Through Search
Redefining Data Analytics Through SearchConnexica
 
MapR Data Hub White Paper V2 2014
MapR Data Hub White Paper V2 2014MapR Data Hub White Paper V2 2014
MapR Data Hub White Paper V2 2014Erni Susanti
 
Austin fraser sap hana presentation
Austin fraser sap hana presentationAustin fraser sap hana presentation
Austin fraser sap hana presentationShane Sale
 
MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...
MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...
MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...MongoDB
 

Was ist angesagt? (20)

Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...
Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...
Swapna Tammishetty CV-Business & Systems Analyst-Data Analyst-Crystal Reports...
 
Bringing the Power of Big Data Computation to Salesforce
Bringing the Power of Big Data Computation to SalesforceBringing the Power of Big Data Computation to Salesforce
Bringing the Power of Big Data Computation to Salesforce
 
Lane Summers - Resume 2015-01-22
Lane Summers - Resume 2015-01-22Lane Summers - Resume 2015-01-22
Lane Summers - Resume 2015-01-22
 
Ha100 notes units 1 and 2 sp08
Ha100 notes units 1 and 2   sp08Ha100 notes units 1 and 2   sp08
Ha100 notes units 1 and 2 sp08
 
Customer-Centric Data Management for Better Customer Experiences
Customer-Centric Data Management for Better Customer ExperiencesCustomer-Centric Data Management for Better Customer Experiences
Customer-Centric Data Management for Better Customer Experiences
 
20100430 introduction to business objects data services
20100430 introduction to business objects data services20100430 introduction to business objects data services
20100430 introduction to business objects data services
 
Ha100 unit 3 hana architecture sp08
Ha100 unit 3 hana architecture sp08Ha100 unit 3 hana architecture sp08
Ha100 unit 3 hana architecture sp08
 
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & AnalyticsMDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
 
Informatica Solution for SWIFT Integration
Informatica Solution for SWIFT IntegrationInformatica Solution for SWIFT Integration
Informatica Solution for SWIFT Integration
 
Benefits of a data warehouse presentation by Being topper
Benefits of a data warehouse presentation by Being topperBenefits of a data warehouse presentation by Being topper
Benefits of a data warehouse presentation by Being topper
 
SAP HANA - Big Data and Fast Data
SAP HANA - Big Data and Fast DataSAP HANA - Big Data and Fast Data
SAP HANA - Big Data and Fast Data
 
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron ApplianceIntegrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
Integrating SFDC and Oracle ERP with IBM Websphere CastIron Appliance
 
Adeel Khan Resume
Adeel Khan ResumeAdeel Khan Resume
Adeel Khan Resume
 
Data Wearhouse (Dw) concepts
Data Wearhouse (Dw)  conceptsData Wearhouse (Dw)  concepts
Data Wearhouse (Dw) concepts
 
Data integration ppt-bhawani nandan prasad - iim calcutta
Data integration ppt-bhawani nandan prasad - iim calcuttaData integration ppt-bhawani nandan prasad - iim calcutta
Data integration ppt-bhawani nandan prasad - iim calcutta
 
Redefining Data Analytics Through Search
Redefining Data Analytics Through SearchRedefining Data Analytics Through Search
Redefining Data Analytics Through Search
 
MapR Data Hub White Paper V2 2014
MapR Data Hub White Paper V2 2014MapR Data Hub White Paper V2 2014
MapR Data Hub White Paper V2 2014
 
Austin fraser sap hana presentation
Austin fraser sap hana presentationAustin fraser sap hana presentation
Austin fraser sap hana presentation
 
MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...
MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...
MongoDB World 2019: Managing a Heterogeneous Data Stack with Informatica and ...
 
Best practices and trends in people soft
Best practices and trends in people softBest practices and trends in people soft
Best practices and trends in people soft
 

Andere mochten auch

Andere mochten auch (8)

1.095.915.347pnnr[1]
1.095.915.347pnnr[1]1.095.915.347pnnr[1]
1.095.915.347pnnr[1]
 
Best French Restaurant
Best French RestaurantBest French Restaurant
Best French Restaurant
 
Listado con temas de nuestro interes
Listado con temas de nuestro interesListado con temas de nuestro interes
Listado con temas de nuestro interes
 
Certificado de resultados
Certificado de resultadosCertificado de resultados
Certificado de resultados
 
Derechos humanos resultado
Derechos humanos resultadoDerechos humanos resultado
Derechos humanos resultado
 
Encadrement des loyers LIlle
Encadrement des loyers LIlleEncadrement des loyers LIlle
Encadrement des loyers LIlle
 
Unified Big Data Processing with Apache Spark
Unified Big Data Processing with Apache SparkUnified Big Data Processing with Apache Spark
Unified Big Data Processing with Apache Spark
 
Acute Severe Asthma
Acute Severe AsthmaAcute Severe Asthma
Acute Severe Asthma
 

Ähnlich wie Processing Big Data At-Scale in the App Cloud

Unleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on SalesforceUnleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on SalesforceDreamforce
 
Introducing the Salesforce platform
Introducing the Salesforce platformIntroducing the Salesforce platform
Introducing the Salesforce platformJohn Stevenson
 
Analyze billions of records on Salesforce App Cloud with BigObject
Analyze billions of records on Salesforce App Cloud with BigObjectAnalyze billions of records on Salesforce App Cloud with BigObject
Analyze billions of records on Salesforce App Cloud with BigObjectSalesforce Developers
 
Toronto dev group mar2019
Toronto dev group mar2019Toronto dev group mar2019
Toronto dev group mar2019rikkehovgaard
 
Unlock SAP - Release the potential of your existing backend systems with Sale...
Unlock SAP - Release the potential of your existing backend systems with Sale...Unlock SAP - Release the potential of your existing backend systems with Sale...
Unlock SAP - Release the potential of your existing backend systems with Sale...Salesforce Deutschland
 
Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015Dreamforce
 
Heroku - developer playground
Heroku - developer playground Heroku - developer playground
Heroku - developer playground Troy Sellers
 
Understanding the Salesforce Architecture: How We Do the Magic We Do
Understanding the Salesforce Architecture: How We Do the Magic We DoUnderstanding the Salesforce Architecture: How We Do the Magic We Do
Understanding the Salesforce Architecture: How We Do the Magic We DoSalesforce Developers
 
Developer Tour on the Salesforce1 Platform
Developer Tour on the Salesforce1 PlatformDeveloper Tour on the Salesforce1 Platform
Developer Tour on the Salesforce1 PlatformSalesforce Deutschland
 
Developers guide to the Salesforce1 Platform
Developers guide to the Salesforce1 PlatformDevelopers guide to the Salesforce1 Platform
Developers guide to the Salesforce1 PlatformJohn Stevenson
 
Integration Strategies in a SaaS Environment
Integration Strategies in a SaaS EnvironmentIntegration Strategies in a SaaS Environment
Integration Strategies in a SaaS Environmentdreamforce2006
 
Loading Data into the Analytics Cloud with Apex
Loading Data into the Analytics Cloud with ApexLoading Data into the Analytics Cloud with Apex
Loading Data into the Analytics Cloud with ApexSalesforce Developers
 
Understanding Multitenancy and the Architecture of the Salesforce Platform
Understanding Multitenancy and the Architecture of the Salesforce PlatformUnderstanding Multitenancy and the Architecture of the Salesforce Platform
Understanding Multitenancy and the Architecture of the Salesforce PlatformSalesforce Developers
 
salesforce and AI, bigdata
salesforce and AI, bigdatasalesforce and AI, bigdata
salesforce and AI, bigdataShen yifeng
 
Salesforce Multitenant Architecture: How We Do the Magic We Do
Salesforce Multitenant Architecture: How We Do the Magic We DoSalesforce Multitenant Architecture: How We Do the Magic We Do
Salesforce Multitenant Architecture: How We Do the Magic We DoSalesforce Developers
 
Best Practices for Integrating with Your ERP
Best Practices for Integrating with Your ERPBest Practices for Integrating with Your ERP
Best Practices for Integrating with Your ERPdreamforce2006
 
The Mystery Is Solved Demystifying Integrations
The Mystery Is Solved Demystifying IntegrationsThe Mystery Is Solved Demystifying Integrations
The Mystery Is Solved Demystifying Integrationsdreamforce2006
 
Boxcars and Cabooses: When One More XHR Is Too Much
Boxcars and Cabooses: When One More XHR Is Too MuchBoxcars and Cabooses: When One More XHR Is Too Much
Boxcars and Cabooses: When One More XHR Is Too MuchPeter Chittum
 
February 2020 Salesforce API Review
February 2020 Salesforce API ReviewFebruary 2020 Salesforce API Review
February 2020 Salesforce API ReviewLydon Bergin
 

Ähnlich wie Processing Big Data At-Scale in the App Cloud (20)

Unleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on SalesforceUnleash the Potential of Big Data on Salesforce
Unleash the Potential of Big Data on Salesforce
 
Introducing the Salesforce platform
Introducing the Salesforce platformIntroducing the Salesforce platform
Introducing the Salesforce platform
 
Analyze billions of records on Salesforce App Cloud with BigObject
Analyze billions of records on Salesforce App Cloud with BigObjectAnalyze billions of records on Salesforce App Cloud with BigObject
Analyze billions of records on Salesforce App Cloud with BigObject
 
Toronto dev group mar2019
Toronto dev group mar2019Toronto dev group mar2019
Toronto dev group mar2019
 
Unlock SAP - Release the potential of your existing backend systems with Sale...
Unlock SAP - Release the potential of your existing backend systems with Sale...Unlock SAP - Release the potential of your existing backend systems with Sale...
Unlock SAP - Release the potential of your existing backend systems with Sale...
 
Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015Lightning connect sap_integration_df2015
Lightning connect sap_integration_df2015
 
Heroku - developer playground
Heroku - developer playground Heroku - developer playground
Heroku - developer playground
 
Salesforce platform session 2
 Salesforce platform session 2 Salesforce platform session 2
Salesforce platform session 2
 
Understanding the Salesforce Architecture: How We Do the Magic We Do
Understanding the Salesforce Architecture: How We Do the Magic We DoUnderstanding the Salesforce Architecture: How We Do the Magic We Do
Understanding the Salesforce Architecture: How We Do the Magic We Do
 
Developer Tour on the Salesforce1 Platform
Developer Tour on the Salesforce1 PlatformDeveloper Tour on the Salesforce1 Platform
Developer Tour on the Salesforce1 Platform
 
Developers guide to the Salesforce1 Platform
Developers guide to the Salesforce1 PlatformDevelopers guide to the Salesforce1 Platform
Developers guide to the Salesforce1 Platform
 
Integration Strategies in a SaaS Environment
Integration Strategies in a SaaS EnvironmentIntegration Strategies in a SaaS Environment
Integration Strategies in a SaaS Environment
 
Loading Data into the Analytics Cloud with Apex
Loading Data into the Analytics Cloud with ApexLoading Data into the Analytics Cloud with Apex
Loading Data into the Analytics Cloud with Apex
 
Understanding Multitenancy and the Architecture of the Salesforce Platform
Understanding Multitenancy and the Architecture of the Salesforce PlatformUnderstanding Multitenancy and the Architecture of the Salesforce Platform
Understanding Multitenancy and the Architecture of the Salesforce Platform
 
salesforce and AI, bigdata
salesforce and AI, bigdatasalesforce and AI, bigdata
salesforce and AI, bigdata
 
Salesforce Multitenant Architecture: How We Do the Magic We Do
Salesforce Multitenant Architecture: How We Do the Magic We DoSalesforce Multitenant Architecture: How We Do the Magic We Do
Salesforce Multitenant Architecture: How We Do the Magic We Do
 
Best Practices for Integrating with Your ERP
Best Practices for Integrating with Your ERPBest Practices for Integrating with Your ERP
Best Practices for Integrating with Your ERP
 
The Mystery Is Solved Demystifying Integrations
The Mystery Is Solved Demystifying IntegrationsThe Mystery Is Solved Demystifying Integrations
The Mystery Is Solved Demystifying Integrations
 
Boxcars and Cabooses: When One More XHR Is Too Much
Boxcars and Cabooses: When One More XHR Is Too MuchBoxcars and Cabooses: When One More XHR Is Too Much
Boxcars and Cabooses: When One More XHR Is Too Much
 
February 2020 Salesforce API Review
February 2020 Salesforce API ReviewFebruary 2020 Salesforce API Review
February 2020 Salesforce API Review
 

Mehr von Salesforce Developers

Sample Gallery: Reference Code and Best Practices for Salesforce Developers
Sample Gallery: Reference Code and Best Practices for Salesforce DevelopersSample Gallery: Reference Code and Best Practices for Salesforce Developers
Sample Gallery: Reference Code and Best Practices for Salesforce DevelopersSalesforce Developers
 
Maximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component PerformanceMaximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component PerformanceSalesforce Developers
 
Local development with Open Source Base Components
Local development with Open Source Base ComponentsLocal development with Open Source Base Components
Local development with Open Source Base ComponentsSalesforce Developers
 
TrailheaDX India : Developer Highlights
TrailheaDX India : Developer HighlightsTrailheaDX India : Developer Highlights
TrailheaDX India : Developer HighlightsSalesforce Developers
 
Why developers shouldn’t miss TrailheaDX India
Why developers shouldn’t miss TrailheaDX IndiaWhy developers shouldn’t miss TrailheaDX India
Why developers shouldn’t miss TrailheaDX IndiaSalesforce Developers
 
CodeLive: Build Lightning Web Components faster with Local Development
CodeLive: Build Lightning Web Components faster with Local DevelopmentCodeLive: Build Lightning Web Components faster with Local Development
CodeLive: Build Lightning Web Components faster with Local DevelopmentSalesforce Developers
 
CodeLive: Converting Aura Components to Lightning Web Components
CodeLive: Converting Aura Components to Lightning Web ComponentsCodeLive: Converting Aura Components to Lightning Web Components
CodeLive: Converting Aura Components to Lightning Web ComponentsSalesforce Developers
 
Enterprise-grade UI with open source Lightning Web Components
Enterprise-grade UI with open source Lightning Web ComponentsEnterprise-grade UI with open source Lightning Web Components
Enterprise-grade UI with open source Lightning Web ComponentsSalesforce Developers
 
TrailheaDX and Summer '19: Developer Highlights
TrailheaDX and Summer '19: Developer HighlightsTrailheaDX and Summer '19: Developer Highlights
TrailheaDX and Summer '19: Developer HighlightsSalesforce Developers
 
Lightning web components - Episode 4 : Security and Testing
Lightning web components  - Episode 4 : Security and TestingLightning web components  - Episode 4 : Security and Testing
Lightning web components - Episode 4 : Security and TestingSalesforce Developers
 
LWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura InteroperabilityLWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura InteroperabilitySalesforce Developers
 
Lightning web components episode 2- work with salesforce data
Lightning web components   episode 2- work with salesforce dataLightning web components   episode 2- work with salesforce data
Lightning web components episode 2- work with salesforce dataSalesforce Developers
 
Lightning web components - Episode 1 - An Introduction
Lightning web components - Episode 1 - An IntroductionLightning web components - Episode 1 - An Introduction
Lightning web components - Episode 1 - An IntroductionSalesforce Developers
 
Migrating CPQ to Advanced Calculator and JSQCP
Migrating CPQ to Advanced Calculator and JSQCPMigrating CPQ to Advanced Calculator and JSQCP
Migrating CPQ to Advanced Calculator and JSQCPSalesforce Developers
 
Scale with Large Data Volumes and Big Objects in Salesforce
Scale with Large Data Volumes and Big Objects in SalesforceScale with Large Data Volumes and Big Objects in Salesforce
Scale with Large Data Volumes and Big Objects in SalesforceSalesforce Developers
 
Replicate Salesforce Data in Real Time with Change Data Capture
Replicate Salesforce Data in Real Time with Change Data CaptureReplicate Salesforce Data in Real Time with Change Data Capture
Replicate Salesforce Data in Real Time with Change Data CaptureSalesforce Developers
 
Modern Development with Salesforce DX
Modern Development with Salesforce DXModern Development with Salesforce DX
Modern Development with Salesforce DXSalesforce Developers
 
Integrate CMS Content Into Lightning Communities with CMS Connect
Integrate CMS Content Into Lightning Communities with CMS ConnectIntegrate CMS Content Into Lightning Communities with CMS Connect
Integrate CMS Content Into Lightning Communities with CMS ConnectSalesforce Developers
 

Mehr von Salesforce Developers (20)

Sample Gallery: Reference Code and Best Practices for Salesforce Developers
Sample Gallery: Reference Code and Best Practices for Salesforce DevelopersSample Gallery: Reference Code and Best Practices for Salesforce Developers
Sample Gallery: Reference Code and Best Practices for Salesforce Developers
 
Maximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component PerformanceMaximizing Salesforce Lightning Experience and Lightning Component Performance
Maximizing Salesforce Lightning Experience and Lightning Component Performance
 
Local development with Open Source Base Components
Local development with Open Source Base ComponentsLocal development with Open Source Base Components
Local development with Open Source Base Components
 
TrailheaDX India : Developer Highlights
TrailheaDX India : Developer HighlightsTrailheaDX India : Developer Highlights
TrailheaDX India : Developer Highlights
 
Why developers shouldn’t miss TrailheaDX India
Why developers shouldn’t miss TrailheaDX IndiaWhy developers shouldn’t miss TrailheaDX India
Why developers shouldn’t miss TrailheaDX India
 
CodeLive: Build Lightning Web Components faster with Local Development
CodeLive: Build Lightning Web Components faster with Local DevelopmentCodeLive: Build Lightning Web Components faster with Local Development
CodeLive: Build Lightning Web Components faster with Local Development
 
CodeLive: Converting Aura Components to Lightning Web Components
CodeLive: Converting Aura Components to Lightning Web ComponentsCodeLive: Converting Aura Components to Lightning Web Components
CodeLive: Converting Aura Components to Lightning Web Components
 
Enterprise-grade UI with open source Lightning Web Components
Enterprise-grade UI with open source Lightning Web ComponentsEnterprise-grade UI with open source Lightning Web Components
Enterprise-grade UI with open source Lightning Web Components
 
TrailheaDX and Summer '19: Developer Highlights
TrailheaDX and Summer '19: Developer HighlightsTrailheaDX and Summer '19: Developer Highlights
TrailheaDX and Summer '19: Developer Highlights
 
Live coding with LWC
Live coding with LWCLive coding with LWC
Live coding with LWC
 
Lightning web components - Episode 4 : Security and Testing
Lightning web components  - Episode 4 : Security and TestingLightning web components  - Episode 4 : Security and Testing
Lightning web components - Episode 4 : Security and Testing
 
LWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura InteroperabilityLWC Episode 3- Component Communication and Aura Interoperability
LWC Episode 3- Component Communication and Aura Interoperability
 
Lightning web components episode 2- work with salesforce data
Lightning web components   episode 2- work with salesforce dataLightning web components   episode 2- work with salesforce data
Lightning web components episode 2- work with salesforce data
 
Lightning web components - Episode 1 - An Introduction
Lightning web components - Episode 1 - An IntroductionLightning web components - Episode 1 - An Introduction
Lightning web components - Episode 1 - An Introduction
 
Migrating CPQ to Advanced Calculator and JSQCP
Migrating CPQ to Advanced Calculator and JSQCPMigrating CPQ to Advanced Calculator and JSQCP
Migrating CPQ to Advanced Calculator and JSQCP
 
Scale with Large Data Volumes and Big Objects in Salesforce
Scale with Large Data Volumes and Big Objects in SalesforceScale with Large Data Volumes and Big Objects in Salesforce
Scale with Large Data Volumes and Big Objects in Salesforce
 
Replicate Salesforce Data in Real Time with Change Data Capture
Replicate Salesforce Data in Real Time with Change Data CaptureReplicate Salesforce Data in Real Time with Change Data Capture
Replicate Salesforce Data in Real Time with Change Data Capture
 
Modern Development with Salesforce DX
Modern Development with Salesforce DXModern Development with Salesforce DX
Modern Development with Salesforce DX
 
Get Into Lightning Flow Development
Get Into Lightning Flow DevelopmentGet Into Lightning Flow Development
Get Into Lightning Flow Development
 
Integrate CMS Content Into Lightning Communities with CMS Connect
Integrate CMS Content Into Lightning Communities with CMS ConnectIntegrate CMS Content Into Lightning Communities with CMS Connect
Integrate CMS Content Into Lightning Communities with CMS Connect
 

Kürzlich hochgeladen

H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Scott Keck-Warren
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfAlex Barbosa Coqueiro
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfAddepto
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.Curtis Poe
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsMiki Katsuragi
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteDianaGray10
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 

Kürzlich hochgeladen (20)

H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024Advanced Test Driven-Development @ php[tek] 2024
Advanced Test Driven-Development @ php[tek] 2024
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
Unraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdfUnraveling Multimodality with Large Language Models.pdf
Unraveling Multimodality with Large Language Models.pdf
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
Gen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdfGen AI in Business - Global Trends Report 2024.pdf
Gen AI in Business - Global Trends Report 2024.pdf
 
"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.How AI, OpenAI, and ChatGPT impact business and software.
How AI, OpenAI, and ChatGPT impact business and software.
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data PrivacyTrustArc Webinar - How to Build Consumer Trust Through Data Privacy
TrustArc Webinar - How to Build Consumer Trust Through Data Privacy
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Vertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering TipsVertex AI Gemini Prompt Engineering Tips
Vertex AI Gemini Prompt Engineering Tips
 
Take control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test SuiteTake control of your SAP testing with UiPath Test Suite
Take control of your SAP testing with UiPath Test Suite
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 

Processing Big Data At-Scale in the App Cloud

  • 1. Processing Big Data At Scale Naren Chawla Senior Director, Product Management (nchawla@salesforce.com) Prashant Kommireddi @prashant1784 Leverage platform-native Data Pipelines for ETL
  • 2. Safe Harbor Safe harbor statement under the Private Securities Litigation Reform Act of 1995: This presentation may contain forward-looking statements that involve risks, uncertainties, and assumptions. If any such uncertainties materialize or if any of the assumptions proves incorrect, the results of salesforce.com, inc. could differ materially from the results expressed or implied by the forward- looking statements we make. All statements other than statements of historical fact could be deemed forward-looking, including any projections of product or service availability, subscriber growth, earnings, revenues, or other financial items and any statements regarding strategies or plans of management for future operations, statements of belief, any statements concerning new, planned, or upgraded services or technology developments and customer contracts or use of our services. The risks and uncertainties referred to above include – but are not limited to – risks associated with developing and delivering new functionality for our service, new products and services, our new business model, our past operating losses, possible fluctuations in our operating results and rate of growth, interruptions or delays in our Web hosting, breach of our security measures, the outcome of any litigation, risks associated with completed and any possible mergers and acquisitions, the immature market in which we operate, our relatively limited operating history, our ability to expand, retain, and motivate our employees and manage our growth, new releases of our service and successful customer deployment, our limited history reselling non- salesforce.com products, and utilization and selling to larger enterprise customers. Further information on potential factors that could affect the financial results of salesforce.com, inc. is included in our annual report on Form 10-K for the most recent fiscal year and in our quarterly report on Form 10-Q for the most recent fiscal quarter. These documents and others containing important disclosures are available on the SEC Filings section of the Investor Information section of our Web site. Any unreleased services or features referenced in this or other presentations, press releases or public statements are not currently available and may not be delivered on time or at all. Customers who purchase our services should make the purchase decisions based upon features that are currently available. Salesforce.com, inc. assumes no obligation and does not intend to update these forward-looking statements.
  • 3. Topics Big Data Processing Problem and Proposed Solution Data Pipeline Deep-dive Demo Key Use-cases Customer Stories Summary Q&A
  • 4. Problem ERP HCM SCM Logs 1. Acquire & Store Data 2. Prepare Data (Cleanse, Augment, Transform, Join) Data Lake / EDW 4. Take Action Customer Success Platform 3. Analyze Wave Firewall •  Cost and complexity of managing external data platforms •  Slow time-to-value, poor support for ad-hoc analysis •  Inability to deliver high- value packaged analytic solutions
  • 5. Solution ERP HCM SCM Logs/Machine Data 4. Take Action Salesforce Apps 3. Analyze Wave Firewall • Greater ease-of-use, consistent end-to-end experience • Greater flexibility and faster time-to-value • Packaged Analytic Solutions 2. Prepare Data Data Pipelines / Async Query 1. Acquire & Store Data BigObjects
  • 6. Data Pipelines Overview Currently in Pilot Data Pipelines Programmatic language based on Apache Pig plus whitelisted UDF libraries (Piggybank, DataFu) Multi-tenancy resource management, scheduling, job monitoring and management Data Sources Data Targets SObjects BigObjects Wave Data Sets External Objects Files Archive Objects SObjects BigObjects Wave Data Sets External Objects Files Archive Objects Generate mapReduce Jobs Hadoop Big Data Processing
  • 8.
  • 9. BigObjects vs. SObjects SObjects BigObjects Use cases CRM transactional data Read-only immutable data Data volumes <50m Rows Billions of Rows Field types All Types Strings, numbers, dates, json Query Real Time Query Response Blend of real time and asynchronous query response determine by size of result set Transactions ACID transactions Record Level Consistency Access Management Full Sharing User Permissions and Field-level Security APIs Full Support SOQL, Async Query, Data Pipelines Triggers Full Support None Reports Full Support Limited CRTs Search Full Support None
  • 10. DEMO
  • 11. Transformations ●  JOIN ●  FILTER ●  UNION ●  MERGE ●  GROUP ●  DISTINCT ●  ORDER BY ●  RANK ●  LIMIT ●  … and many more
  • 12. Key Use Cases Big Object Ext Object Files sObject Wave sObject sObject Native Big Data Processing Data Prep for Descriptive Analytics Data Enrichment to turn “Insight into Actions” Big Object Ext Object Files sObject Wave sObject Handling Semi-structured Data JSON, HTML, XML and other complex semi- structured data...
  • 13. Customer Stories Gamification - based on experience points update user levels Computing Partner Scorecards Asset Management Analytics Analytics Large volume data processing (250M + records). Trawl the rewards and update user- objects. Later, will like to use analytics. Scorecard determines status which in turn determines pricing, resources that partners have access to assist in sales. Calculated multiple times every week for Partner Accounts (70h+). Account assignment at account/ office/contact levels. Will like to run daily Correlate game-play data with customer interaction to improve customer retention, loyalty, etc. Multi-org consolidation; White-space analysis.
  • 15. Roadmap Themes 1.  Resource Management/Fair Allocation 2.  Predictive Analytics 3.  Business Analyst/Salesforce Admin Interface
  • 16.
  • 17.
  • 18. Summary & Next-Steps Why Data Pipeline? ●  Massive Parallelism (10-40X performance improvement) ●  Overcome governor limits ●  Work towards Data Lake Architecture ●  Reduce complexity/cost - 100% Platform-Native Resources ●  Implementation Guide - http://docs.releasenotes.salesforce.com/en-us/summer15/release-notes/ rn_forcecom_data_pipelines.htm Join the Pilot Program Any questions: nchawla@salesforce.com
  • 20. And make any adjustments needed before loading. FUTURE
  • 21. BigObjects External SObjects •  New object type optimized for extremely large row-count •  Use cases: read-only data from external systems, point-of-sale data, connected product event data, clickstream data, etc. •  Backed by HBase as a System of Record •  Integrated into platform via External sObject framework, Phoenix, Pliny HBase Phoenix SQL Pliny SOQL Platform
  • 22. Data Pipelines Overview Data Pipelines Programmatic language based on Apache Pig plus whitelisted UDF libraries (Piggybank, DataFu) Declarative tooling for admins and analysts WaveDevConsoleSetup Multi-tenancy Hadoop, resource management, scheduling, job monitoring and management Data Sources Data Targets SObjects BigObjects Wave Data Sets External Objects Files Archive Objects SObjects BigObjects Wave Data Sets External Objects Files Archive Objects Data Set Objects Snapshot for provenance tracking Generate Data Pipelines Generate mapReduce Jobs Data Processing Data Set Objects Snapshot for provenance tracking Remove Data Sets Object Declarative Tooling - bring it later
  • 23. Customer Name Brief Description Use-cases Cloud App CloudApps increases organisational performance by enabling, encouraging, enhancing and measuring behavioural change using gamification Large volume data processing (250M + records). Trawl the rewards and update user-objects. Later, will like to use analytics. EMC Computing Partner Scorecards Business Partner scorecards help partners track whether they qualify for a particular Partner Tier status (Gold, Silver, Platinum). Tier status determines pricing, resources that partners have access to assist in sales. Scorecards are calculated multiple times every week for Partner Accounts. This takes 70h+ to calculate. When being processed Scorecards are zero'ed out and a Partner cannot not see the details of why they are in a certain status. In order to process them in a shorter window (~10h), they've reduced the total number of Partner Accounts that qualify for the Business Partner program from 22K to 780. Legg Mason Asset Management Legg Mason has built an internal process to updates account assignment at account/office/contact levels. They will like to do this more frequently, but async batch apex process is causing them to hit several limits and preventing them to run this process daily. Activision Video Game Developer Activision want’s to correlate game-play data with customer interaction to improve customer retention, loyalty, etc. Currently, they load game-play data every 2 weeks, they will like to do that daily. Plus, use Pipeline to join game play data with Case records and use Analytics to drive insight (for example, impact of service issue on gaming behaviour) Financial Force ERP on Platform FF gets files in emails and they have to do manual downstream processing to generate invoices, etc based on this incoming files. They want to leverage Pipelines to scale and automate some steps USPS Business Transformation USPS wants to combine CRM data with external data (from Equifax) to marry physical address with digital identity for a user. They expect 500 million external records. And they will build transformational applications based on this data (For ex, twitter handle on envelopes, Uber for Customer Stories
  • 24. Data Pipelines Roadmap (WORK ON THIS SLIDE) -  Spark for internal customers -  Wave connectors -  Better error handling -  Monitoring improvements -  Basic limits 198 Winter ’16 / DF15 -  Resource management -  Scheduler -  Performance / optimization -  Hardening 200 Spring 16 -  Metadata API -  Simple Monitoring -  Dev Console integration -  Logging improvements -  Deployment to HBase servers 196 Summer ‘15 Pilot II Pilot III GA (stretch goal)
  • 25. Salesforce.com Confidential External SObjects BigObjects •  New object type optimized for extremely large row-count •  Targeted functionality •  Use cases: read-only data from external systems, point-of-sale data, connected product event data, clickstream data, etc. •  Backed by HBase as a System of Record •  Integrated into platform via External sObject framework, Phoenix, Pliny HBase Phoenix SQL Pliny SOQL Platform 18 6 2.4
  • 26. Salesforce.com Confidential BigObjects vs. SObjects SObjects BigObjects Use cases CRM transactional data Write-once / Read-only data from external systems, point-of- sale data, connected product event data, clickstream data, etc Data volumes <50m Rows Billions of Rows Filed types All Types Strings, numbers, dates Query Realtime query response Blend of real time and asynchronous query response determine by size of result set Transactions ACID transactions Eventually consistent Access Management Full Sharing Object Perm Based, Sharing Descriptors in future APIs Full Support REST, SOQL, Bulk Triggers Full Support None Reports Full Support Limited CRTs Search Full Support None