When it comes to Analytics and Reporting , There is a fine line between HindSight to Insight to Foresight . With the evolution of BigData technology, there is a need in deriving value out of the larger datasets, not available in the past. Even before we can start using the new shiny technologies, there is a need of understanding what is categorized as reporting or business intelligence or Big Data and Analytics. Based on my experience, people struggle to distinguish between reporting, Analytics, and Business Intelligence.
1. Big Data : From HindSight To InSight To
ForeSight
-Delivering Data Driven Business Insights
Adopt
MarketInnovate
Sunil S Ranka
Director – Big Data and Advance Analytics
2. Key Topics
 About Jade
 About Me
 What is Big Data
 Meaning of HindSight to Insight to Foresight
 How to make progression
 Business Impact
 Real World Example
 Next Steps
4. Services High-Tech Manufacturing Energy Social Media & Entertainment
5 Global Delivery
Centers
8 Offices
Worldwide
Atlanta
Pune
Noida
San Jose
Los Angeles
London
Hyderabad
San Diego
Global Delivery Model Serving Many Industries
5. Strategic
Partnerships
Salesforce.com
Sales, Service, Marketing,
force.com
Testing
Tools/Frameworks
QC, QTP, Selenium, LoadRunner,
JIRA Bugzilla, JUnit, TestNG
Microsoft
Dynamics, SharePoint,
Office 365, Lync, BI
Custom Development
Java, .Net, J2EE, Product
Engineering, Open Source
Technologies
Integration
Oracle SOA, Tibco, Weblogic,
Oracle Cloud Platform, ICS, JCS,
Mulesoft, Dell Boomi
Infrastructure
Management
IBM AIX, HP-UX, RHEL,
OEL Linux, Windows Server
Cloud Financials, Projects, SCM,
HCM and EBS Financials,
Procurement, Value Chain, CRM,
Demantra, Agile, GRC
Oracle EBS Suite
ServiceNow
IT Service Automation Applications,
CreateNow Development Suite,
Orchestration, Discovery
Big Data & Analytics
Hadoop, KNIME, R, Tableau, Hadoop
9. About Me
• Venture Partner : Investing and Advisor with early stage startups focusing on Data.
• Director – Big Data and Advance Analytics
• Oracle ACE (Business Intelligence with Proficiency in Big Data)
• Extensively worked with fortune 500 leaders.
• Held positions of Head Of Product Development, Architect, etc.
• http://sranka.wordpress.com, sunil_ranka
• Featured Tech writer for IT Next Magazine.
• Speaking engagements at following conferences :
• COLLABORATE ( 2009, 2010 , 2011 ,2012, 2013,2015)
• BIWA SIG TechCast Series (2010 , 2011 , 2012, 2013,2014,2016),
• NorCal OAUG-2010 at Santa Clara Convention Center, CA
• Session speaker at NoCouG in San Francisco
• Oracle Open World ( 2009 , 2010 , 2012)
My Tag Line :: “Superior BI is the antidote to Business Failure”
11. Data is the new Oil. Data is just like crude. It’s
valuable, but if unrefined it cannot really be used.
– Clive Humby, DunnHumby
11
We have for the first time an economy based on
a key resource [Information] that is not only renewable,
but self-generating. Running out of it is not a problem,
but drowning in it is.
– John Naisbitt
12. Big Data and Analytics is Helping
Smarter Revenue
Management
Smarter Healtcare
Analytics
$16Billion
Reduced
Improper Payment
Smarter Crime
Prevention
Helps detect life
threatening conditions
up to 24 hours sooner
30%
Cut
serious crime
by
Tax Agency
* Courtesy - IBM
13. Big Data Definition
No single standard definition…
“Big Data” is data whose scale, diversity, and complexity require new
architecture, techniques, algorithms, and analytics to manage it and
extract value and hidden knowledge from it…
14. What is Big Data
Big data Represents new data features created by today’s Data Driven Organization for Decision
Making
volume
Variety
Velocity
Value
Data At Scale
Terabyte To Petabyte of Data
Data In Many Forms
Structured, unstructured, text, Media
Data In Motion
Analysis of stream data to make decision in real time
Data with Insight
Deriving valuable insight from the data
Characteristicsofbigdata
15. Harnessing Big Data
 OLTP: Online Transaction Processing (DBMSs)
 OLAP: Online Analytical Processing (Data Warehousing)
 RTAP: Real-Time Analytics Processing (Big Data Architecture & technology)
15
16. Who’s Generating Big Data
Social media and networks
(all of us are generating data)
Scientific instruments
(collecting all sorts of data)
Mobile devices
(tracking all objects all the time)
Sensor technology and networks
(measuring all kinds of data)
 The progress and innovation is no longer hindered by the ability to
collect data
 But, by the ability to manage, analyze, summarize, visualize, and discover
knowledge from the collected data in a timely manner and in a scalable
fashion
16
17. The Model Has Changed…
 The Model of Generating/Consuming Data has
Changed
Old Model: Few companies are generating data, all others are consuming data
New Model: all of us are generating data, and all of us are consuming data
17
19. Essentials for Analytics Strategy
Where are we ?
• What business decisions do we not have sufficient
information ?
Where do we want to be ?
• What is our vision for information accessibility and
usage?
• What should the high-level BI roadmap of initiatives look
like ?
What capabilities will get us there ?
• What capabilities are required to make information
available and useful ?
What are the dependencies ?
• What metrics should we use to manage the
implementation and fulfill BI business goals ?
• How should we design the processes, applications, and
organization to fulfill our BI vision?
• What toolsets should I use to fulfill our BI vision?
20. Definitions
Hindsight ( What happed ?)
Understanding of a situation or event only after it has happened
or developed – What lesson can be learnt ?
Insight ( Why it is happening?)
The capacity to gain an accurate and deep intuitive
understanding of a person or thing. – What matters now ?
Foresight ( What will happen ?)
The ability to predict or the action of predicting what will happen
or be needed in the future. –What matters next ?
21. Challenges in Handling Big Data
 The Bottleneck is in technology
 New architecture, algorithms, techniques are needed
 Also in technical skills
 Experts in using the new technology and dealing with
big data
21
22. What’s driving Big Data
- Ad-hoc querying and reporting
- Data mining techniques
- Structured data, typical sources
- Small to mid-size datasets
- Optimizations and predictive analytics
- Complex statistical analysis
- All types of data, and many sources
- Very large datasets
- More of a real-time
22
23. Value of Big Data Analytics
 Big data is more real-time in
nature than traditional DW
applications
 Traditional DW architectures (e.g.
Exadata, Teradata) are not well-
suited for big data apps
 Shared nothing, massively
parallel processing, scale out
architectures are well-suited for
big data apps
23
24. Analytics Maturity Pyramid
No Reporting
Struggling to get basic information
Reactive Analytics
Concerned with current Issues
What Happened ?
Diagnostic Analytics
Hindsight
Why it Happened ?
Predictive Analytics
Insight
What will Happened?
Prescriptive Analytics
Foresight
What should I do ?
Hindsight
insight
foresight
God Bless You!!
25. How it changes ?
Traditional Reporting Business Intelligence Analytics and Big Data
Push Pull Predictive
Fixed Format Interactive/Slice and Dice Interactive and business driven
Typically transacation oriented Applies to all business functions, “Front
Office” and “Back Office”
Applies to mostly front office CRM and
product development
Mostly internal data silos Still mostly internal and structure data,
but bringing together more data silos
Combines internal and significant
external data, often unstruture and
large data
Implemented post transactional system
implementations
Implemented post transactional system
but tighly coupled with transaction
system
Implemented as a business capability,
with dedicated analytics team.
Technology is not a differentiator More technology differentiator, but
leaning more on packaged solution
Many specialist and highly
differentiating cutting edge
technologies
“rear view mirror”(what happened ?) Still “rear view mirror”,but looking at
what happened and why ?
Forward Looking
27. Business Value
BI Analytics
Big Data
Analytics
BI Big Data and BI
Proactive
Reactive
Data Size
Analytics
Capability
• The lower left quadrant represents traditional business
• In the upper left quadrant, you have traditional analytic
processing technologies performing more complex
assessments.
• The lower right quadrant represents the use of big data
technologies to expedite hindsight reporting.
• The upper right quadrant is the sweet spot – big data
analytics – the combination of big data technologies with
predictive and hybrid analytics.
29. Big Data Needs Diversified Skill Sets
Math and
Operations Research
Expertise
Develop analytic algorithms
Visualization
Expertise
Interpret data sets,
determine correlations and
present in meaningful ways
Tool Developers
Mask complexity and
analytics to lower skills
boundaries
Industry Vertical
Domain Expertise
Develop hypothesis, identify
relevant business issues,
ask the right questions
Data Experts
Data architecture, management,
governance, policy
Decision Making
Executive and
Management
Apply information to solve
business issues
"By 2015, big data demand will reach 4.4 million jobs globally, but only one-third of those jobs will be filled."
Source: Gartner "Gartner's Top Predictions for IT Organizations and Users, 2013 and Beyond: Balancing Economics, Risk, Opportunity and Innovation" 19 Oct 2012
34. Analytics Cloud/OnPrem
Data Cloud/OnPrem
Hive Metastore
Elastic Cloud HDFS
Infinite Compute
Hadoop/Spark
Ingest Transform Analyze
External
Dashboards
Internal
Dashboards
Tableau Excel R Zeppelin
Web interface for distributed users
Data set definition
Social metadata dictionary
Export Web interface to dash-
boarding, query, and
data dictionary
Integrated ingestion,
transformation, and
query application for
business analysts
World-class, elastic
Big Data infrastructure
Hybrid Analytics Cloud/On Premises
35. Analytics Cloud/OnPrem
Analytics Cloud/OnPrem
Hive Metastore
Elastic Cloud HDFS
Infinite Compute
Hadoop/Spark
External
Dashboards
Internal
Dashboards
Tableau Excel R Zeppelin
Web interface for distributed users
Data set definition
Social metadata dictionary
Export Web interface to dash-
boarding, query, and data
dictionary
Integrated ingestion,
transformation, and query
application for business
analysts
World-class, elastic
Big Data infrastructure
Build reports
and
dashboards
Build outgoing
connectors
Ingest Transform Analyze
Business
Analytics, data
science
training
Write ETL and
perform data
engineering
Build
connectors
Hybrid Analytics Cloud/OnPrem
37. On-Premise DW to Cloud DW Migration for Leading Web Search Company
 Data base Availability : Most of the time ParaAccell DB database was down
due to Node Failure / Disk Failure. This was highly impacting on data load
process and relevant data is not available when it is required.
 Performance Issue : The query response time was low, which was impacting on
Reports performance.
 Cost / Support Issue : The overall maintenance cost for PADB was higher and
support from vendor was not up to the mark.
Jade Solution:
 Snowflake is cloud based DWH, which has cut down hardware cost
 Zero maintenance cost on hardware and software
 Required ongoing maintenance of database will be handled by Snowflake.
 Implemented Unified framework for Data Load
Business Problem:
Solution Architecture:
Value Addition:
Before Upgrade
After Upgrade
Analyze
Pilot
Migration
Migration
Roadmap
Migrate
 Analyze the current PADB environment and current Inventory
 Analyze the dependencies
 Identify pilot candidate for Migration
 Migration Analysis for pilot candidate
 Migration of pilot candidate
 Migration Validation of pilot candidate
 Build Migration of all objects.
 Include the learning from pilot migration
 Migrate Data from PADB to Snowflake
 Validate the migrated objects
 Point upstream and downstream application to snowflake
Decommissi
on / Other
 Implement Data Security and include required new changes
 Build unified framework to monitor the Data load and performance
 Decommissioned PADB schema
*** Received TDWI-Best Practices Award for 2016 under Emerging Technologies and Methods Category .
38. Segmenting Customer Base for Target Marketing
Business Problem :
 Identifying segment of customers who could respond to Target Marketing
Jade Solution:
 Performed clustering analysis in order
to group similar customers together
based on the numerical variables such
as Revenue, IT budget, # of
employees, # of IT employees &
manager count
 Since variables are measured on
different scales, the data is
standardized
 The distance matrix between each of
observations is calculated using the
clustering functions in the R package
 The predictive model that is created is
validated using silhouette plot
39. Smart Sense – Automation of Infra / App Monitoring and Management​
Business Problem:
 Manual monitoring of IT operations
decreasing efficiency and increasing costs.
 Applications downtime causing business loss
and customer trust​
 Lack of automated solutions for repeatable
problems causing organizations to re-vent
the wheel
 False alarms and unwanted alerts taking
away the focus on fixing the core problems
at hand​
Jade Solution:
 Smart Sense to automate Infra / App
monitoring and management thus freeing
the resources to focus on value added tasks​
 Self learning system that adopts to dynamic
environment thus alerting only for critical
business scenarios​
 Analyze to determine anomalistic behavior.
Solution Features
Issue Detection & Correlation with real-time events and
alert streams
Continuous Learning and Self-Optimizing​
Predictive Maintenance
40. Harnessing Predictive Analytics for Increased Cash Flow​
ERP
 Invoice base amount​
 Payment Term​
 ERP​
 No of total invoice paid​
 No of total outstanding invoices  Random Forest Trees​
 Association Rule​
 Cluster Analysis​
INCREASED
CASH FLOW​
Analysis Across
“Order-to-Cash” to
Improve DSO​
Business Problem:
 Accounts receivable (AR) can be a source of financial difficulty for firms when they
are not efficiently managed and are underperforming​
 Lack of systems and process that Identifies high risk invoices and customers for
better collection strategies​
 Better financial contingency measures if systems can predict with high accuracy if an
invoice will be paid on time or not and can provide estimates of the magnitude of
the delay​
Jade Solution:
 Jade’s AR Analytics improves Cash Flow by:
 Reducing Days Sales Outstanding (DSO)
 Freeing up cash​
 Enabling customized collection actions tailored for each invoice or customer through
predictive models​
 Effective management of AR would have positive impact on the financial performance of the firm​
 Provides solution for reducing outstanding receivables through improvements in the collections
strategy.​
41. Closing Thoughts
 today is all about engaging with analytics, in the future, it will be about
engaging the analytics battle with the help of technology.
 The leader will be decided by whoever has been more effective in
harnessing all this data to make better decisions
 Using the right technology to funnel, process and analyze all this data
for valuable actionable insights and decisions helping effective
business outcomes
Oil which is the fuel for modern economy for centuries. However, Oil in its raw form has little value. It needs to be refined and separated into a large number of consumer products, from petrol and kerosene to asphalt and chemical reagents used to make plastics and pharmaceuticals. It is also used in manufacturing a wide variety of materials.
Big Data is just like oil, in it’s raw form it provide no value to enterprise, until it is processed and valuable and actionable business insights are “distilled”.
Just like the technology that made available 100 years ago to discover oil and process it to consumable products. Big Data technology is going to transform and revolutionize the way enterprise get and use.
