Business Intelligence in Agile Development framework

1. Doing BI the Agile
Way
Rajesh Nadipalli
Rev 3
Jan 2012

2. Presentation Objective
Traditional BI practices are generally slow in realization or change.
Typical IT projects take over 4 months before the business gets value
and change management is not quick either.
Using my personal experiences, I am laying out a simple blue print for BI
practice that is really Agile – both from technology and business.
rajesh.nadipalli@gmail.com

3. What’s in the Presentation
Recommendations for…
 High Level Process Flow
 Executive Sponsorship
 Business Architecture
 Initial Analysis
 Agile Requirements
 Maturity
 TechnologyArchitecture
 Platform Selection
 Agile Development
rajesh.nadipalli@gmail.com

4. High Level Process Flow
Business Sprint Development
Requirements
(high level) Sprint
2
Validate
Executive Business & Plan Initial Sprint Sprint
Release 1 3
Sponsorship Technology
proposals (theme
based) Sprint
4
Technology
Platform
(components)
Update Release Backlog
and work on next theme
(Release)
For Agile Process Overview, see these slides:
http://www.slideshare.net/nvvrajesh/agile-process-in-a-nutshell
rajesh.nadipalli@gmail.com

5. Executive Sponsorship
In current landscape where companies are faced with data growth
year-over-year is over 20%. Getting the right data into the
hands of the right people in time is a challenge which is the BI
promise.
However, to run a successful BI practice, you need to ensure the
management is on board with this strategy. It is also important
that communications with sponsors is consistent and timely.
rajesh.nadipalli@gmail.com

6. Business
Architecture

7. Business Architecture – Initial Analysis
A good start from the Business side is to..
• Understand current landscape
• Whatmetrics are used today
• Who uses them, are they stewards for this data
• How do they co-relate data across different systems
• Understand the pain points
• Who (roles) lacks information today
• What is loss of productivity due to incorrect or delayed information
• Are there multiple definitions for the same metrics
• Is there a common set of governed hierarchies?
• Build an ROI analysis to justify the program
rajesh.nadipalli@gmail.com

8. Business Requirements (high level)
Let’s take an example: You are planning to start a Portfolio Management Dashboard
• Plan the Categories of Metrics
• Program Health
• Resource Health
• Financial Health
• Define Roles of users who will use your dashboard
• Program Manager
• Vice President
• Financial Analyst
• Define Data Sources
• PPM system
• HR system
• Financial system
• Budget system
rajesh.nadipalli@gmail.com

9. Business Requirement Details
(Single Release)
• Define Hierarchies
• Fiscal Calendar (Quarters, Months as it relates to your Organization)
• Business Reporting (Business Unit -> VP -> Manager -> Employee)
• Identify which hierarchies have good sources and which need management
• Define Metrics Definition
• Program Manager
• Vice President
• Financial Analyst
• Define Data Sources
• PPM system
• HR system
• Financial system
• Budget system
rajesh.nadipalli@gmail.com

10. From Metric Definition to Effectiveness
Define Use Predict Effectiveness
Metric has
Automate Detect
Name analytical
collection Patterns
value
Metric has
Description Scorecard Intelligence predictive
value
Business Prevent Metric has a
Dashboards
owner issues ROI
Alerts /
Formula
Notifications
System of Decision
record making
Hierarchy
Maturity Model
rajesh.nadipalli@gmail.com

11. Technology
Architecture

12. Technology Platform Selection
Dashboards, DAAS
Typical User needs - Operational
Dashboards for canned reports, Ad-hoc
query interface, data as a service
Dashboards Ad-hoc DAAS
Operational & Metric Data Store
Operational Data Store like HDFS to host
copy of all known source data; Metrics
Cache would be a smaller set of computed
data; MDM is governed master data Operational DS Metrics Cache MDM
Data Processing Layer
ETL should be easy to extend, support
multi user development and connect to
various sources and targets. Example Map
ETL Monitoring
Reduce, Pentaho ETL, Talend
Source of Records
Plan for databases, files, web
services, internal and external SAAS
providers
Databases Cloud Files
rajesh.nadipalli@gmail.com

13. adapted from Forrester
Technology Stack by User Functionality
Advance Visualization Reporting Alerts Dashboard
Presentation Disconnected Usage Mobile Office Tools Ad-hoc Analytics
Integration
Metrics/KPIs Scorecards Planning
Performance
Management Performance Management Strategy and Objective
Collaboration ECM Portals eLearning
Supporting
Applications Knowledge Management MDM Metadata Repositories Life-cycle Mgmt.
Data/text/Video Mining Guided Analytics Natural Language Processing
Analytics OLAP Operational DSS Predictive Analytics
Behavioral Targeting Web Analytics Self-Service Analytics
Data Quality Report Mining Service Discovery Accelerators
Discovery & Registry
Integration EAI / SOA DB Accelerators BAM/CEP BPEL/BPM/BRE
ODS, DW, Data Marts EII (Enterprise Information ETL Unstructured and Web
Integration) Data Integration
Columnar DBMS Hierarchical/XML In-Memory Appliances
Data Multidimensional OLAP Multivalve DBMS RDBMS
Streaming DBMS Search Indexes Data Cache
Infrastructure Network Servers Storage Cloud
rajesh.nadipalli@gmail.com

14. Source: Oracle
Capability Maturity for a Practice
rajesh.nadipalli@gmail.com

15. Sprint Development
• Connect the dots with sample data
• To ensure the ETL, Data and Report developers are in-sync, first create some sample data
• Discuss the data flow and expected results at each step
• Estimate task effort for each user story
• Build Data Model & Map Reduce specifications
• Create a data model
• Create stubs for map reduce jobs
• Parallel Development
• ETL developer can work towards details of the job
• Report developer can work towards the user interface
• Integration
• Keep sufficient time for integration and resolve issues as a team
rajesh.nadipalli@gmail.com

16. Thank You
rajesh.nadipalli@gmail.com