KPI Partners E-Book: The Project Analytics Framework
Camp It, June 2012, How To Design Your Bi Architecture To Capitalize on New Technologies
1. How to Design Your BI Architecture
to Capitalize on New Technologies
Craig Jordan
Advisor, Business Intelligence
American Family Insurance
(c) 2012 by Craig Jordan. All rights reserved 1
2. My Background
• Software architect for 17 years
• Experience with business intelligence, client-
server and web applications
• Currently responsible for technical leadership
for our business intelligence architects.
• Creating a 2-4 year roadmap of the data
suppliers, storage structure, and consumers
(c) 2012 by Craig Jordan. All rights reserved 2
3. Today’s Topic
• How to decide where to start in terms of
revisiting the architecture
• How to incorporate new technologies into the
architecture
• How to govern new technology adoption to
prevent unnecessary proliferation
• When to abandon a technology adoption
(c) 2012 by Craig Jordan. All rights reserved 3
4. First, some background
• What is architecture?
• Why would revisions be necessary?
(c) 2012 by Craig Jordan. All rights reserved 4
5. Software architecture is…
• An architecture is the fundamental organization of a
system embodied in its components, their
relationships to each other and to the environment
and the principles guiding its design and evolution
• Key Concepts
– Includes structure and behavior
– Includes decisions and rationale
– Architectural elements have internal and external relationships
• Purpose
Enable business strategy through the appropriate and effective use
of technology
(c) 2012 by Craig Jordan. All rights reserved 5
6. Architectural impact of new technologies
New technologies could…
•Provide new components
•Make existing components obsolete
•Re-organize the internal structure of your BI
environment
•Change the relationships between the components
of your architecture and its environment (users,
other systems)
•Change the principles that govern the evolution of
your BI environment
(c) 2012 by Craig Jordan. All rights reserved 6
7. How to think about an architecture
Anemone World Map
Blueprint
City Plan
(c) 2012 by Craig Jordan. All rights reserved 7
8. What is Innovation
• innovate (verb)
to introduce something new;
to make changes in anything established.
• Innovation (noun)
the creation of better or more effective
products, processes, services, technologies, or
ideas that are accepted by markets,
governments, and society.
(c) 2012 by Craig Jordan. All rights reserved 8
9. A Model of Innovation
New
Innovation
How 1 3
Status Quo 2
Same
Same What New
(c) 2012 by Craig Jordan. All rights reserved 9
10. A Model of Innovation
New
Innovation
Pain Points
Proofs of Concept
Prototypes
Pilot Projects
How
Expanded Vision
Status Quo Enhancements to
current technologies
Same
Same What New
(c) 2012 by Craig Jordan. All rights reserved 10
11. New
Architecture innovation
example
How
Improve XML acquisition processing
Same
•Benefits Same What New
Simpler development & maintenance
Seamless integration with standard ETL approach
•Drivers
New operational systems navtively integrate via XML
•Business Case
Developer productivity
More uniform operating environment
(c) 2012 by Craig Jordan. All rights reserved 11
12. New
Architecture innovation
example
How
Operational use of ETL tools
Same
•Benefits Same What New
Operational data integration reuse BI data integration
expertise
Better teamwork
•Drivers
Initial data loads for new operational systems
System integration requirements
•Business Case
Developer productivity
Lower operating cost
(c) 2012 by Craig Jordan. All rights reserved 12
13. New
Architecture innovation
example
How
Expand vision for business intelligence
Same
•Benefits Same What New
Simplify data analysis for new cross-functional teams
Prepare for advanced analytics requirements
•Drivers
Organizational change
Operational system implementations
•Business Case
Support analysis of critical business initiatives
Competitive advantage
(c) 2012 by Craig Jordan. All rights reserved 13
14. (c) 2012 by Craig Jordan. All rights reserved 14
15. A business is a legally recognized organization designed to provide goods and/or
services to consumers, businesses and governmental entities. A business is
consumers
typically formed to earn profit that will increase the wealth of its owners and grow
the business itself. The owners and operators of a business have as one of their
main objectives the receipt or generation of a financial return in exchange for
work and acceptance of risk.
risk
(c) 2012 by Craig Jordan. All rights reserved 15
16. (c) 2012 by Craig Jordan. All rights reserved 16
17. Intelligence is a very general mental capability that, among other things, involves
the ability to reason, plan, solve problems, think abstractly, comprehend complex
reason plan problems abstractly
ideas, learn quickly and learn from experience. It is not merely book learning, a
ideas experience
narrow academic skill, or test-taking smarts. Rather, it reflects a broader and
deeper capability for comprehending our surroundings — “catching on”, “making
sense” of things, or “figuring out” what to do.
(c) 2012 by Craig Jordan. All rights reserved 17
18. A revised vision for business intelligence
Original New
Scope: Scope:
Business intelligence consists of Business intelligence is the use and
enterprise reporting and strategic analysis of information that enables an
analysis organization to best lead, decide,
measure, manage and optimize to
Tools: achieve efficiency and financial benefit.
ETL, BI Platform, Limited java/web,
New/Enhanced Tools:
data warehouse DBMS
ETL, BI Platform, data warehouse
Latency: DBMS, XML, DR / HA infrastructure,
standard DBMS
Daily, monthly, quarterly, ad hoc
New Latency:
Team: Near real-time
BI development team, ad hoc analysts
Additional Team Members:
Business analysts,
(c) 2012 by Craig Jordan. All rights reserved OLTP architects/developers 18
19. Revised scope of business intelligence
Strategic Operational
Report Status Quo
Analyze
Future
Monitor
Innovation
Predict Opportunities
Recommend
Decide
(c) 2012 by Craig Jordan. All rights reserved 19
20. Broadened perspective of the
purpose of a BI environment
rary
st Bed A Lib
e l Te
M od
tive dio
re d
ic
’s Stu
tist
A P
Ar
An
ine e
ldm Lin
nG
o bly
atio Assem
orm
Inf
An
(c) 2012 by Craig Jordan. All rights reserved 20
22. How to incorporate
• Identify technologies of interest
• Maintain an awareness of significant projects
being considered
• Conduct project-independent trials
• Conduct project-specific proofs-of-concept
• Identify & execute pilot projects
• Move into mainstream use, as appropriate
(c) 2012 by Craig Jordan. All rights reserved 22
23. Challenges
• Architecture investigations and POCs take time
and money, but for innovation they are
required
• Pilot projects incur risk, but project leaders are
risk averse
• Sound business intelligence architectures
require enterprise scope, but business
intelligence solutions are typically
implemented as projects
(c) 2012 by Craig Jordan. All rights reserved 23
24. Solution
• Strong support
The BI Radar Projects
• Longer timeline
• Specific rqts that justify
•Target your project- innovative technology
• Tolerance for risk
independent research
•Motivate project leaders to
engage in innovation
Technology innovations
• Maturity suitable for
needs
• Alignment with staff and
technologies
(c) 2012 by Craig Jordan. All rights reserved 24
25. How to think about an architecture
Anemone World Map
Blueprint
City Plan
(c) 2012 by Craig Jordan. All rights reserved 25
26. Architecture as a city plan
• Long range plan
• Addresses larger problems
• Allows for independent
development with coordination
• Provide just-in-time
infrastructure
• Adjusts to changing pressures
and requirements
City Plan
(c) 2012 by Craig Jordan. All rights reserved 26
27. Where to start
1. Determine where there are existing
requirements that you could improve and
whether there is enough force to support the
move
2. Find ways to meet new requirements with
existing technologies that expand your
architecture naturally
3. Combine 1 and 2 leveraging the natural
forces / project requirements
(c) 2012 by Craig Jordan. All rights reserved 27
28. How to govern
• Clearly articulate an architecture strategy that
specifies the purpose of primary components
and tools
• Allow tools to compete for placement in the
strategy
• Emphasize the benefits of limited proliferation
• Govern tool selection by principles
• Define the triggers for revising tool placement
(c) 2012 by Craig Jordan. All rights reserved 28
29. Articulate architecture strategy
Architecture Drivers
Completeness Consistency Integration Speed to market Accessibility Cost of ownership
Source Operational Analytic Semantic BI Tool Layer
Systems Data Layer Data Layer (Data
Access) General Purpose
Operational Data Detailed Data Marts Layer • BI Platform BI User
Stores • Customer • MS Office Suite
Policy • Customer • Policy
• Policy • Billing Function-Specific
• Billing • Claims • Actuarial
• Claims • Product • Fraud Scoring
Billing NRT • Product • Channel On On
Nightly
• Channel • Finance Demand Demand
• Finance • Reinsurance Examples of Business Intelligence
• Reinsurance • Investment Operational and strategic reporting
Claims Statutory reporting
• Investment
Regulatory reporting
Analysis Sandboxes Aggregate/Summary
Data Marts Function-specific and cross-function
• Profit/Loss View analysis
Customer
• Point in Time
Dashboards
• Workflow
Scorecards
Cube Web Presentation
Agent Analysis Sandboxes Canned Report Delivery
On Demand Ad hoc Report/Analysis
SOA Data Mining
On Demand Predictive Analytics
Security | Archival | Access Monitoring | Automated Distribution | Metadata | Contribution | Collaboration
(c) 2012 by Craig Jordan. All rights reserved 29
30. Govern tool selection by principles
• No single BI platform tool is sufficient
• Not all BI requirements will be met onsite
• Repeated data integration should be automated
• Limiting required skillset helps deepen expertise
• Success depends, in part, on access to industry
resources (contact developers, vendors)
• Most decisions have a business perspective
• Most decisions are cross-functional
(c) 2012 by Craig Jordan. All rights reserved 30
31. Define triggers for revising decisions
Examples of these triggers include
•Failed POCs
•Pilot projects that are unable to apply
technologies as expected
•Major (“platform”) releases
(c) 2012 by Craig Jordan. All rights reserved 31
32. When to abandon
Project A
Simply: When a technology
cannot achieve the value
expected in the vision
Projects
Technology innovations
Technology B
(c) 2012 by Craig Jordan. All rights reserved 32
33. When to abandon, an example
1. Architect-led learning and project-independent
trial completed
2. Potential pilot project identified
3. Candidate architectures defined
4. Architect-led “sales pitch” to management
5. Technology team trained in the new technology
6. Release 1 for pilot project attempted with
vendor support
Outcome: Technology abandoned because it could
not provide expected developer efficiency and
phased implementation of target architecture
(c) 2012 by Craig Jordan. All rights reserved 33
34. When to revisit, an example
1. Architect-led learning
2. Potential pilot project identified
3. Candidate architectures defined
4. Architect-led “sales pitch” to management
Outcome: Management unsupportive due to lack of potential value –
no immediate project with need
When new, well-supported projects, began
8. Revisit proposed architecture and value
9. Gained management support
10. Architect-led project-specific POC completed by technical lead
11. Value demonstrated
Outcome: Technology adopted and in place today
(c) 2012 by Craig Jordan. All rights reserved 34
35. How to Design Your BI Architecture
to Capitalize on New Technologies
Q&A
(c) 2012 by Craig Jordan. All rights reserved 35
36. How to Design Your BI Architecture
to Capitalize on New Technologies
Thank you
craig.jordan@amfam.com
(c) 2012 by Craig Jordan. All rights reserved 36
Hinweis der Redaktion
I am fortunate to work with a talented and dedicated team of architects
Before we start to talk about revising your architecture it would be best for us to be on the same page regarding what “architecture” means and why revising it would be necessary.
Cell Phone Example: Cell phone introduced to the home environment Could argue whether it simply provided new ways to accomplish existing tasks or if it was an entirely new component Made some components obsolete (land line, …) Reorganized the internal structure of the environment – phone is no longer centrally located Changed the relationship of components internally – phone is no longer physically attached to the house Changed relationships to the external environment – phone is no longer institutional, rather it is personal Changed the principles governing the evolution of the home – such as how many phone lines a home could support.
Since an architecture is supposed to support the accomplishment of business strategy & objectives. Hopefully you are revising it to do something new for the business or something that your business already does better.
Doing something new can be further broken down: Doing something new that someone else is doing (internally or externally) Doing something new that no one else is doing (internally or externally) Each of these cases must be handled differently.
Doing new things (WHAT) where your current tools/techniques are insufficient leads you to the upper right. Sometime this is a matter of projecting pain points that would exist if you simply tried to move into the lower righthand quadrant. One of the techniques for envisioning a future environment for our BI solutions involved > Evaluating our business needs against the scope that our environment currently handled regularly. This lead to an awareness of gaps in scope – where there were needs in the organization for BI solutions that our team and environment were not fully ready to address > Evaluating our technical enviornment against the technologies necessary to meet the gaps in scope