TechEd 2008 presentation given in NZ and Australia.

2. Session Code: ARC206 Managing complexity in a Software plus Services World Amanda Jackson Fronde Systems Group Ltd

3. Presentation Outline (hidden slide): Speaker instructions: Complete this slide to assist your SME (subject matter expert) in evaluating your presentation flow, topic coverage, demo integration and alignment of content to your session description and level. Title: Managing Complexity in a Software plus Services World Technical Level: All levels Intended Audience: Developers, Enterprise Architects Objectives (what do you want the audience to take away from this session): 1. Understand what EA is and how things can go wrong 2. Understand complexity and how a system can end up being complex 3. Learn how to manage complexity and be successful using SIP Presentation Outline (including demos):

4. Presentation Overview Introduction Redefining Enterprise Architecture Understanding complexity Partitioning Mathematics of complexity SIP Summary

5. Amanda Jackson Consultant/Developer with Fronde Systems Ltd Started NZ Girl Geek Dinners in 2007 In the past has covered most roles in IT including: QA, Network Admin, Source Control Manager, Build Management, Games Development, Change Management

6. What is this talk about? Simplifying architecture through: Modeling complexity Partitioning complex systems By simplifying architecture we can more easily: Add business value Benefit from trends like Software plus Services

7. Software plus Services Combines Internet services with client and server software Increasingly complex, globally distributed systems More choices in the ways to access and manage software A new way of thinking is required at the EA level!

8. Simplify & Redefine Before we can simplify and redefine we need to understand: What Enterprise Architecture is Why we need Enterprise Architecture What can go wrong How things fail

9. What is Enterprise Architecture? Documentation or processes that describe: Structure Organization Behaviours Planning Changes

10. Enterprise Architecture “An enterprise architecture is a description of the goals of an organization, how these goals are realized by business processes, and how these business processes can be better served through technology.” Roger Sessions - Simple Architectures for Complex Organizations

11. Benefits of a good Architecture Adds value to an organization Is concerned with all aspects of an organization Ensures a good ROI Helps ensure successful projects

12. Why we need EA Based on a number of studies done in the last 10 years (such as The KPMG Canada Study, The Chaos Report and others), it has been established that more than 50% of IT projects fail outright! Andy Blumenthal – Director of Enterprise Architecture & IT Governance for the US Coast Guard

13. What can go wrong? Failed projects Loss of income

14. Why EA fails Excessive complexity Processes are: Expensive Time consuming Poorly defined Results are: Not testable Not reproducible Not directly usable Value is not delivered

15. EA Failures Four years ago the government announced to a grateful NHS a national IT program to become the world’s largest civil computer scheme. But after a breathless start, delivery dates for key software were missed, the full costs of implementation have always been unclear, and experts are divided over whether the scheme is too ambitious to ever work as originally planned. Questions that need to be answered on NHS IT plan by Tony Collins, April 2006 in ComputerWeekly,com

16. What does this mean for EA? In October 2007, Gartner predicted that 40% of all existing Enterprise Architecture programs will be shut down by 2010. Fewer than 5% of firms use Enterprise Architecture effectively. Ross, Weill & Robertson – Enterprise Architecture as a Strategy.

17. What is the problem?

18. Existing Methodologies Influenced by Object Oriented design & analysis Pre-date Software-plus-Services Continuously evolving but not catching up! Do not deal well with the new S+S design model

19. The new Enterprise Architecture Focus on complexity at the enterprise level Understand the mathematics of complexity Create a model for complexity Define a process for controlling complexity Test solutions against that model

20. What is Complexity? A function of the number of states that system can find itself in!

21. Software System Complexity values (heads, tails) penny; penny = read (penny_sensor); if (penny == heads) message (“Penny is heads”); if (penny == tails) message (“Penny is tails”); end; Complexity = States Per VariableVariables

22. Prepare Penny/Heads Memo Penny = Heads Memo RE: Penny Outcome: Heads Read Penny Prepare Penny/Tails Memo Penny = Tails Memo RE: Penny Outcome: Tails Business System Complexity Complexity = Branches per Decision Point Decision Point

23. Software & Business Comparison When considering complexity we can see that: Variables are like decision points States per variable are like branches per point

24. Mathematical Foundations

25. Partitioning The most important complexity control strategy Divide and conquer to reduce complexity Reduced complexity = improved efficiency

26. The Power of Partitioning Number of autonomous systems: 1 Number of states per variable: 6 Number of variables: 12 Number of states: 2,176,782,336 Number of autonomous systems: 2 Number of states per variable: 6 Number of variables: 6 Number of states: 93,312 Reduction in complexity: 99.57%

27. Looking at it another way… 1 Bucket / 2 Dice a.k.a. 1 autonomous system 2 Buckets / 1 Die Each a.k.a. 2 autonomous systems 1 Die = 6 states 1 Die = 6 states 1 Die = 6 states 2 Die = 6 6 states Total states this bucket: 36 Total states for 2 buckets: 12

28. And now with 12 variables… 1 Bucket / 12 Dice 2 Buckets / 6 Die Each 3 Buckets / 4 Die Each 1 Die = 6 states 12 Die = 612 Total states: 2,176,782,336 66 = 46,656 states per bucket 64 = 1,296 states per bucket Total states: 93,312 Total states: 3,888

29. What is a partition? A set of subsets that divide a larger set All elements of the larger set live in one, and only one, of the subsets

30. Five Laws of Partitions Must be true partitions Definitions must be appropriate to the problem at hand The number of subsets must be appropriate The size of the subsets must be roughly equal The interactions between subsets must be minimal and well defined

31. Set 1 Set 2 Set 3 Set 4 Set 5 Which are partitions? Unpartitioned

32. What do we partition? Consider same-category-as

33. Same-category-as Food Stationery Reading Kitchenware

34. Our new partitions Food cereal soft drink flour lollies Stationery pen pencil notebook Reading newspaper Kitchenware knife cup

35. SIP (very brief) Overview SIP describes the main approaches used for: Complexity control Partitioning Simplification Iterative Delivery Overall goal is to align the IT and Business processes so they work together effectively

36. Partition Identification Start at the highest possible view of the enterprise then attempt to partition If partition is successful – repeat Keep repeating until you cannot partition any more At each stage of partitioning, assign types and deployment information

37. Start at the highest possible view Retail Organization Operations Internet Sales Retail Sales Catalogue Sales Brand Awareness Product Awareness Planning

38. Identify Synergies Retail Organization Operations Internet Sales Retail Sales Catalogue Sales Brand Awareness Product Awareness Planning

39. Create simpler partitions Retail Organization Operations Sales Marketing Planning

40. Keep following your path Retail Organization Operations Sales Marketing Planning Sales Retail Distribution Retail Retail Sales Reporting Inventory

41. Partition Simplification Construct sub-classes of a partition. Any functionality that can be removed from an element should be removed

42. Software plus Services Hugely flexible Integrate traditional software with distributed services Or not… Put services in a cloud for direct user access A dizzying, and very impressive array of S+S services The ability for the architecture to get hugely bloated and complex if not planned well – a.k.a. Bloatware!

43. How do we partition S+S? Keep It Simple Architect the simplest possible complexity Make it obvious which elements live where Make it easy to see where partitions lay Don’t repeat elements

44. How do we partition S+S? Start at the highest possible view Create a high level architecture Show each service and application as they relate to each other Show clearly defined links

45. How do we partition S+S? Architect and partition each solution Only those relevant to your current objective Make each architecture as simple as possible Don’t repeat elements in multiple architectures Result? It’s easier to see solution links Result? It’s easier to redesign the whole

46. Case Study in Complexity Complexity can creep into even the most extensively planned project Unchecked complexity leads to project failure Using Simple Iterative Partitions might have saved this project, even when it was well into failure mode

47. NPfIT Launched in June 2002 Automate and centralize NHS record keeping Automation of all patient care information Access to any patient record by any authorized health care professional in the UK Ability to book appointments with any health care facility in the UK Automation of prescription services

48. NPfIT Functionality Regional Clinical Information Systems Initial cost - approximately NZ$14 billion Infrastructure Systems Initial cost – approximately NZ$3.2 billion Shared Applications Initial cost – approximately NZ$425 million

49. NPfIT Overview Multi-billion dollar project Contracts split between at least a dozen vendors Covers a geographic area of close to 100,000 square miles Offers services to 60 million people Expected to process 300 transactions per second

50. NPfIT Architecture Overview Region1 CIS H H H H H H H H H H H Region2 CIS Region5 CIS H H CRS H H Central Apps C&B ETP H H H H H H H Region3 CIS Region4 CIS H H Contact PACS H H

51. Current status of NPfIT In crisis almost from the first day One year into the project most vendors were having problems relating to each other The worst off, the Accenture/iSoft partnership.

52. Accenture/iSoft By Sept 2006 Accenture had enough and abandoned the project They walked away from over $5 billion in revenues They wrote off in excess of $700 million they had already spent They agreed to pay over $100 million to settle legal obligations.

53. Overall Cost of NPfIT? Estimated cost range is $68 to $142 billion

54. Simple Iterative Partitions Phase 1 – audit of organization readiness Phase 2 – working on the partitioning Phase 3 – simplify the partitions Phase 4 – prioritize subsets in the partitions Phase 5 - iteration

55. Phase 1 Highlights deep distrust between NHS factions Delivers extensive training in the nature of complexity Focuses on managing complexity as the absolute highest priority

56. Phase 2 Already considerable effort made to partition However, partitions were wrong FIVE different implementations of the same, very complex system This was done on purpose due to a highly suspect business rule SIP forces removal of the dodgy business rule, reduces complexity by 80% Create high level partitions

57. Phase 3 Simplify partitions further Remove up to 90% complexity from the 20% remaining Remove unnecessary functionality Make sure all technical requirements can be traced back to a business requirement

58. Summary Complexity of business processes is linked to decision points and the paths created by those decision points Complexity of software systems is linked to the number of variables and the number of significant values those variables can take Partitioning is the single major factor in reducing complexity

59. Roger Sessions Roger Sessions, author of ‘Simple Architectures for Complex Enterprises’, has been an invaluable source of help and information whilst preparing this talk. He very kindly allowed me to use material from his book as part of this presentation. Roger is the CEO of ObjectWatch, the creator of SIP and an Enterprise Architecture expert.

60. Q & A

61. Resources www.microsoft.com/teched Tech·Talks Tech·Ed Bloggers Live Simulcasts Virtual Labs http://microsoft.com/technet Evaluation licenses, pre-released products, and MORE! http://microsoft.com/msdn Developer’s Kit, Licenses, and MORE!

62. Related Content Breakout Sessions (session codes and titles) Interactive Theater Sessions (session codes and titles) Hands-on Labs (session codes and titles) Hands-on Labs (session codes and titles)

63. Track Resources Resource 1: Roger Sessions - roger@objectwatch.com - www.objectwatch.com Resource 2: Simple Architectures For Complex Enterprises - ISBN-13: 978-0735625785 Resource 3: amanda.jackson@fronde.com – fee free to email with any questions

64. Please complete an evaluation

65. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.