A presentation to Interspire, Sydney (July, 2011), looking at patterns of software growth and change, and strategies for managing complexity in large legacy applications.

1. Evolving Software
How to manage explosive growth
without exploding code

2. Why does software change?
Adding a feature
Growth
Behavior is modified
Fixing a bug
Repair
Improving structure
Refactoring
Behavior stays the
same
Improving performance
Optimization

3. The balance of value
!
Creating features and fixing bugs adds value
directly to the product
!
We can measure the success of these activities by
monitoring conversions and sales
!
Performance optimization is empirical
!
We get hard data on whether we did it right or not
!
Structural change is difficult to quantify
!
Impacts on features, bugs, and performance
!
Highly subjective

4. Why structural change matters
!
Software growth is not constant
!
Bursts of activity; release and QA cycles
!
We can’t anticipate all future requirements
!
We want change with minimum cost and effort
!
Requires code to be supple and adaptable
!
We want predictable results from change
!
Requires effects of code changes to be localized
!
Requires loosely coupled, easily testable objects

5. Measuring Programming
!
Creative activity: no statistical feedback to tell us
whether we are reaching our goal in code
!
Little or no correlation between time spent coding
and quality of the result
!
We can mine data from code syntax; version
control history; bug trackers
!
But the same metrics have different meanings,
depending on the particular team and codebase

6. Looking at code on a larger scale
!
Methods per class 70
!
God class 60
!
Lines per method 50
!
Complex method 40
Revisions per code file
Metric
!
30
!
Churn 20
!
Bug fixes per code file 10
!
Instability
0

7. Statistical Similarities
!
All large code bases seem to share common
statistical features
!
The shapes of many software metrics conform to
power law or log-normal distributions
!
AKA: 80/20 rule, Pareto Principle, ‘long tail effect’

8. The shape of software
!
These statistical discoveries give us a new way of
looking at widely held ideas about software
engineering:
!
The ‘Big Ball of Mud’ is the wrong metaphor — even
messy legacy systems have a definite shape
!
The ‘Lego Hypothesis’ is invalid — object oriented
software is not constructed by connecting reusable,
modular components

9. What we say versus what we do
!
Traditionally, we describe effects like overly long
methods and complex classes as ‘code smells’
!
But these effects seem to be common to every
large code base in existence
!
As software engineers, we have to account for this
in our methodology

10. Behavioral Economics
!
Psychological explanation for why code tends
towards certain shapes
!
Stronger incentives for getting something working
quickly than sacrificing time and mental effort to
create new abstractions
!
Easier to reason about code changes in one place
!
Naming things is one of the most difficult problems
in software development

11. Actor Networks
!
Sociological explanation for why code tends
towards certain shapes
!
Conway’s Law: structure of systems tends to
mirror that of the organization building the system
!
The boundary of an application is an entirely social
construction
!
Rationale for coding decisions is lost when
developers leave the team or new requirements
overwhelm an existing system

12. Accidental Complexity
!
A by-product of the technology used to solve the
problem
!
Cost of change increases with the volume of code
!
Code that is difficult to change
!
Unavoidable
!
Indicator of code not evolving to fit changing
requirements
!
Indicator of a mismatch between chosen abstractions
and the shape of the problem domain

13. Granularity
!
Choice of programming language matters
!
Translating a high-level concept into actual code
!
Coarse grained APIs are inflexible and require heavy
customization and workarounds
!
Fine grained APIs are hard to use and require large
amounts of wrapping code and sequential calls
!
Architectural style:
!
Convention or Configuration
!
Library or Framework

14. How Buildings Learn
!
Layers of a building grouped according to different
rates of change:
!
STUFF
!
SPACE-PLAN
!
SERVICES
!
SKIN
!
STRUCTURE
!
SITE

15. How Applications Learn
!
Layered architecture
!
Lower levels wrap infrastructure that changes less
frequently
!
Upper levels wrap business logic that changes
constantly
!
Domain driven design
!
Minimize accidental complexity through a rigorous
approach to language
!
Look for implicit models hidden in existing code and
flesh them out

16. Ubiquitous Language
!
A living document of all the terms, concepts, and
relationships that drive product features
!
Bridging product level thinking and technical
architecture
!
Models behaviour from a software point of view
!
Encapsulates essential complexity
!
Changes and evolves in sync with the product

17. Relentless Refactoring
!
Refine the model; improve the language
!
Discover new organizing concepts
!
Focus on the relationships between components
!
Each object has a single responsibility
!
Small pieces, loosely joined
!
Break apart entanglements
!
Extract new classes and methods from existing code
!
Extract new applications from existing code

18. Component Boundaries
Application Logic
!
Dependencies are
transitive
Domain Model !
Lower level packages
should never depend on
higher level packages
Data Store

19. Evolving Architecture Patterns
!
Strangler App
!
Introduce new architectural abstractions that wrap the
existing system like a creeping vine that eventually
overtakes the host tree
!
Parallel Rewrite
!
New architecture develops side by side with the old
system within a single app
!
Extract App
!
New applications and service layers are spawned off
the original system