Presented at Velocity Conference in September in New York City.
The information age is replacing the industrial age and corporations are going through changes as big as their original adoption of bureaucracy as an organizing principle.
This talk describes the notion of "Intentional Emergence" in a corporate setting. The idea that managers and technologists should create conditions for emergent outcomes rather than always focusing on the outcome itself.
DevOps and other mechanisms for a more dynamic IT culture are important to this end.
2. intentional: done on purpose.
deliberate.
emergence: the arising of novel and
coherent structures, patterns and
properties during the process of self-organization
in complex systems.
3.
4.
5.
6. Q: The traditional corporate / government
enterprises have the same technologies as
Silicon Valley.
So why do they deliver innovation so
differently?
30. And, maybe it takes a network
to build a network to interact
with a network.
31. Constructal Law:
For a finite-size system to persist in time (to live)
it must evolve in such a way that it provides
easier access to the imposed currents that flow
through it.
-From Design in Nature
More and more of the “currents” imposed on the
modern corporate enterprise are informational and
digital.
32. If you want information flow, architect
your systems to promote connections
33. “It is no exaggeration to say that if we had had to rely on
conscious central planning for the growth of our industrial
system, it would never have reached the degree of
differentiation, complexity, and flexibility it has attained.
…
Any further growth of its complexity, therefore, far from making
central direction more necessary, makes it more important than
ever that we should use a technique which does not depend on
conscious control.”
Friedrich Hayak, The Road to Serfdom
38. Gall’s Law: A complex system that works
is invariably found to have
evolved from a simple system
that worked.
So, we need to make lots of little
systems. And help them grow,
adapting quickly as they do.
41. “One” starts
here
Building the long tail of IT contribution. On
purpose.
42. Generativity = “a system’s
capacity to produce
unanticipated change
through unfiltered
contributions from broad
and varied audiences.”
43.
44. Some things that contribute to
generativity
Open Source
Software
Open
Standards
Runtime
Platforms
Low hurdles
for initial
project start
Small world
networks
Open Data
Open API’s
Simple rules
20% time Variable Cost
…
Community
48. Q: What changes to policy,
architecture, technology, or
culture would enhance long tail
emergence in the company you
work for?
49. Q: How can we better
impedance match our
organizations to the
decentralized and emergent
world we are immersed in?
50. “The test of a first-rate intelligence is the
ability to hold two opposed ideas in the
mind at the same time, and still retain the
ability to function.”
- F. Scott Fitzgerald
This talk is about creating fertile soil for emergence in the traditional corporate enterprise where DevOps is one of the important enablers.
Ask for people’s backgrounds: “Web” or “Enterprise”
Let’s start with a definition to baseline the discussion.
In a nutshell I think the web (and Silicon Valley) represent an emergent way of doing business …
… while the corporate enterprise remains stubbornly planned and reductionist. The information age is rewarding the former and causing pain to the latter in at least some quarters - pain that is beginning to force change. This talk is about broadening the way we think about that change.
If I’m honest, this session isn’t really about DevOps. I’ve been exploring these and related ideas for a while, and for this go around I just added DevOps to the title so that the Velocity editorial team would notice it. On the other hand, this talk in whatever version, has always been at least partially about DevOps even when I didn’t put that in the title.
None of these talks have been definitive, prescriptive. They are all me wrestling with the ideas out loud.
My interest in this topic started with this question. I was doing work for the government / DoD and I was struck by the fact that, despite big budgets, they couldn’t seem to make great thing happen quickly.
The answer has something to do with how they go about things. The center of effort just seems different in the two environments. Part of that might be scale and maturity, but that doesn’t seem to tell the whole story.
It’s much deeper than process and tools. The differences seem to be more about world view, culture, and evolutionary path.
Silicon Valley and the corporate enterprise aren’t just two stacks and two approaches to technology, they are different evolutionary lines, the manifestation of different adaptive schemes that succeeded in different evolutionary contexts (you might not like the corporate enterprise, but it is largely a successful form).
Aside: evolution makes a good framing for the discussion of progress, because evolution is in the direction of increasing complexity (on average). As the dynamic environment becomes more complex, the organizations interacting with it become more complex too to adapt. Hierarchy and bureaucracy have built in limits to the complexity that they can handle so this transition is qualitative.
This should be obvious - but sometimes we miss the big things in our focus on the minute - the industrial age is in transition into the information age and the things that made those environments different are fading.
As the industrial age unfolded, railroad networks connected markets and made the rise of national brands possible. Digital networks are connecting everyone and everything and will make even more dramatic changes possible. Rapidly changing the scale and nature of the dynamic environment in which they operate.
More than just hierarchy, rules, and specialization though, bureaucracy was our first corporate systems architecture. It was the original technology stack that businesses used for passing messages, storing state, and processing data.
Say what you will, but bureaucracy was a successful evolutionary adaptation.
Unfortunately, in today’s networked and fast moving dynamic environment, bureaucracy remains a low band filter with delay, only the big project bass notes get through, slowly, and its impact on maneuver is predictable.
It was effectively adapted for the industrial age, but as a rationalist product of reductionist, modernist, centralized, hierarchy it is increasingly out of step with our decentralizing expansionist world.
Btw, you may notice that I tend to mix metaphors and draw ideas from lots of places…I find useful analogies in the second law of thermodynamics, from evolution, complex adaptive systems, theories of intelligence, neuroscience, military strategy and etc.
I take inspiration from Col John Boyd. He is known as a grand strategist, but I think of him as a grand synthesizer. People who attended his lectures describe the breadth of his source material and that his talks were as much about the intellectual journey as they were about the end strategy.
I make absolutely no claims of being like Boyd, but I like the way his thinking gave permission to look far afield for analogy and inspiration.
Ok, back to the topic, sometimes I think talk about silicon valley, the enterprise, open, closed, ITIL, DevOps, centralized, decentralized, etc. are really just code words for the culture struggle between modernism and post modernism in the workplace. And it’s a struggle that isn’t confined there. It’s throughout our society.
The industrial age informed and created one kind of social fabric, the network / information age is creating another.
The network age doesn’t just extend industrialization, it leaves it behind. Modernism and reductionism may be in their final steep decline. They are being supplanted by a new kind of network empiricism that will exert its own gravitational force on our social and organizational fabric. George Bush, with his perfect illustrations of reductionism (“You are with us or against us”) will probably be our last president of the modernist era.
Reductionism remains intuitively appealing to us as humans in need of simple narratives of cause and effect but is increasingly out of step with our complex networked world.
So let’s go through a few different ideas that apply to this transforming enterprise.
Bureaucracy hasn’t sat completely still. The network is making its way into our traditional bureaucracies and and is causing them to evolve into hybrid hierarchy / networks, some form or another of a post-bureaucratic enterprise. Often this isn’t on purpose of course, it lacks the “intentional part” - these networks are overlays on to the hierarchy of the official organization chart.
And we *need* that increasing internal organizational complexity because a hierarchical organization can never be smarter than the people who run it, but our extended ecosystem is just too complex for that small group of managers to understand and react to. So we need internal structures, processes, and time cycles that are better able to cope.
We need the corporation to be smarter than the people that run it. And in fact, smarter than the sum of all of the people in it.
We must be both emergent and planned. The greatest challenge to our organizations is being able to hold both of those ideas in our cultural minds at the same time.