IDS Impact Innovation and Learning Workshop March 2013: Day 1, Panel Session 1 Richard Hummelbrummer, Martin Reynolds, Bob Williams
1. IDS WORKSHOP
Impact, Innovation, Learning
Panel Session
Values, Learning, and Systems
A framework for critical rigour
in impact evaluations
Richard Hummelbrunner
ÖAR Regionalberatung Graz, Austria
Martin Reynolds
Senior Lecturer, The Open University, UK
Bob Williams
Independent Consultant, Wellington NZ
2. CORE STATEMENT
…. evaluation asks and answers questions
about the quality, value, and/or importance of
things ….
If we’re not doing that, we’re not actually
doing evaluation.
Jane Davidson: Genuine Evaluation March 2013
3. The System Field
Different
systems
traditions which
have shaped
contemporary
systems
practice
Some Streams of Systemic Thought
(Draft update — May 2001)
* * *
Originated in 1996 by Dr. Eric Schwarz, Neuchâtel, Switzerland.
Extended in 1998, including items from the The Story of Philosophy by Will Durant (1933).
Elaborated in 2000-2001 from many sources for the International Institute for General Systems Studies.
Currently a research project of the IIGSS.
- - -
This rendition is the property of the International Institute for General Systems Studies.
All Rights Reserved.
Errors and omissions in this chart are solely attributable to the IIGSS.
Anti-chaos &
Adaptation
Stuart Kauffman
Criticality
Per Bak
Self-organization
Dissipative
Systems
Ilya
Prigogine
Complex Dynamical
Systems
Ralph Abraham
Chaos Theory
M.J. Feigenbaum
Nonlinear
Systems
Systems
Biology
Robert Rosen
Ecodynamics
Kenneth
Boulding
General
Systems
Theory
Majilo
Mesarovic
Dynamic Programming
Richard Bellman
Instrumental
Pragmatism
John Dewey
Psychological
Pragmatism
William James
Pragmatic Semiotics
Charles S. Peirce
Whole Systems
Design
Harold Nelson
General Systems
Philosophy
Ervin Laszlo
Dialectical
Idealism
Georg Hegel
Evolutionary
Philosophy
Herbert Spencer
Neuchatel
Evolutionary
Model
Eric Schwarz
Natural Evolution
Charles Darwin
1809-1881
Cybernetics
Norbert Wiener
Information
Theory
Claude
Shannon
Classical
Thermodynamics
Clausius,
Boltzmann
Semantic Theory
Donald MacKay
Informatics
(Computer Science
& Engineering)
Mathematics of
Feedback &
Electrodynamics
James
Clerk-Maxwell
Feedback governor
James Watt
Sciences
of the
Artificial
Herbert
Simon
Social Systems
Talcott Parsons
Sociological
Systems
Walter Buckley
Social Entropy Theory
Kenneth D. Bailey
Autopoietic Social
Systems
Niklas Luhmann
Management Cybernetics
Peter Senge
Appreciative
Systems
Gregory
Bateson
Social Ecology
Eric Trist,
Fred Emery
Family
Systems
Therapy
Watzlewick
Systems Ecology
Howard T. Odum
Programmable Loom
Joseph-Marie Jacquard
Analytical Engine
Charles Babbage
Theory of Computation
Alan M. Turing
Rational Philosophy
Aristotle
384-322BC
Symbolic Logic
George Boole
Symbolic Logic
Augustus de Morgan
Applied
Mathematics
John von Neumann
Artificial
Intelligence
Marvin Minsky
Cellular
Automata
Artificial
Neural Nets
Designing Systems
Bela H. Banathy
Teleonics
Gyuri Jaros
Critical Systems
Michael C. Jackson,
Robert Flood,
Werner Ulrich
Total Systems
Intervention
Robert Flood
Multi-Methodology
John Mingers
Metamodeling
John Van Gigch
Self-organized
Economics
W. Brian Arthur
Infodynamics
Stanley Salthe
Conventionalism
Henri Poincaré
Fractal Systems
Benoit
Mandelbrot
Artificial Life
Chris Langton
Mathematical
Social Science
Anatol Rapoport
Chaotic Systems
Edward Lorenz
Complex Systems
(Santa Fe Institute)
Homeostatic Systems
Walter Cannon
Queuing
Theory
Holism
Jan Smuts
Tektology
A.A. Bogdanov
Knowledge
Sciences
Brian Gaines
Engineering
Archimedes
Systems
Development
John Warfield
Linkage
Propositions
Len Troncale
Idealistic Philosophy
Plato
Systemic Development
Richard Bawden
Synergetics II
R. Buckminster
Fuller
Rheomodal
Systems
David Bohm
Cybernetic Semiosis
Douglas R.
Hofstadter
Transformational
Linguistics
Naom Chomsky
General Semantics
Alfred Korzybski,
Systems Trends
Kenneth E.F. Watt
Geophysiology (Gaia)
James E. Lovelock
Hierarchy Theory
Howard H. Pattee
Developmental
Structuralism
Jean Piaget
Co-creative Process
Hector Sabelli
Holonomic Systems
Ken Wilber
Communicative
Action
Jürgen
Habermas
Complex Evolutionary
Systems
Peter M. Allen
Knotted Systems
Louis Kauffman
Interpretive Systemology
Ramses Fuenmayor,
Hernan Lopez-Garay
Informational
Microdynamics
Vladimir Lerner
Social Semiotics
Floyd Merrell
Socio-Cybernetics
R. Felix Geyer
Pansystems
Wu Xuemou
Critical Ontology
Michel Foucault
Metasystem Transition
Turchin, Heylighen, Joslyn
Systemic Selfness
Paul Ryan
Homeorheotic
Systems
William Irwin
Thompson
Systems Semiotics
Luis Rocha,
Howard Pattee
Epistemology of Science
Sir Arthur Eddington
Time
Dilation
Lorentz
Topology of
Meaning
R. Ian Flett
Philosophy of Physics
John Archibald Wheeler
Mathematical Topology
Catastrophe
Theory
René Thom
Geometrodynamics
Albert Einstein
Quantum Systems
Planck, Bohr,
de Broglie, Dirac,
Schrödinger
Semantic Pragmatism
Ludwig Wittgenstein
Geometry of Meaning
& Reflexive Universe
Arthur M. Young
Abstraction of
Physical Laws
Galileo Galilei
Observational
Astronomy
Tycho Brahe
Heliocentric Astronomy
Nicolaus Copernicus
Socio-Technical
Systems
Harold Linstone
Software
Theory
Social
Psychology
Abraham
Maslow
Systems
Anthropology
Margaret Mead
Mathematical
Astronomy
J. Kepler
Structuralism
Claude
Levi-Strauss
Programming
Ada Lovelace
Engineering Design
Leonardo da Vinci
Human Science
Giambattista Vico
Geological
Superorganism
James Hutton
Biosphere
Eduard Suess
Earth as
Organism
Yevgraf
Korolenko
Earth as Organism
Vladimir Vernadsky
Species Cybernetics
Lotka & Voltarra
Holonics
Jeffrey
Stamps
Politics of Benefit
Cicero
Electronic Digital
Prototype
Atanasoff & Berry
Electronic Digital
Computing
Eckert & Mauchly
Operating Systems
Brooks et al
Operating Languages
Kenneth Iverson et al
Endosymbiosis
Lynn Margulis
Fuzzy
Systems
Lofti Zadeh
Fuzzy Logic
Bart Kosko
Calculus of
Variations
Dialectical
Materialism
Karl Marx
Paradigmatic
Revolutions
Thomas Kuhn
Periodic
Table
Dmitry
Mendeleev
Systematic Medicine
Hippocrates
Medicine
Celsus
Clinical Medicine
Thomas Sydenham
Conservation of matter
Antoine Lavoisier
Physical Chemistry
Robert Boyle
Antibacterials
Alexander Fleming
Electromagnetic
Propagations
Heinrich Hertz
Antisepsis
Lister
Clinical
Psychiatry
Benjamin Rush
Conservation
of Energy
Joule
Intrinsic
Uncertainty
Heisenberg
Superstrings
Brian Greene, et al
Evolution
Alfred Wallace
Pendulum Clock &
Wave Theory of Light
Christiaan Huygens
Determinism
Laplace
Scientific
Philosophy
Jacques Monod
Mathematical
Nominalism
Gottfried von Leibniz
Immaterial
Empiricism
George
Berkeley
Ecology
C.S. Holling
Bio-cybernetics
Manfried Eigen
Transcendentalism
Ralph Waldo
Emerson
Natural Philosophy
Nicholas von Cusa
Political Hegemony
Frederick the Great
Semiotic
Umwelt
Jakob von
Uexküll
Bio-semiosis
Jesper
Hoffmeyer
Comparative Linguistics
Benjamin Lee Whorf
Atomisitc
Linguistics
Antoine Fabre
d'Olivet
Symbolic Logic
Willard Van
Orman Quine
Operating
hardware
Blauw et al
Linear
Programming
Optimality
Theory
Synergy
Peter A. Corning
Appreciative Systems
William E. Smith
Holonics
Arthur Koestler
Mathematical Philosophy
Pythagoras
c.580-500BC
Geometry
Euclid
Logical Positivism
Rudolf Carnap
Process
Henri
Bergson
Meta-Skepicial
Empiricism
David Hume
Political Pragmatism
John Locke
Political
Romanticism
J.J. Rousseau
Critical Transcendental
Idealism
Immanuel Kant
Fluxation
& Unified Opposites
Heraclitus
c. 540-480 B.C.
Cynic
Philosophy
Antisthenes
Cynicism
Diogenes
Dialectic
Stoicism
Zeno of Citium
Religious Stoicism
Epictetus
Stoicism
Marcus Aurelius
Cosmological Unity
Giordano Bruno
Philosophy of the Whole
Baruch Spinoza
Nature Philosophy
von Schelling
Will & Idea in Nature
Arthur Schopenhauer
Perspectivism
Friedrich Nietzsche
Noosphere
Teilhard
de Jardin
Decision
Theory
Geometric
Programming
Utility
Theory
Laws of Form
George
Spencer-Brown
Blown Up Systems
Shoucheng OuYang
Structured
Software
Yourdon,
Constantine
, DeMarco,
Critical Theory of
Society
Max Horkheimer,
Theodor Adorno
Ecological
Cybernetics
Garrett Hardin
I Ching
2000-1000 B.C.
Social Cybernetics
Stuart Umpleby
Eco-pathology
Rachel Carson
Sructuralist
Mathematics
Bourbaki
KEY:
whit e general syst ems
red cybernet ics
black physical sciences
blue mat hemat ics
magent a comput ers & informat ics
green biology & medicine
yellow symbolic syst ems
orange social syst ems
olive ecology
gray philosophy
cyan syst ems analysis
purple engineering
General
Systemology
D.H. McNeil
Anthropological
Linguistics
Edward Sapir
International Encyclopedia of
Systems & Cybernetics
Charles François
1997
Mathematical
Logic
Bertrand
Russell
General Systems Theory
Yi Lin
Classical Mathematics
Development
of Mentation
Walter Lowen
Punctuated Evolution
Stephen J. Gould
Communicational Cybernetics
Klaus Krippendorff
Atomism
Democritus
Sophistic
Philosophy
Socrates
Organizing relations
Needham, Woodger
Rationalistic
Dualism
René Descartes
Analysis of
Complexification
Conrad H.
Waddington
Radical
Constructivism
Ernst von
Glasersfeld
Autopoietic
Systems
Rod Swenson
Unity & Stasis
Parmenides
Atomism
Leucippus
Dialectic
Zeno of Elea
Naturalism
Thales
Reciprocities
Anaximander
Scientific
Positivism
Francis Bacon
Scientific Empiricism
Roger Bacon
Scientific Method
Robert Grosseteste
Pan-ecology
John Muir
Systems
Philosophy
Mario Bunge
Modeling
J.L. le Moigne
Inheritance
Gregor Mendel
Genomics
Craig Venter,
Francis Collins
Systemics of
Benefit
Warren Ziegler
Eco-futurism
Hazel Henderson
Critical
Phenomenology
Edmund Husserl
Empirical
Sensationalism
Etienne Condillac
Evolutionary
Organicism
Saint-Simon
Empirical
Positivism
Ernst Mach
Positivism &
Sociology
Auguste Comte
Positivism
Lucretius
Positivism
Epicurus
Positivism
Aristapippas
Utilitarianism
Jeremy Bentham
Empirical
Utilitarianism
J.S. Mill
Transcendental
Idealism
Johann Fichte
Psychological
Determinism
Thomas Hobbes
Determinism
Voltaire
Progressivism
Condorcet
Intentionality
Franz Brentano
Existentialism
Martin Heidegger
Demographic
Cybernetics
Thomas Malthus
Political Economy
Adam Smith
Modern
Economics
John Maynard
Keynes
Economics
Paul Samuelson
Simulation
Problematiques
& Planning
Hasan Ozbekhan
Autopoietic
Systems
Humberto
Maturana,
Francisco
Varela
Harmony of Opposites
Lao Tzu
c. 570-490 B.C.
Statistics
Karl Pearson
Pagan Celts
Philosophy of
Science
Fritjof Capra
Metaphysics of
Science
Gary Zukav
Economic
Cybernetics
David Ricardo
Information
Theory
L. Brillouin
Linguistic
Mathematics
Hekki Heiskanon
Relativism
Postmodernism
Evolutionary
Positivism
Richard Dawkins
Deconstructionism
Jacques Derrida
Relativity of
Rationalites
Feyerabend
Structuralistic
Semiology
Ferdinand de
Saussure
Sociology
Emil
Durkheim
Narratology
Roland Barthes
Language &
Symbolism
Comparative
Languages
William Jones
Literary Semiotics
Umberto Eco
Logical Positivism
A.J. Ayer
Scholasticical
Empiricism
Thomas Aquinas
Personalism
Renouvier
Rationalism
Avicenna
Rationalism
Averroës
Psychoanalysis
Sigmund Freud
Analytic
Psychology
Carl Jung
Scholasticical
Sciences
Albertus Magnus
Scientific Philosophy
Paul C.W. Davies
Philosophy of Physics
& Mathematics
F. David Peat
Electromagnetism
Ampere, Kirchhoff,
Faraday et al
Electrostatics
Benjamin Franklin
Magnetism
Gauss
Biostructures
(chirality)
Louis Pasteur
Medicine
Moses
Maimonides
Scholastical Logic,
Dialectic & Linguistics
Peter Abelard
Scholasticism
Saint Anselm
Noetic Systems
Walter J. Ong
Semiotic Catholicity
Randolph F. Lumpp
Media
Semiotics
H. Marshall
McLuhan
Skepticism
Michel
Montaigne
Materialism
Denis Diderot
Management
Science
Philosophy of Regularity
Philosophy of Irregularity
River Map & Luo Book
Sociality
& Self
Confucius
YinYang
Fu Xi
Divination
Formal
Regularity
Zhu Xi
Tessellations
Roger Penrose
General Systems
Evolution
Erich Jantsch
Ecological
Demographics
Paul Ehrlich
Stabile Systems
Claude Bernard
Paradoxical
Imagery
M.C. Escher
Anatomical
Medicine
Galen
Medicine
Avenzoar
Medicine
Taddeo Alderotti
Medicine
Guy de Chauliac
Anatomical Medicine
Anreas Vesalius
Chemical Medicine
Aureolus Paracelsus
Microbials
von Leeuwenhoek
Circulation
William Harvey
Modern Theoretical Physics
Classical Physics
Critical Physics
Ronald W. Moses
Cosmological
Physics
Stephen Hawking
Environmentalism
David Brower
Dialectical
Naturalism
Friedrich
Engels
Grand Unified Treories
Steven Weinberg
Fuzzy Systemics
Vladimir Dimitrov
General System Theory
Ludwig von Bertalanffy
Operational Research
Russell L. Ackoff,
Stafford Beer
First Order Cybernetics
Sciento-politics
Ian Mitroff
Political
Systemics
Anthony Wilden
Military
Hegemony
Alexander
the Great
Scientific
Intuitionism
Michael Polanyi
Self-Reference
& Autonomy
Control Theory
Operations Research
General Systems
Purposeful Systems
Russell L. Ackoff
Philosophy
of Systems
Thomas
Cowan
Experimentalism
E.A. Singer
Inquiring Systems
C. West Churchman
General Systems
Thinking
Gerald M.
Weinberg
Management Cybernetics
Barry Clemson
Systems Cybernetics
W. Ross Ashby
Organicism & Process
Alfred North Whitehead
General Systems
Modeling
George Klir
Perceptual
Control
Theory
William T.
Powers
Living Systems
James Grier Miller
Control
System
Engineering
Systems Dynamics
Jay W. Forrester,
Donella Meadows
DNA
Crick,
Watson
Immunization
Jonas Salk
Directive
Correlation
Sommerhoff
Conversation
Theory
Gordon Pask
Politics of Cynicism
Niccolo Machiavelli
Mechanistic
Physics
Isaac Newton
Critical
Epistemology
Karl Popper
Linear &
Nonlinear
Mathematics
John Casti
Hyper-
mathematics
Charles
Muses
Soft Systems
Methodology
Peter Checkland
Rational
Mechanics
Approximation-
Estimation Theory
Grey Systems
Sifeng Liu, Yi Lin
General Tropodynamics
Soucheng OuYang, Yi Lin
Algebra
Aryabhata
Algebra
Brahmagupta
Algebra
Bhaskara
Algebra
Omar Khayyam
Algebra
Fibonacci
Rotational
Invariance
P.W. Bridgeman
Indian
Philosophy
Scientific Management
Frederick W. Taylor
Recyclical Universe
Hindu Mythology
Competitive
Evolution
Thomas Huxley
Systems &
Procedures
Heroic Legends
Homer
Rosetta Stone
Egypt
Applied
General Semantics
S.I. Hayakawa
Cyber-semiotics
Søren Brier
General Semiotics
Thomas Sebeok
Synergetics
Hermann Haken
Comparative
Anthropology
Franz Boas
Systems Engineering
Harold Chestnut,
Andrew P. Sage
Systems Analysis
Arthur D. Hall,
Van Court Hare
Second Order
Cybernetics
Heinz von Foerster
Topology of
Cybernetics
Warren
McCulloch
Viable Systems
Stafford Beer
Holographic
Systems
Karl Pribram
Constitutional
Pragmatism
James Madison
Philosophy of
Systems
Archie Bahm
Behavioral
Psychology
B.F. Skinner
Positivism
Josef
Popper-Lynkeus
Mathematical Logic
Gottlob Frege
Measure Theory
Giuseppe Peano
Projective
Geometry
Georg Cantor
Imperial Hegemony
Roman Empire
Genetics
William Bateson
Social Cybernetics
Geoffrey Vickers
Democracy
Cleisthenes
Cuneiform
Sumerians
Hieroglyphics
Egyptians
Alphabet
Phoenicians
Regularized
Language
Indo-Europeans
Democratic Idealism
Thomas Jefferson
Democratic Development
Pericles
Astronomy
Babylon
Classical Biology
Wholistic Biology
Barry Commoner
Formalized
Logic
Semio-Physics
René Thom
Group Theory
Felix Klein
Linear
Analysis
J. Fourier
Algorithmic
Complexity
Theory
G. Chaitin
Ideographics
Chinese
Pictographics
Aboriginals
Formalism
David Hilbert
Linear
Analysis
Dirichlet
Social
Systems
Sciences
Field
Theory
Evariste
Galois
Algebraic
Functions
Bernhard
Riemann
Analytic Geometry
Pierre de Fermat
Analytic Functions
Leonhard Euler
Analytic Mechanics
Joseph-Louis
Lagrange
Imperial
Hegemony
Napoleon
Probability Theory
Blaise Pascal
Algebra
Diophantus
Social Psychology
Willis F. Overton
Mathematics of
Concentric Spheres
Callippus
Astronomy
Eudoxus
Geometry
Theudius
Law
Solon
Conic Sections
Apollonius
Atomic
Weights
John Dalton
Inorganic
Chemistry
Organic
Chemistry
Liebig
Bio-Taxonomy
Carolus Linnaeus
Gradual Geological
Evolution
Charles Lyell
Geological
Tectonics
Symmetry
A. Zee
Manual
Transcriptions
Monastic Scribes
Printing
Johannes
Gutenberg
Observational
Astronomy
Hipparchus
Geocentric
Astronomy
Ptolemy
Polymath Philosophy
Nicholas Oresme
Quaternions
William
Hamilton
Internet
ARPA
Telecommunications
& Data Bases
James Martin
Apocalyptic
Dualism
Zoroaster
c.628-551BC
Numerical
Analysis
Hypervisors
Emulation
Collateration
& Hypertext
Ted Nelson
World Wide Web
Tim Berners-Lee
Systemic Perspectivism
Experimental
Psychology
Wilhelm
Wundt
Psychological
Modeling
Clark Hull
Clinical
Psychology
Techno-
Psychology
Psycho-
Biology
Psycho-
Physics
Morphogenesis
D'Arcy
Thompson
Geo-Impact
Evolution
Eugene
Shoemaker
Mathematical
Biology
A.J. Lotka
Eigenvalues &
Superposition
Daniel
Bernouli
Genetic
Algorithms
Sturm-
Louiville
Theory
Systems
Analysis
Perturbation
Theory
Generalized
Uncertainty
J. Ford
Documentary
& Narrative
Phonemes
Roman Jakobson
Hieroglyphics
Mayans
Social
Anthropology
Bronislaw
Malinowsky
Illuminationism
Saint Augustine
Rational
Voluntarism
John Duns Scotus
Nominalism
William of Ockham
Logical Nominalism
Jean Buridan
Interpretive
Physics
B.J. West
Mechanical Calculator
Wilhelm Schickard
Arithmometer
Charles de Colmar
Tabulating Machine
Herman Hollerith
Rapid
Miniaturized
Computing
Nanotechnology
Encoding
Schemes
R.W. Hamming
Philosophy of
Nature & Science
Carl von
Weizsäcker
Bifurcated
Cultures
C.P. Snow
Evolutionary
Systematization
Pierre Duhem
Pattern
Recognition
Psycho-
Metrics
Ecological
Evolution
Ernst Haeckel
Molecular
Genetics
Empirical
Metaphysics
George Henry
Lewes
Evolutionary
Philosophy
Richard L. Coren
Comparative
Psychology
C. Lloyd Morgan
Emergent Process
Samuel Alexander
Linear Algebra
Benjamin Peirce
Algebra of Logic
Ernst Schröder
Associative Algebras
William Clifford
Statistical
Biology
Karl Pearson
Modern
Statistics
Jerzy Neyman
Statistical
Mechanics
J. Willard
Gibbs
Model Theory
Bernhard
Bolzano
Model Theory
Alfred Tarski
Model Theory
Michael Morley
Metalogic
Alonzo Church Metalogic
Kurt Gödel
Intuitionism
L.E.J. Brouwer
Category
Theory
Saunders
Mac Lane
Double
Refraction
Augustin
Fresnel
Metric
Units
Astronomical
Measurement
Pierre Mechain
Sociological
Phenomenology
Max Scheler
Philosophy of
History
Wilhelm Dilthey
Renaissance
Platonism
Giovanni Pico
Number Theory
Marin Mersenne
Polymath Physics
Robert Hooke
Industrial Sociology
G. Elton Mayo
Philology, Linguistics
& Grammar
Ethnolinguistics
Wilhelm von
Humboldt
Ethnolinguistics
Johann Herder
Etymology
Jacob Grimm
Tagmemics
Kenneth L. Pike
Stratificational
Grammar
Sydney Lamb
Glossematics
Louis Hjelmslev
Morphology
Historicism
Benedetto Croce
Conceptualism
Ernst Cassirer
Tropology
Practical Medicine
Egyptians
Colonial
Hegemony
British Empire
Cultural
Hegemony
USA
Moral
Philosophy
Friedrich
Schiller
Organicism
Edward Stuart Russell
Philosophy
of Biology
Morton O.
Beckner
Number Theory
G.H. Hardy
Philosophy
Archelaus
Philosophy
Anaxagoras
Cognitive
Psychology
Aaron Beck
Structural Linguistics
N.S. Trubetzkoy
Realism
G.E. Moore
Analytical
Philosophy
Gilbert
Ryle
Sophism
Protagoras
Game
Theory
Automata
Theory
Software
Measurement
Capers Jones
Binary Numbers
China
Meta-languages &
Linguistics
U. of Nalanda, India
Algorithms
Donald Knuth
Structuralist
Psychology
Edward Titchener
Gestalt
Psychology
Max
Wertheimer
New Criticism
John Crowe
Ransom
Elementalism
Empedocles
Developmental
Psychology
Cognitive
Science
Allen Newell
Mechanistic Materialism
Julien La Mettrie
Computationism
Jerry A. Fodor
Literary
Criticism
I.A. Richards Standardized
Linguistics
C.K. Ogden
Critical Philology
Desiderius Erasmus
Humanism
Petrarch
Cognitive
Linguistics
George Lakoff &
R.W. Langacker
Intensional
Logic
Montague
Linguistic Logic
Jerrold J. Katz
Philosophy
of Art and
Language
Susanne K.
Langer
Problem-
Sovling
Propositional Logic
Chrysippus
Religious Stoicism
Cleanthes
Diogenes of Babylon
Roman Stoicism
Panaetius
Roman Stoicism
Poseidonius
Roman Stoicism
Seneca
Tectonic
Engineering
John Roebling
Industrial
Design
Coherentism
Otto Neurath
PERT
Delphi
Conditional
Probability
Thomas Bayes
Enlightenment
Jean d'Alembert
Ecology
Ethology
Konrad Lorenz
Objective
Information
T. Stonier
Biological
Evolution
Jean Lamarck
Vorticles
William
Thomson
Sociobiology
Edward O. Wilson
Systematics
Ernst Mayr
Interpersonal
Psychology
R.D. Laing
Cybernetic
Epistemology
Ranulph
Glanville
Mathematical
Economics
Antoine Cournot
Idealistic Liberalism
William Godwin
Mathematical
Psychology
R.D. Luce
Neo-Platonism
Plotinus
Social
Psychology
Kurt Lewin
Gestalt
Psychology
Wolfgang
Köhler
Mathematical
Economics
Vilfredo Pareto
Psycholinguistics
Sociolinguistics
Computational
Linguistics
Linguistic
Psychoanalysis
Jacques Lacan
Group
Dynamic
Sociometrics
Political
Sociology
Max Weber
Econometrics
Philosophy of
Law
Hans Kelsen
Eugenics
Francis
Primate
Anthropology
Jane Goodall
Primate
Linguistics
Linguistic Science
Leonard Bloomfield
Phonology
Physics As
Metaphor
Roger S. Jones
Applied
Transcendentalism
Henry David
Thoreau
Instrumental
Rationality
Peter Ramus
Methodized
Encyclopedias
Johann
Heinrich
Alsted
Conscientization
Paolo Freire
Robotics
Biodiversity
Urban
Ecology
Louis Wirth
Biotechnology
Genetic
Engineering
Green
Revolution
Norman
Borlaug
Heredity
P-L. de Maupertuis
Psychoanalysis
Josef Breuer
4. Core System Concepts
Understanding interrelationships associated with a
situation;
Engaging with contrasting perspectives regarding a
situation, and
Reflecting on boundaries of such representations and
interactions
5. INTERRELATIONSHIPS – Dynamics, Context, Complexity
• What is the structure of interrelationships within the situation (how are
components of intervention and/or effects connected)?
• What is the nature of the interrelationships (e.g. strong/weak, fast/slow,
direct/indirect, linear/non-linear)?
• What are the patterns that emerge from the interrelationships between
intervention and effects over time (e.g. simple, complicated, complex)?
• How context sensitive are these patterns (different effects at different times
or in different contexts)?
• How will this affect the ways in which effects can be associated with the
intervention? Understand what really works in what circumstances?
6. INTERRELATIONSHIPS – Relevance to Impact Evaluation
Emergence
Unintended & Unanticpated Consequences
Need For Keeping Measures Adaptable Rather Than Fixed
8. PERSPECTIVES– Relevance to Impact Evaluation
Untangling map from territory
Assessing why things worked out the way they did
Challenging idea of single “system”
10. BOUNDARIES– Relevance to Impact Evaluation
Broadening the range of values (utility, rights, virtues)
Identifying the values underpinning the intervention
Identifying the values underpinning the impact evaluation
11. Types of Learning
Effects Intervention
Values,
assumptions
Rules,
knowledge
Single loop learning
Double loop learning
Triple loop learning
Adaptive actions
Underlying causes and consequences
Mechanisms for learning, coping with problems and changes
12. Types of Learning and Systems Concepts
Perspectives Boundaries
Single loop learning
Double loop learning
Triple loop learning
Interrelationships
Are we doing
things right?
Are we doing the right things?
Is rightness buttressed by mightiness (or vice versa)?
Questions purpose, goals
Power for determining purpose and goals
Intervention Effects
13. A systems - based framework for rigor in evaluation
Intervention
Perspectives
Boundaries
Single loop learning
Double loop learning
Triple loop learning
Interrelationships
Instrumental
values
Critical /
Political
values
Intrinsic /
personal
valuesEffects
14. (ii) Partial in
serving some
stakeholder parties
including
practitioners - or
interests - better
than others
(i) Partial in
representing only a
section rather than
the whole of the
total universe of
interrelationships in
any context that
matters
All systems are partial
Systemic rigour and impact evaluation
Framework 2 (triangulation): 3 systems concepts in action
15. 1. Complex
situations or
Contexts of
change
and uncertainty
All Systems are partial in the sense of :
1. representing only a section rather than the total
universe of considerations
and
2. serving some parties - or interests - better than
others
Political space for exploring boundary judgements
Socio-economic space
for exploring
judgements of
‘fact’
Learning space for
exploring value
judgements
(i) Framing
interrelationships and
interdependencies
(ii) Framing engagement
with multiple
perspectives
3. Systems and other conceptual Tools
(iii) appreciating limits on
boundaries of interrelationships
and perspectives
Framework for justifying boundaries of:
(i) Framing interrelationships
(ii)Framing multiple perspectives
2. People
or stakeholders/
practitioners
Critical rigour and impact evaluation
Framework 3 (systems thinking in practice): 3 systems concepts in action
16. Relevance to Impact Evaluation Emphasis of CAS and Critical
System
Inter-
relationships
Emergence and uncertainty of impacts = systemic effect from interaction
of parts rather than single causality
… whole is more than the sum of it’s parts
CAS investigates the territory
Critical systems investigates the
implications of different maps of the
territory
Multiple
Perspectives
Complexity of ‘impacts’ arise from interaction between different
stakeholders and stakes
CAS illustrates different perspectives
predominantly from value-perspective
of utility
Critical systems explores the value
component of different perspectives
and ‘developing value’
Boundary
Judgements
Systems thinking explores single, double and triple loop learning in
terms of not only:
1. Is it being done right (efficacy, efficiency)? but also…
2. Is it right thing to do (effectiveness and ethics)? and...
3. Is rightness confused with mightiness (politics and power
relations)?
CAS largely focuses on efficacy,
efficiency and effectiveness
Critical systems focuses in additon on
effectiveness, ethics and power
Critical systems
Relevance and complexity
17. General
• Good impact evaluation requires address to perspectives and boundaries in addition (& in relation)
to complexity of interrelationships
• How might systems concepts complement and enhance (rather than replace or merely add to)
existing evaluation tools?
Interrelationships
• Measures of success (in a system) must adapt to change in circumstances (in situations)
• In what way can systems ideas improve the rigour and relevance of impact evaluation?
Perspectives
• Purposeful (as against ‘purposive’) developmental evaluation requires acknowledgement of, and
engagement with, ‘victims’ of intervention amongst others, in generating new value
• What are the conditions under which using systems ideas in impact evaluation is feasible and
value-able?
Boundaries
• Learning in impact evaluation requires address not only to utility (who learns what when?), but
changes in rights (and values) and responsibilities (power relations) ….triple loop learning
• What are the conditions under which using systems ideas in impact evaluation might be worthwhile
(good, right and politically just)?
Challenges from a (critical) systems perspective
Questions for discussion
18. Systems Concepts in
Evaluation. An
expert
anthology
(AEA
publication)
eds. Bob
Williams & Iraj
Imam
2007
Systems Approaches to
Managing
Change: a
practical guide
eds. Martin
Reynolds &
Sue Holwell
2010
Systems Concepts in
Action : a
practitioners
toolkit
eds. Bob
Williams &
Richard
Hummelbrunner
2010
Richard Hummelbrunner
Senior Partner
ÖAR Regionalberatung Graz
Austria
+43 316 31 88 48 - 15
http://www.oear.at
email: hummelbrunner@oear.at
Martin Reynolds
Senior Lecturer
The Open University
UK
+44 (0)1908 654894
http://sites.google.com/site/jjntest1/Ho
me/people/martin-reynolds-1
email: martin.reynolds@open.ac.uk
Bob Williams
Independent Consultant
Wellington
New Zealand
(64) 21 254 8983
http://www.bobwilliams.co.nz
email: bob@bobwilliams.co.nz
Contacts
...and some book publications