1. Walking on two legs:
LEARNING EVALUATION
• Göran Brulin, Senior Analyst and professor,
Swedish Agency for Economic and Regional
Growth
• Sven Jansson, National Coordinator for
Evaluation and Monitoring, Swedish ESF
Council.
1

2. Walking on two legs:
LEARNING EVALUATION
According to the strategy “Europe 2020” an economy
based on knowledge and innovation should be
developed. EU should promote a resource efficient,
greener and more competitive economy that leads to
growth. It should foster a high-employment economy that
help deliver social and territorial cohesion!
2

3. Smarter, inclusive growth calls for
a change in how programs and projects
are conducted and funds carried out!
• There is a lack of learning approaches in
evaluations!
• Evaluations are conducted just for the sake of
evaluation!
• Evaluations are ritual and symbolic activities
rather than processes for critical and constructive
knowledge formation!
3

4. Reflections after the last programming
period regarding learning and evaluation:
- The Swedish National Audit Office: too much detailed control and
monitoring – too small investments in learning to support regional
development!
- Swedish Institute for Growth Studies: big investment in regional
development projects from the EU funds, almost no effects on regional
growth can be identified!
- Swedish Agency for Public Management: too many small projects; too
scattered project portfolio; hardly any cooperation between the Regional
and Social funds!
- The Guidelines from the Commission: the traditional mid-term
evaluations were often conducted too late and had little impact on the
implementation of the programmes; should be replaced by ongoing
evaluation in projects and of programmes this programming period!
4

5. Learning spiral:
Joint evaluation ”Public debate”
of the for sustainable
implementation regional growth
organisation and jobs!
”Evaluation
capacity building”
”Ongoing evaluation” of
programmes and in projects.
5

6. The main task in the new generation of
evaluation is to organise reflective learning
processes for continuous improvement!
• Mid term evaluation came too late and was too expensive;
minor impact in managing of the programmes.
• Quantitative rather than qualitative focus.
• Indicatororiented evalutions means heavy focus on activities in
the projects (at the expense of the overall objectives). How
convincing are the core indicators?
• Limited learning in the projects, in the program context and
between stakeholders in national framework (labour market
policy and regional growth policy actors).
6

7. Actions taken in Sweden during the
programming period 2007 – 2013:
• Ongoing evaluation of the eight ERDF programs,
and in major strategic projects.
• Ongoing evaluation in ESF (OP) and in major of
project.
• Joint evaluation between the ESF and ERDF of
the implementation organisation.
• Joint university course and reader in Learning
Through Ongoing Evaluation at five universities
7

8. Actions taken in Sweden during the
programming period 2007 – 2013
• Learning network between evaluator creates
learning and capacity building
• Joint learning conferences and seminars in each
of the eight regions/structural programmes
• Support project on Learning and Strategic Impact
8

9. Learning evaluation in the
organisation of implementation:
• The ongoing evaluation has initially focused on the effectiveness
processes of preparation of applications, case handling and
prioritisation.
During the second half of the programming period, the ongoing
evaluation:
• focus more on learning and strategic orientation.
• and on whether the implementation organisation has managed to
assimilate documented experience and steer towards the
programme goals better.
A main result of this effort has meant that customer focus has risen
in both funds and initial conflicts in the implementation
organisation have been solved!
9

10. Learning evaluation at the programme level:
• An ongoing evaluation and process support structure of the eight
Swedish regional fund programmes (ERDF:s) are carried out.
• Teams of researchers are continuously studying the progress of the
programmes, analysing the evaluation of the projects, and evaluating
how the work is progressing at regional level. The main task is to
ensure that the project portfolio is well balanced in terms of the
programme’s goals. Results and impacts are in focus.
• Learning processes between the projects and in the programmes have
been initiated and enhanced.
10

11. Learning evaluation at the project level:
• Larger projects are requested to carry out ongoing
evaluation. They are supposed to set up a plan for ongoing
evaluation, call for tender and learn from the ongoing
evaluation to support continuous improvements.
• This evaluation approach supports project leader as well as
managing authority and other stakeholders with knowledge.
• In general ongoing and learning evaluation at the project
level has meant a clearer focus on job creation, regional
growth and innovation.
11

12. The Social Fund support learning through
ongoing evaluation by different means:
• The Process Support Projects which aims to increase
opportunities for different actors in the system to set up
new projects that are innovative and have an additional
edge compared to traditional labour market policy.
• To shape learning to a wider circle of stakeholders and
system changes five Thematic Project, which are
designed to identify, refine and disseminate lessons
learned from the projects and programme, have been
started.
12

13. Structural fund programs could do better to
support innovation, job creation and growth:
• if they work as ”venture capitalist” with ongoing
evaluation of projects and learning processes within and
between the projects.
Thereby, both experiences from processes and knowledge
about ”products” and methods can be gathered.
By participation in public debate insights are spread and
regional development are energized.
13