This document summarizes Colston Sanger's presentation on "Collaborative Evaluation in Multi-Stakeholder Projects" given at the University of Portsmouth. The presentation covered how Sanger began experimenting with teaching agile project management methods in an agile way within university programs. It discussed using real or near real projects, frequent decision points, communication, and forms of authentic assessment. It also covered developing a collaborative approach to evaluation where students and stakeholders help determine what counts as good work. Finally, it discussed taking these lessons back out into practice through further conversations with experienced project managers.

1. Collaborative Evaluation in MultiStakeholder Projects
SBS research seminar
University of Portsmouth
18 May 2011
Colston Sanger
HEA BMAF Teaching Research and Development Award, 2009-10
Faculty Learning and Teaching Fellow 2008-9, 2007-8
sangerc@lsbu.ac.uk

2. Agenda
1.
2.
3.
4.
5.
How it all began
Teaching agile project management in an agile way
Collaborative evaluation
Taking this back out into the „real world‟
Contributions, questions

3. 1.
How it all began …

4. How it all began …

5. Returning to HE

At University of Surrey, 2002-3




Within the BSc WBL and MSc CASS programmes
Working in an experiential, Gestalt-informed way
Self, peer and tutor assessment
At London South Bank University, 2006
Within the MBA programme
The opportunity to develop a Managing Projects unit

A series of funded informal experiments



Faculty Learning and Teaching Fellowships, 2007-8, 2008-9
HEA BMAF Teaching Research and Development Award, 2009-10

6. 2.
Teaching agile project management in an agile way

7. Managing Projects

Managing Projects



Not Project Management
… as a team sport
Requires
involvement, engagement

Working on real or „near real‟
projects

Staying alive to change


Frequent decision points
Communication, simplici
ty, feedback, courage

8. Intended learning outcomes
Knowledge and
understanding


Describe …
Identify similarities,
differences, connections …
Intellectual skills



Evaluate…
Analyse …
Exercise appropriate
judgement …
Practical subject-specific
skills


Develop …
Demonstrate …
Transferable skills




Manage own learning …
Communicate effectively …
Work with others …
Recognise and support
followership, and be
proactive in leadership

9. Calibrating the process of
learning




Learning agreement
Check-in
Weekly project
progress reviews
End of unit project
retrospective

10. Staying alive to change

Frequent decision points

Communication, simplicity, feedback,
courage

And also forms of congruent or
„authentic‟ assessment

11. Near real projects

You are members of the Special
Projects team at the London Borough
of Cross River

Your Chief Executive had a GREAT
IDEA …

A summer Street Ski Jump Festival!
Ski jump
San Francisco 29/9/05

12. A summer ski festival

13. The great glass elevator

Landmark urban
regeneration
The “Elevator”

A solution to the inner
London housing crisis

Unlimited student
residences for Cross
River University…
Castle House,
Elephant & Castle

14. A London village fête?

15. 2010-11 Project Challenge

16. End of unit project
retrospective

17. 3.
Collaborative evaluation

18. What counts as good work?
„[In] the learning that
professionals do outside the
academy, learning outcomes are
rarely specified in explicit terms.
What is required of the learner is
embedded in a professional
practice…
Before learning can even
commence there is a need
for learners to identify for
themselves what they need
to learn, taking into account
a range of contextual
factors, and to judge what
counts as good work.’
(Boud and Falchikov, 2006)

19. What counts as good work?
Scaling
1.
Stakeholder Engagement

2.
Critical Path

3.
Doesn’t yet know what is required / Knows what is required / Has estimated when and
how much /Resources acquired for use
Added Value

5.
Didn’t know there was such a thing / Heard of it / Worked it out for this project / Know
where you are on/off it / Always on track
Investigation of resources

4.
Not yet identified / Identified / Contacted / Agreed / Engaged
Agreement fulfilled to spec / Skills development for participants / Gained Council and
employer support / National media recognition / Community takes over running of
event, social regeneration
Persuasiveness of presentation

Boring, mumbled, did not hold my attention at all / Confused, chaotic /
Understandable, but doesn’t add up / Coherent, but missing ‘So what? / Lively,
interesting, presents a compelling case

20. 4.
Taking this back out into the „real world‟

21. Well then… What now?

Collaborative
evaluation in multistakeholder projects

Not multi-criteria
decision analysis

Exploratory, grounded
theory approach


Not even sure what the
question is yet
Conversations with
experienced programme
and project managers

22. Some preliminary
data points
A good project is when you have a good
plan. Good in the sense that it takes into
account risks, resources, schedules, costs.
And you also have all the prelims:
stakeholder engagement, champion
engagement, you train all your staff,
everybody's working as a team, singing
from the same song sheet, all those sorts
of things…
Setting scope in an R&D project: When it's that kind of
project, you're basically hired to set the scope: to do the
background investigation, look at the needs, identify the
stakeholders, work out their influences. And then with their
input, the client's input and maybe a supplier input, you can
then define the scope. In that respect, that is a project in its
own right - and whether you get to do the project itself or
someone else takes it on, that's a different matter.
Good achievement comes … when
everybody understands what you are
trying to do and they contribute to that to
make it happen. There's always a part of
luck, but I think you make your own luck.
And that's where the effort comes in,
because if the effort is put in properly then
things should go forward moderately well.
Balancing effort and output in multi-project
scenarios. Effort is basically costing money.
Outputs are added-value benefits. So it's really a
cost-benefit analysis in a strange sort of way. The
project manager is saying how does the client get
the best deal they can get ... It's about allocating
resources fairly. The reality is that whoever is
screaming loudest gets the most response.
Major disconnect between the textbooks
and reality. The textbook will say: block your
time out … But the reality is, nothing works in
those smooth ways... Or [it assumes] you're
doing projects with reasonable people whose
time blocks can be compromised to yours.
It's a 'perfect world' situation: but the reality
is, people do their damndest to mesh with
your timetable, but it's not [always] going to
work.
Benefits realisation, post
project:
Generally, no-one in the
industries I work in tend to
pay me for post-project
monitoring. Therefore my
hands are basically free as
soon as the last invoice
has been paid.

23. Contributions, questions