Slides from my talk to APM South Wales and West of England, in Bristol, 20 Feb 2014. Why do projects fail? What sort of failures are OK? How can portfolio and programme managers improve their success rates? How do you set up an effective assurance programme? What is the PMO's role in all this?
1. Supporting Reviews & Assurance
The Role of the PMO
Supporting Reviews & Assurance
APM, Feb 2014
1
2. Outline
ïź Projects fail
ïź Thatâs OK
ïź Portfolio mgt is easier than project mgtâŠ
ïź âŠ provided you can get the right information
ïź Setting up an assurance programme
Supporting Reviews & Assurance
APM, Feb 2014
2
3. Why do projects fail?
Supporting Reviews & Assurance
APM, Feb 2014
3
Simon Schoeters
4. We lose touch with reality
Supporting Reviews & Assurance
APM, Feb 2014
4
Christiaan Triebert
5. Failure becomes obvious
when we run back into reality
Supporting Reviews & Assurance
APM, Feb 2014
5
Simon Schoeters
11. But they often donât have the
information they need
Supporting Reviews & Assurance
APM, Feb 2014
11
IsaacMao
12. But they often donât have the
information they need
Supporting Reviews & Assurance
APM, Feb 2014
12
derekGavey
13. But they often donât have the
information they need
Supporting Reviews & Assurance
APM, Feb 2014
13
maveric2003
14. But they often donât have the
information they need
Supporting Reviews & Assurance
APM, Feb 2014
14
Jeroen_bennink
15. Thatâs where the PMO helps â
providing clear information
Supporting Reviews & Assurance
APM, Feb 2014
15
Grand Canyon NPS
16. Right Information â Skills â
Process â Stakeholders
Supporting Reviews & Assurance
APM, Feb 2014
IvanWalsh.com
16
Kheel
Centre, Cornell
17. The role of assurance is to
provide validated information
Supporting Reviews & Assurance
APM, Feb 2014
17
owenwbrown
18. PMOs can work at three levels
Supporting Reviews & Assurance
APM, Feb 2014
18
ell brown
19. How do we set up an effective
programme of reviews?
Supporting Reviews & Assurance
APM, Feb 2014
19
Colin_K
20. 1) Make reviews the norm
Supporting Reviews & Assurance
APM, Feb 2014
20
Lars P.
21. 2) Be clear who youâre serving
Supporting Reviews & Assurance
APM, Feb 2014
21
Bird Brian
22. 3) Tune investment to risk
levels
Supporting Reviews & Assurance
APM, Feb 2014
22
digitalmoneyworld
23. 4) Use a mix of review types
Supporting Reviews & Assurance
APM, Feb 2014
23
vie_ascenseur
24. 5) Negotiate clear objectives
for each review
Supporting Reviews & Assurance
APM, Feb 2014
24
Richard_of_England
25. Control parameters
Baseline
Criteria
Reference
Models
Review execution
Inputs
Artefacts & other
items to review, plus
supporting details.
Outputs
Analysis Loop
Go / No -go
decision.
6) Run a structured process
Recommendations to
improve review artefacts
Supporting Reviews & Assurance
APM, Feb 2014
Feedback
to improve
reference
models
25
Improved
artefacts.
26. 7) Focus on evidence
Supporting Reviews & Assurance
APM, Feb 2014
26
Simon Schoeters
34. Summary
ïź We engage with risks to achieve rewards
ïź Sometimes the risks win
ïź Aim is to eliminate unnecessary failure
ïź Program & portfolio mgrs have a lot of tools
ïź They need information in order to use them
ïź Reviews & assurance provide this information
ïź PMOs can do a lot to set up effective reviews
Supporting Reviews & Assurance
APM, Feb 2014
34
35. Unnecessary FailureâŠ
⊠happens when people with the skills,
resources and authority to act effectively
donât get full, validated information about
project status and issues.
The role of reviews and assurance is to
provide this information.
Supporting Reviews & Assurance
APM, Feb 2014
35
37. Graham Oakes Ltd
ïź Making sense of technologyâŠ
ï± Many organisations are caught up in the
complexity of technology and systems.
ï± This complexity may be inherent to the
technology itself. It may be created by the pace of technology change. Or it may arise from
the surrounding process, people and governance structures.
ï± We help untangle this complexity and define business strategies that both can be
implemented and will be adopted by people throughout the organisation and its partner
network. We then help assure delivery of implementation projects.
ïź ClientsâŠ
ï±
ï±
ï±
ï±
ï±
ï±
ï±
ï±
ï±
ï±
ï±
Cisco Worldwide Education â Architecture and research for e-learning and educational systems
Council of Europe â Systems for monitoring compliance with international treaties; e-learning systems
Dover Harbour Board â Systems and architecture review
Greenpeace â Project reviews, enterprise architecture, digital strategy
MessageLabs â Architecture and assurance for partner management portal
National Savings & Investments â Helped NS&I and BPO partner develop joint IS strategy
The Open University â Enterprise architecture, CRM and product development strategies
Oxfam â Content management, CRM, e-Commerce
Thames Valley Police â Internet Consultancy
Sony Computer Entertainment â Global process definition
Amnesty International, Endemol, Intel, tsoosayLabs, Vodafone, âŠ
Supporting Reviews & Assurance
APM, Feb 2014
37
Hinweis der Redaktion
Can give a long list â fuzzy requirements, mismatched goals, lack of sponsorship and stakeholder buy-in, lack of resources & skills, poor communications.Weâve known about these for decades. The literature keeps reiterating them. Yet our projects keep failing. Thereâs something going on beneath thisâŠ
We get overloaded â too much going on, too much info coming in, to keep on top of it all. We miss stuff, make bad connections, donât hear warnings from team members, etc.Weâre subject to biases and gaps in our cognition (Daniel Kahneman: Thinking, Fast & Slow)We fool ourselves. We hope weâll recover.We create incentives to hide failure.Most project mgt is about building mechanisms to keep in touch with reality â status reports, plans (make it more obvious when weâre deviating), earned value, risk mgt (triggers tell us if risk has come into play), etc.
Eventually we run back into reality. We have to hit it at some point, when we need to deliver into outside world. Can no longer fool ourselves and failure becomes obvious.Great statistics from my time at Psygnosis, showing how people face up to failure as they start to hit major milestonesKey message: failure typically happens a long time before it becomes obvious.This is the real problem: now have less time to deal with the failure, and itâs more likely to have grown and cascaded, causing related failures.
Projects are one-ff activities. Level of novelty & research & learning. Engage with risks to achieve rewards. Sometimes the risks win.Oil industry is great example.
Inherent = built into the statistics of what weâre trying to do. Some percentage will fail by virtue of complexity, unknown nature, etc. That percentage varies from domain to domain â high in research, low in stuff we do over and over again with little variation. Whatever level, itâs not fundamentally a problem â a sign weâre attending to meaningful risk/reward trade-offs.Unnecessary = failure that happens because we donât get the basics right. We donât plan. We donât communicate. We donât resource appropriately. Etc. Avoidable failure that we donât avoid. This is the real problem.
Only have one project. Either it succeeds or it doesnât. You tend to get tarred with the failure â a real problem in orgs with blaming or risk-avoiding cultures.
Can achieve higher level goal provided the proportion of failure is within tolerance. Still want to eliminate the unnecessary failures, but can live with the inherent ones (& have tools to mitigate these too)Portfolio management, esp, is all about getting a balanced level of risk, not zero risk.
Can shift resources between projectsTypically have larger pool of expertise â can access specialists that arenât fully available to project managersCan adjust schedules - delay one in order accelerate othersTypically have better access to senior stakeholders â can work with them to remove blockages, add resources, adjust scope & objectives
4 slides:Thereâs a lot going on â hard to keep track of it all; significant stuff gets lost in amongst all the detail and the sheer weight of info hitting usStatus reports paint a rosy view â project managers may have fooled themselves. Optimism is a human bias & esp in project managers â convince themselves there chances of success are better than average. Even if aware of issues, donât want to share failure due to machismo & independence, or for fear of being pushed aside (valid fear: many senior managers micro-manage & intervene too early).Status reports become ritualised â lots of cut and paste. Become an overhead â donât add value for project manager and team, so they put minimal effort into themPeople have agendas behind the info they reveal and the view they have of the project
4 slides:Thereâs a lot going on â hard to keep track of it all; significant stuff gets lost in amongst all the detail and the sheer weight of info hitting usStatus reports paint a rosy view â project managers may have fooled themselves. Optimism is a human bias & esp in project managers â convince themselves there chances of success are better than average. Even if aware of issues, donât want to share failure due to machismo & independence, or for fear of being pushed aside (valid fear: many senior managers micro-manage & intervene too early).Status reports become ritualised â lots of cut and paste. Become an overhead â donât add value for project manager and team, so they put minimal effort into themPeople have agendas behind the info they reveal and the view they have of the project
4 slides:Thereâs a lot going on â hard to keep track of it all; significant stuff gets lost in amongst all the detail and the sheer weight of info hitting usStatus reports paint a rosy view â project managers may have fooled themselves. Optimism is a human bias & esp in project managers â convince themselves there chances of success are better than average. Even if aware of issues, donât want to share failure due to machismo & independence, or for fear of being pushed aside (valid fear: many senior managers micro-manage & intervene too early).Status reports become ritualised â lots of cut and paste. Become an overhead â donât add value for project manager and team, so they put minimal effort into themPeople have agendas behind the info they reveal and the view they have of the project
4 slides:Thereâs a lot going on â hard to keep track of it all; significant stuff gets lost in amongst all the detail and the sheer weight of info hitting usStatus reports paint a rosy view â project managers may have fooled themselves. Optimism is a human bias & esp in project managers â convince themselves there chances of success are better than average. Even if aware of issues, donât want to share failure due to machismo & independence, or for fear of being pushed aside (valid fear: many senior managers micro-manage & intervene too early).Status reports become ritualised â lots of cut and paste. Become an overhead â donât add value for project manager and team, so they put minimal effort into themPeople have agendas behind the info they reveal and the view they have of the project
Role of the PMO is to help people make good decisions. In particular, it ensures they have info they need, appropriate processes, necessary skills, right stakeholders, etc, to be able to make good decisions.
Role of the PMO is to help people make good decisions. In particular, it ensures they have info they need, appropriate processes, necessary skills, right stakeholders, etc, to be able to make good decisions.
Assurance supports this info-centric view of the PMO. Ensures availability of clean, validated information about status of projects, risks, etc, to stakeholders at all levels. Enables decisions can be grounded in realistic understanding of the project / programme / portfolio â itâs current state, risks, team capabilities, etc.Exec level â help shape portfolio, redirect resources, identify & manage dependencies, etc â PMO is key stakeholder here, as are typically supporting execsProject mgr level â help see what is really going on & hence identify strategies to intervene effectively â PMO is less of a stakeholder here; may be direct to ProjMgr (but PMO still supports assurance â success is common goal)3 key points about assurance & reviews:Provide independent, validated information so people can intervene effectively in their projects, programmes & portfoliosProvide outside perspective, helping cut through many of the cognitive gaps, biases and politicsStructured approach also helps overcome cognitive gaps = review approach
PMO ensures info gets to the place where it can be acted on most effectively. Tends to work at 3 levels:Executive: Shape of portfolio; Interdependencies & side-effects; gaps; opportunities; risks; resource availability & allocationManagement: What works / what doesnât (Methods, tools, lessons learned); What canât I see because Iâve got my nose to the grindstoneAdministrative: Status, statistics, RAGs, Risk Registers, etc â single view of âtruthâSo natural home for methodology, etc. May be home for project managers, career development, etc.If info flow within org were perfect, wouldnât need a PMO â PMO is a driver to continuously improveâŠ
A dozen pointers for you.
If you only review projects when theyâre in trouble, sending a review gives message âyour projectâs in troubleâ. Project mgrs resist this message â can be pretty threatening. Means theyâre less willing to open up about whatâs going on. Cuts off the information flow.Unexpected reviews add overheads â people need to prepare materials, reschedule meetings, make space available, etc. Disruption is much lower if reviews are planned into project from outset.Ad hoc reviews also tend to be more heavyweight â happening infrequently, so need ot drill down into details & ask a lot of questions. Canât do much in a lightweight review at this point â barely get up to speed. Adds to disruption.PMO does a lot to set the context for this â sets expectation that reviews will be norm, builds it into methods & tools & templates
Can be project mgr, programme/portfolio mgr, or senior exec â as per earlier discussion of their into needs â links to their âsphere of influenceâ & capability to actIf not clear, will deliver wrong info (too detailed, not actionable with resources and power they have, etc)Even if focused on needs of sponsor or exec, need to try to deliver value to project team â rely on them for interviews, access to docs, etc. Give value in return for this disruption â draft recommendations that they can use, benefiting from the experience and perspective of the reviewers. (Doesnât help to be condescending as do this!)Also need to get the communication protocols around this right, e.g. team will hide info if see no benefit for themselves and/or see fear of over-intervention by senior mgrs. E.g. Psygnosis protocol.NB thereâs a risk here â assurance should be a redundant info channel, supporting & not supplanting info flows from project mgr & team. Thereâs a risk that these normal flows wither as people become too dependent on the flow from the assurance teams.PMOâs help frame objectives for reviews & hence set this up
People ask how much to invest? Hard question â we lack good statistics.Tune it to the expected failure rate in your org â if projects overrun by an average of 40%, then what percent of budget is it worth spending to halve this? That puts a ceiling â start lower and work up to it if getting results.Consider film industry â completion bond costs 2-5% of production budget. Reasonable to spend same on IT project.PMO again helps set the conditions for this discussion â assess overall risk levels
Frequent, regular lightweight vs larger, milestone-based gate reviews (latter are more disruptive & require more effort from project & review teams, but go deeper, heavyweight means generally too infrequent to spot trends until too late; former are good for spotting trends; combining the two makes a lot of sense)Formal vs informalSelf-assessmentvs peer vs independent outsiderProcess vs outputsProject, design, code, ⊠- all depend on the objectivesPMO helps design the review programme
Canât be both broad and deep (within bounded time & budget) â either skim across entire scope of project, or delve deeply into 1 or two aspectsOften good to go T-shaped â broad initial phase, then drill into 1 or 2 areas selected by first phaseWork with stakeholders (exec, sponsor, project mgr, team) to agree what areas youâre going to focus on & how deep youâll go â will influence time needed, people youâll need to talk to, etcJust like a project really â time spent definign clear objectives will repay itself during the reviewPMO helps facilitate the negotiation
Quickly walk through the processPMO incorporates this process into body of tools, methods, etc, it manages
Everyone has opinions. May be driven by evidence, but also by prejudice, bias & agendas. Need to cut through this â goal is to deliver info thatâs validated by links to facts.Gets very political if canât back findings with evidence â comes down to contest of credibility, political power, etc. Can be very messy. Not my idea of fun.Look for dates, metrics, photos. Try to get attributable quotes from people. When making an inference, be clear that it is an inference, and link back to the facts supporting it. Likewise link recommendations to your inferences of whatâs going on & hence back to the underlying facts.Can take time to gather clear, documented facts. Not essential if relationships are good and findings are uncontentious, but definitely essential in political environments or a lot is at stake.
Military saying: âAmateurs discuss tactics. Professionals discuss logistics.âReview requires a lot of organising â getting docs, getting security passes, getting access to systems, booking meeting rooms, scheduling meetingsEsp time-consuming for external review teamsIf you donât plan for the time this takes, your review will be behind schedule before it even starts â ends up wasting even more of the project teamâs time.A lot of this is about reducing load on the project team â review often isnât focused to help them, so need to work carefully to keep them onsidePMO often best placed to provide administrative support, access to systems & tools & docs, etc, to minimise burden on project & review teams
Project teams are busy â none of them have spare time. Review is getting in way of delivery activities (even if it does end up adding value through perspective and expertise of reviewers).Work from existing project artefacts â donât ask team to create special docs, fill in questionnaires & checklists, etc just for the review. (Planning reviews into project helps ensure the artefacts will be there.)Reviewers need to understand the standard docs produced within this org (if defined), so can get up to speed quickly & not disrupt project team.Plan logistics so as to minimise disruptionTreat people with respect â turn up for interviews on time, read project docs in advance so know the basic context, donât overrun, thank them for their timePMO is managing the environment for this (logistics, review programme shape, etc)
Cover:Orgâs project methods, tools, artefactsTypes of review, review processConducting interviewsGathering evidence & building a chain of inferenceWriting and presenting findings so theyâre well-formed, actionable, concisePMO is natural place to provide this training, and to manage the pool of skilled (& apprentice) reviewers â build review capability within the org
Itâs all largely a waste of time if donât act. (Some benefit from regularly shining light on project: people tend to maintain risk registers, etc, rather than let them lapse.)Review & project teams lose heart iif donât see action from reviews â stop taking them seriously / start cutting cornersNB reviewers need to deliver actionable findings â backed by evidence; within authority, skills & resources of recipient; aligned to wider goals and politics. Lot of review teams shoot themselves in the foot here â requires experienced, savvy people who are well connected to organisational context and/or have appropriate senior backing.PMO may drive action at portfolio level; monitor action at project level â ensure findings from reviews are being acted on, track those actions
Are reviews happeningAre findings being acted onAre projects delivering better as a resultAgain, this is an area where PMO gets heavily involved â monitoring stats about reviews, tracking project-level actions after review team has left, managing portfolio-level actions
For me, this is the really big benefit of reviews â bunch of people see whatâs working & not working across range of projects in org; talking to people in different projects. Natural channel to disseminate news about what is & isnât working. Build this into their operations.Can do this tacitly (through discussions that happen as part of reviews) and explicitly (by adding items to review checklists, updating tools & methods & standard doc templates & etc).Also a way to build skills â (1) give people a chance to see a wide range of projects; (2) give project mgrs a chance to talk to experienced PMs running reviews.Iâd give this at least as much emphasis as the communications side.This again is very close to the PMOâs mission: they typically own the tools, methods, templates, etc, and are responsible for building PM skills and capabilities within org.
But comms is the prime purpose â ensuring we get clean info to the stakeholders who can act on it.Come back to the risk I identified at point (2) â Assurance is a way of supporting effective stakeholder communications â ensures that people get the info they need to act effectively. But it shouldnât replace the normal flows â it supports, doesnât supplant. If people getting too dependent on assurance for info, then you have a problem within your project management & teams that needs to be addressed head on.
Add a point about PMO â naturally placed to help set them up & maintain them
Who I amIndependent consultantDo 2 things â help set up project (untangle complexity); help keep in touch with whatâs going onUnusual perspective on assurancePortfolio of mid-size projects rather than single large programmeDifferent twists, but aligns to where many organisations are at, so will share experienceAgenda