Presentation at the Ontario Institute of PMAC conference in Niagara Falls. Co-presented with Leopold Koff (Conestoga College). Material also supplied by Kim Dooling (Town of Oakville). Presentation brought to you by QCsolver.ca

1. 2012 Annual Conference Presentation
Supply Chain Risk and
Performance Monitoring:
The use of automated systems to
help you manage key areas of your
organization

2. Which sector are you in?
Pick one!
1. Municipal
2. Academic 31
(University/College)
3. School Board
4. Health Care
5. Broader Public Sector
6. Private
3
7. Federal/Provincial 2
0 0
1 1
1 2 3 4 5 6 7

4. How many Canadian employees work for your
organization? Pick only one!
1. 1 to 9 12
2. 10 to 24
3. 25 to 99
8
4. 100 to 249 7
5. 250 to 499
6. 500 to 999 4
3 3
7. 1000 to 2499 2
8. 2500 + 0
1 2 3 4 5 6 7 8

5. Other than OIPMAC which
Organization(s) do you belong to?
Pick as many as you would like!
1. OPBA 24
2. CPPC
3. NIGP
4. OECM
5. CMC
6. HSCN 8
7. Construction Association 5
8. ISM 4
9. Other 0 0 0 0 0
1
10. None 1 2 3 4 5 6 7 8 9 10

6. Survey Says!
OPBA = Ontario Public Buyers Association
CPPC = Canadian Public Procurement Council
NIGP = National Institute of Governmental Purchasing
OECM = Ontario Educational Collaborative Marketplace
CMC = Canadian Association of Management Consultants
HSCN= Healthcare Supply Chain Network

7. How many competitive bids did your
group issue in 2011?
20
1. 1 to 49
2. 50 to 99
3. 100 to 149
9
4. 150 to 199
5. Greater than 200
1
0 0
1 2 3 4 5

8. How many competitive bids did the
Town of Oakville issue in 2011?
180

9. On average how many vendor performance
evaluations do you do per year?
Pick only one!
22
1. None
2. 1 to 10
3. 11 to 25 6
4. 26 to 49 3 3
2
5. 50+
1 2 3 4 5

10. Survey Says!

11. Which tools or methods do you use?
Pick as many as you would like!
23
1. Checklists
20
2. Forms
17 17
3. Templates
4. User Guides & Manuals 11
5. Progress meetings
7
6. Performance 6
documentation
7. 3rd Party verification
1 2 3 4 5 6 7

12. Which tools or methods do you use?

13. What triggers an evaluation?
1. Periodic 19
18
2. Budget Threshold
3. Contract performance 15
4. Project profile 13
5. New vendor
6. Project complexity
7. Client Dissatisfaction
6 6
8. Repeat vendors 5
9. Length of Engagement 3 3
2
10. Every Project
1 2 3 4 5 6 7 8 9 10

14. Survey Says!

15. Why Measure Performance?
• Support better decision making
• Support better communication
• Provide performance feedback
• Motivate and direct behavior
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 15
Professor, Supply Chain & Operations Management
School of Business & Hospitality

16. Problems with Purchasing and Supply
Chain Measurement and Evaluation
• Too much data
• Wrong data
• Measurements that are short-term focused
• Lack of detail
• Drive the wrong performance
• Measures behavior vs. accomplishments
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 16
Professor, Supply Chain & Operations Management
School of Business & Hospitality

17. Performance Measurement
• Effectiveness
– Extent to which, by choosing a certain course of
action, management can meet a previously
established goal or standard
• Efficiency
– Relationship between planned and actual
sacrifices made to realize a previously agreed-
upon goal
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 17
Professor, Supply Chain & Operations Management
School of Business & Hospitality

18. Metrics Key
• SPECIFIC
• MEASURABLE
• ATTAINABLE
• RESULTS ORIENTED
• TIME-BASED
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 18
Professor, Supply Chain & Operations Management
School of Business & Hospitality

19. Develop Specific Measures
• Objectivity • Directly related to
• Clarity organizational
• Use of accurate and objectives
available data • Joint participation
• Creativity • Dynamic over time
• Non-manipulative
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 19
Professor, Supply Chain & Operations Management
School of Business & Hospitality

20. Measurement System Characteristics
• Effective measurement requires consistent and reliable
data
• Periodic review of measurement system to …
– Eliminate unimportant or unnecessary measurements
– Add new criteria as required
– Reevaluate objectives or targets
– Objective versus Subjective criteria
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 20
Professor, Supply Chain & Operations Management
School of Business & Hospitality

21. Measurement System Characteristics
• Measurement is not free
• Not all aspects of performance lend themselves to
quantitative measurement
• Purchasing and supply management is better served
by a few precisely defined and thoroughly
understood measurements
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 21
Professor, Supply Chain & Operations Management
School of Business & Hospitality

22. Measurement and Reporting Frequency
• Reporting frequency to buyer
– Day-to-day performance for troubleshooting and
expediting
• Reporting frequency to supplier
– Routinely summarized monthly or quarterly
– Annual face-to-face meeting
• Never delay reporting supplier’s poor performance
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 22
Professor, Supply Chain & Operations Management
School of Business & Hospitality

23. Uses of Measurement Data
• Identify poor performing suppliers
• Support supply base optimization and rationalization
efforts
• Determine future purchase volume allocations
• Identify performance improvement opportunities
• Make sourcing decisions
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 23
Professor, Supply Chain & Operations Management
School of Business & Hospitality

24. Supplier Performance Management
• How does buyer know how well any given supplier is
performing?
• Consists of methods and systems to collect and
provide information to measure, rate, or rank
ongoing supplier performance
• Acts as supplier “report card”
Leopold Koff, B.A., B.Ed., CPSM, CPM, CSCMP 24
Professor, Supply Chain & Operations Management
School of Business & Hospitality

25. Working with Contractors
WHAT IS THE PURPOSE OF A CONTRACTOR PERFORMANCE
EVALUATION?
• To ensure that contracts stay on track
• To deal with issues and problems as they arise
• To develop a relationship with contractors and an
understanding of what is expected
• To document poor performance to alleviate the never
ending cycle
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

26. Steps to Good Performance Management
• Establish and implement procedures for
review of performance problems and/or
disputes
• Have a consistent ranking system and
procedure – follow it!
• Conduct regular inspections-project managers
• Have written report, minutes, notifications
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

27. Documenting Performance
• The MOST crucial part of performance management
• Weekly site meetings – up date the contractor on
upcoming issues or problems that have occurred.
Get feedback from the contractor on how he feels
the contract is going – take his suggestions to heart.
• Distribute site meeting minutes as soon after the
meeting as possible
• Ask the contractor to note any exception to the
minutes in an email or memo.
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

28. Documenting Performance
• Any performance issues MUST be discussed with the
contractor
• Contractor give reasonable time to correct the issue
• DOCUMENT all issues – copy contractor on all
documentation
• Suggestion: Two part report form – initialed by
contractor
• Never put yourself in a position where the contractor
can say, “I had no idea!”
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

29. Types of Documentation
• Contract Check List
– Details deliverables and project milestones
• Observation Record
– Document observations of Contractor Performance
• Complaint Record
– To document and follow up on complaints
• Discrepancy Record
– To note contract discrepancy, copy to contractor with expected
actions.
• Field Diary
– Used to keep track of daily records – be careful what you write.
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

30. Types of Documentation
• Summary Evaluation Report
– This is the contractors formal evaluation.
– Conducted annually and/or upon completion
– Reports can be done more frequently for
probationary or problem contractors
– All notes and documentations are attached
– Meet with contractor to discuss final outcome
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

31. Contract Management
• Be flexible
• Exercise sound judgement
Contracts are not held together by forms, surveys, diaries or
other monitoring tools – but by the contract team!
Effective contractor monitoring depends more on the skills of
the MONITORS than the monitoring methods
• A delicate balance of time and money vs the
potential for disruption a service delivery.
Kim Dooling, CPPB
Manager, Purchasing and Risk Management
Financial Planning

32. Examples of Automated Systems
RAQS
TQRDC
VPM

34. On a scale from 1 to 5 what is the average
score you give to your contractors when you
do evaluations?
17
1. Poor 12
2. Acceptable
3. Standard
4. Good 3
2
5. Excellent 0
1 2 3 4 5

35. Central tendency State of Florida
Overall Rating
70%
66%
5105 Surveys conducted in April 2010
60%
50%
40%
31% Overall Rating
30%
20%
10%
3%
0%
>= 1.00 to <= 2.80 >= 2.81 to <= 3.20 >= 3.21 to <= 5.00

37. Ministry of Transportation Ontario
MTO uses the Registry, Appraisal & Qualification
System (RAQS) to manage vendor qualification and
vendor performance for infrastructure procurement
RAQS is an integrated solution with support for:
• MTO user roles
• Process workflows
• Internal system interaction
• Internal and external communications

38. RAQS Key Functional Areas
– Vendor management
• Vendor registration & qualification
• Vendor performance assessment
• Vendor rating
– Project tendering
• Project notice advertising
• Document distribution
• Contractor communications
– Bid results and evaluation
• Results evaluation / trend analysis
• Awards

39. Simplified 3M team rating (TQRDC)
T – Technology
Q – Quality
R – Responsiveness
D – Delivery
C – Cost
Courtesy: Jeff van Geel
Manager Sourcing & Accounts Payable

40. Courtesy: Jeff van Geel
Manager Sourcing & Accounts Payable

41. Simplified Scorecard
1. Focus on improvement
2. Present facets of performance without
aggregation. Do not dilute message.
3. Communicate dashboard – detail in
meeting
4. Scope: all material & services
Courtesy: Jeff van Geel
Manager Sourcing & Accounts Payable

42. All information for exercise only
Courtesy: Jeff van Geel
Manager Sourcing & Accounts Payable

43. 1. Construction – Commodity - Contract
2. Contractor - Consultant
3. Consistency - Customizable
4. Crowdsourcing – Collaborative - Comments
5. Compare - Contrast
6. Credible – Context
7. Cost

44. ABC Contracting Grand River Ford’s
Excavation Landscaping

45. • For ratings
that are
“Below
Standard”
and
explanation
must be
entered
Easy to create, fill out and explain!

46. Easy colour
coding to
spot what
needs to be
reviewed
quickly.
Data can be
exported for
further
calculations/
analysis if
required.

47. 1. Effective communication
throughout engagement 25%
2. Quality of resources
3. Availability of resources to carry
out contract 17%
16%
4. Quality of the final deliverables 15%
5. Providing value added services
10%
6. Maintaining timelines/deadlines 8%
7. Budget/cost control
5%
4%
8. Having a vendor contact for
dispute resolution
1 2 3 4 5 6 7 8

48. Survey Says

49. What are your greatest challenges to
having an effective VPM! 30
Pick more than one!
25
1. Not enough time
2. End user input difficult to
15 15
get
3. Poor systems
9
4. Not required
5. Missing linkage between 4
contract and performance
6. Comparative Consistency
1 2 3 4 5 6

50. Do you share performance results with
contractors - consultants?
27
1. Yes
2. No
9
1 2

51. Do you tie performance management data
to proposal evaluations and selection?
23
1. Yes
2. No
9
1 2

52. 52

53. Gerald Ford CSCMP Leopold Koff CSCMP
grford@csolutions.org
226-474-1169 lkoff@conestogac.on.ca
905-296-4003
Kim Dooling CPPB
KDooling@oakville.ca
Corporate sponsors 53