1. Request for Proposals (RFP)
Evaluation Training
Aviation Procurement Unit
March 26, 2009

2. 2
AGENDA
When to Use an RFP Approach?
Overview of the Evaluation Process?
What are the Expectations for Evaluation Committee Members?
What is the Approach to the Evaluation of Proposals and Scoring Guidelines?
What is the Approach to Evaluating Technical Proposals?
What is the Approach to Evaluating Cost Proposals?
Overview and Steps of Consensus Scoring?
Best Practices for Evaluators
Next Steps
Questions and Answers

3. 3
Best Value Should Always be the
Purpose of the Evaluation Process?
A Best Value Award is one
which optimizes quality,
performance, cost, time, and
efficiency in a fair manner to
the contractor community

4. 4
When is an RFP the Right Approach?
The Bid process is the default solicitation option.
When we are not sure of the best solution and multiple options
exist.
When performance based incentive contracts are used.
When complex factors in addition to price are very important.
When there are complex requirements.
When it is a new program/ project that has not been tested.
When it is desirable to find out if there are alternative options to
what is normally performed.

5. 5
What is the Best Way to Handle
Criteria for RFP Evaluations?
All evaluation criteria MUST be established prior to the
issuance of RFP.
Criteria should be clearly linked to desirable outcomes.
Be clear and concise in language to avoid confusion
Evaluation information should be documented, dated,
and secured throughout the evaluation process.

6. 6
How Do We Develop an Evaluation
Strategy?
Identify (eliminating) Minimum Qualifications
Define Criteria
Develop Specifications
Decide importance of Oral Interviews/ Presentations
Incorporate Reference Checks into Industry Experience
Decide importance of Site Visits
How important is Cost; Cost to Technical Score Ratio?
Determine distribution of points among each criteria

7. 7
What are Examples of Standard
Criteria in a Technical Proposal?
Corporate Experience:
What are the proponents’ past performance and experience in
similar types of contracts?
What is the experience level of the essential subcontractors?
Project Organization and Staffing:
What is the staffing approach?
What are the qualifications and past performance of the personnel?
Implementation Approach:
What is the adequacy of their work breakdown structure/ project
management?
Is it Comprehensive? Reasonable?
Proposed Deliverables?
Preparation Plan?

8. 8
What are Examples of Standard
Criteria in a Technical Proposal?
Operation/ Management Approach:
Plan for Regular/ Daily Operations?
Approach to Quality Control and Assurance?
What are the Commitments to Performance Standards?
What are the Plans for Maintenance?
What are the Contingency procedures in the event of an emergency?
Corporate and General Capacity:
What is the overall capacity of the organization?
Do they have a plan to sufficiently staff the contract project?
How long has the company been operating?
What is the turnover ratio?

9. 9
Overview of the Evaluation Process
Rules and regulations that are applicable to the
evaluation process should be followed throughout the
entire process.
DOP makes a determination of responsiveness and
responsibility based upon minimum qualifications.
APU facilitates and coordinates process between DOP
and evaluators

10. Overview of the Evaluation Process
Evaluation committee members independently
review proposals based upon the specific
evaluation criteria
Technical and cost scores are assigned
applying the designated evaluation criteria
Oral interviews are conducted, if necessary
Negotiations are conducted with the winning
proponent
10

11. 11
What are Standards of Conduct While
Serving as an Evaluator?
No Conflicts of Interest
No solicitation or acceptance of gifts or anything
of value
Integrity and Honesty during scoring and
communications
Proper handling and discretion of all information;
No Disclosure during process
Practice Ethical Behavior

12. 12
What is Expected from
Evaluation Committee Members?
The member has knowledge or experience regarding the RFP Subject
Matter
The member has sufficient time to dedicate in order to effectively
evaluate the proposals.
Such time is dedicated to reading all proposal related materials
extensively
Attending all Evaluation-related Meetings: Training, Evaluation
Sessions, Oral Interviews, Debriefings, Criteria Planning
Conduct an independent review.
Identify potential deficiencies in a proposal that could result in a
determination of non-responsiveness or non-responsibility.
Communicate any issues to the Contracting Officer immediately.
Limit communications regarding the RFP to those DOA and DOP staff
who are directly assigned to the project.

13. 13
What is the Purpose of Oral
Interviews?
Allows proponents to clarify on areas
that may be unclear in their proposals
Possibly reduces the likelihood of
misunderstandings.
Evaluation committee members determine whether or not oral interviews will be necessary.
•Oral interviews should follow group discussions
•Oral interviews should ALWAYS precede scoring.

14. 14
What is the Purpose of Reference
Checks?
Reference Checks may be conducted with prior customers of the
proponent
Reference Checks may also include references of key personnel.
Examples of Questions include:
Were the projects/ deliverables completed on time?
Were there any problems with the management team?
Were the Project Management plans realistic?
Were there any other significant problems?
Did the Prime Contact stay the same or change over the duration of the
project?
Were the Personnel Knowledgeable and Competent?
Would you do business with this vendor/ contractor again?
How would you rate your level of satisfaction from 1-5?
Results of Reference Checks will be distributed to the evaluation team prior to scoring

15. Ensuring Security and Confidentiality
Keep proposals in a secure place.
Evaluators shall not discuss the contents of submitted proposals
or the procurement activities with any persons outside of the
evaluation team or Procurement staff.
Evaluators shall not communicate the scoring outcomes or
content of proposals and shall not disclose the status of any
proposal.
15

16. 16
How are Proposals Evaluated?
All Proposals shall be evaluated in
accordance with the City’s Code of
Ordinances and the criteria specified
on the Percentage Evaluation Form
and considering the information
required to be submitted in each
Proposal.

17. 17
Evaluation Scoring Guidelines
Scoring will be conducted for each criterion on a scale of 1-10
Technical Criteria will be scored as such:
8-10:Oustanding: Proponent’s proposal exceeds expectation and
demonstrates an excellent ability to reach the goals and objectives of
the procurement. Proposal is consistently strong.
5-7: Highly Effective: Proponent’s proposal demonstrates a good
ability to reach the goals and objectives of the procurement.
3-4: Effective: Proponent’s proposal demonstrates a fair ability to
reach the goals and objectives of the procurement.
1-2: Marginally Effective: Proponent’s proposal demonstrates a
minimal ability to reach the goals and objectives of the procurement.

18. 18
Evaluation Scoring Guidelines
Scores will be adjusted to reflect the weight assigned
to each criterion.
Each criterion must be reviewed according to the
requested items of the RFP and scored based on the
scoring definitions above.
Only the Criteria in the Evaluation Form will
receive scores.
Any Proponents Proposal that does not meet the
minimum requirements of the RFP should be identified
as this may deem them non-responsive.

19. 19
When is a Proposal Deemed
Non-Responsive?
Only the Department of Procurement has authority to deem a
proponent non-responsible and/or non-responsive.
Evaluators should notify the DOP Contracting Officer of
deficiencies and/or clarifications needed.
All minimum requirements will be scored on a pass/fail basis.
Failure to pass all mandatory requirements may deem a proposal
nonresponsive
e.g., number of similarly completed projects
Evaluators should promptly notify Procurement staff if there is a
problem with the proposal.

20. What Approach Should We Use to
Evaluate the Technical Proposal?
Minimum requirements should be evaluated on
a pass/fail basis.
Review the established criteria and related
weights.
Evaluate proposals strictly by DOA’s criteria as
defined in the RFP for each proposal.

21. 21
What Should Technical Scoring Look
Like?
Weight Criteria Company 1 Company 2 Company 3
20% Organization/Resumes of
Key Personnel
10 3 1
30% Overall Experience,
Qualifications and
Performance of
Previous Similar Projects
9 7 4
20% Strategic Approach
To Increasing
Federal Assistance to the
City through Government
Agencies and Congress
8 6 3
Only the criteria indicated will receive scores.
Scoring will be conducted on a scale of 1-10
Technical Scoring should reflect the needs of the Proposal

22. 22
What Approach Should We Use to
Evaluate the Cost Proposal?
After technical aspects of proposal responses are initially
reviewed, then cost proposals are reviewed.
Cost proposals may reflect price escalation, deceleration due to
performance, time, CPI, etc. may be applied.
Cost Evaluation Must be independent of the Technical Evaluation.
A Clear Formula should be used to Calculate Cost Scores.
Scores should be directly linked solely to costs proposed on a
formulaic basis.
Cost proposals/ financial offers must be taken at face value when
the cost proposal is evaluated.

23. Examples of Cost Proposals…
Performance Based Contract
RFPs may read like this…..
Please propose Base Costs
and Incentives associated
with performance beyond
mandatory requirements…
E.g., Beyond 90% Customer
Satisfaction Scores
consecutively for one year will
lead to a 3% bonus (e.g..,
janitorial)
Cost Savings Contract RFPs
may read like this…..
Please propose Base Costs and
Incentives associated with cost
savings beyond mandatory
requirements…
E.g., Company X will receive
10% of all cost savings beyond
performance guarantee (e.g..,
workers’ compensation)
23

24. Let’s Take a Look at the Cost Proposal
Scoring…
The lowest fee offer receives the maximum points possible (10) . In the event,
that it is a revenue generating proposal, it would be the highest offer.
Points are distributed based upon percentile in which the fee offer is found in
relationship to the lowest offeror.
Formula Used:
Lowest Offer/Offer at hand= Percentage Relationship to Lowest Offer
24
EXAMPLE Cost Offer Points
Cost Proposal 1 $780k 10 points Lowest Offer
Cost Proposal 2 $1.3 million 6 points 780K/1.3 million= .6
Cost Proposal 3 1.8 million 4.3 points 780k/ 1.8 million= .43

25. 25
Let’s Recap all Scoring Elements…
Percentage Weights Criteria
10 Corporate Experience
15 Project Organization and Staffing
15 Implementation Approach
20 Operation/ Management Approach
10 Corporate and General Capacity
10 Cost/ Financial Offer
15/0 OCC (Minority Participation)
5 Financial Capacity
100 TOTAL

26. 26
How Does Consensus Scoring Work?
Consensus Scoring consists of the
Entire Evaluation Committee
participating in Joint Discussion,
Assessment, and Scoring of
Technical and Cost Criteria in order to
develop one unified score for each
criteria.

27. 27
Team discussion leads to greater sharing of each
other’s Expertise in cross- functional projects, which
are quite common in DOA RFPs
Generates clarification where misunderstandings may
Exist
Ensures that all evaluators are truly engaged in the
Evaluation process
Let’s Review the Benefits of
Consensus Evaluation and Scoring…

28. Let’s Review the Benefits of
Consensus Evaluation and Scoring…
Encourages deeper probing, questioning, discussion,
and analysis of proposals to the evaluation criteria
A unified understanding of responses to all proposals
is reached
Eliminates discrepancies among scores
Overall, the result is a more comprehensive and
accurate assessment of the proposals
28

29. 29
How Should Individual Evaluators Prepare Prior to
the Consensus Evaluation Session?
Review thoroughly and comprehensively all proposal responses,
repeating review of Responses where necessary.
Keep in mind how the information relates specifically to the stated
criteria in the RFP.
Identify all points of information in the responses that need
clarification
Identify the pros and benefits of each proposal response.
Identify the drawbacks, cons, and concerns of each proposal
Response.
Evaluators should not assign scores during independent review of
proposal.

30. 30
What are the Steps to the Consensus
Scoring Evaluation Session?
The Evaluation team continues until all desired elements of discussion are
exhausted by all evaluators.
Each Member indicates points of information where they need clarification and ask
questions to fill such voids, when possible, with other evaluation team members.
Each Member identifies the Pros and Benefits of each proposal.
Each Member identifies the Cons, Costs, and Concerns of each proposal.
Each Member communicates their assessment of the Overall Quality of each
Proposal Response based upon each criteria.
Only after all Proposals are discussed, then each proposal response is scored by
the team based upon all criteria.
The Facilitator records the Final scores for Documentation and Reporting.

31. 31
What is the Role of the Facilitator in
the Consensus Evaluation Session?
Provide Structure and Ensure that all Evaluators Participate and
Contribute to the Process in a Balanced and Generous manner
Monitor and Ensure that an Objective Stance is upheld
throughout the entire Evaluation Process.
Keep the Team Focused and Ensure that Each Criteria is
Discussed by the Entire Team
Remind Evaluators to Focus Strictly on the Evaluation Criteria
Ensure that all Needed Clarification is Obtained
Calculate Final Scores and Rankings

32. Important Documentation Steps for
Consensus RFP Evaluations…
Technical Evaluation matrix (one score for each company)
Cost Evaluation Scores (one score for each company)
Total Final Score Matrix
Evaluation Session Notes by Facilitator
IMPORTANT: The Facilitator/ Procurement Staff takes notes during
the Collaborative Evaluation Session.
32

33. 33
Best Practices in Evaluations
Make every effort to differentiate the quality among proposals reviewed.
RFP Evaluations for each contract are expected to be complete within 10
days.
Keep in mind that you are making a decision that has an impact on
significant dollars, often millions- be dedicated and focused.
Cost proposals should be evaluated strictly based on a formula.
Technical proposals should be evaluated strictly by the already developed
criteria.
Benchmark costs to ensure reasonableness of proposed costs.
Thoroughly read the evaluation criteria.
Attend every scheduled evaluation team meeting and contribute to the
discussion.

34. 34
Now the Recommendation has to be
approved…
The Project Manager and respective AGM writes a
supporting letter for the recommendation by the
Evaluation Committee.
The General Manager sponsors the recommendation
and sends to the Chief Procurement Officer.
City Council and mayor must approve the
recommendation through legislation in order for the
City to enter into the contract with the proponent.
The Contract must be signed by the GM, Law, the
CPO, the Mayor and the Clerk.

35. 35
Questions & Answers
Aviation Contracts Area
Lepora Manigault, Procurement Officer – 404.530.2421
Justina Mann, Contracts Supervisor – 404.530.5408
Diane Akinsipe, Contracting Officer – 404.209.2922
Erin Jenkins, Contracting Officer – 404.209.3171