Presentation by Tony Limas of Granite Construction titled "Quantifying Risk of End Result Specifications," delivered at the California Asphalt Pavement Association (CalAPA) Spring Asphalt Pavement Conference April 25-26, 2018 in Ontario, CA.

1. 1
Quantifying Risk of End Result
Specifications
CalAPA Spring Conference
April 25 - 26, 2018
Ontario, CA
Tony Limas
Granite Construction Inc.

2. Today’s Discussion
2
Understanding Risk(s)
Evolution of Specifications
Tools for Measuring Risk
Determining Risk
Risk Evaluation Examples
Risk vs. Number of Observations
Quiz
Questions…

3. Statistics – Ugh…
3

4. Understanding Risk(s)
4

5. Types of Risks
Buyer’s Risk β = Risk of Accepting “Bad” Material
Seller’s Risk α = Risk of Rejecting “Good” Material
5

6. Sellers Risk α
When specifications contain unreasonable or
unattainable material tolerances it is likely that
a contractor, providing a product using all the
care and skill normally exercised within the
industry, will fail to meet the specified
acceptance requirements. Such specifications
are said to be unbalanced assigning excessive
risk to the contractor (seller) and thus not
suitable for use.
FHWA - NHI Course No. 13442
6

7. Buyers Risk β
7
The probability that an acceptance plan will
erroneously fully accept 100 percent of
rejectable quality level (RQL) material or
construction with respect to a single
acceptance quality characteristic. It is the
risk the highway agency takes in having
RQL material or construction (products)
fully accepted.
FHWA

8. Evolution of Specifications
Pre 1958
 Acceptance Primarily based on Inspections vs Test Results
 Specification Tolerances Primarily based on Anecdotal or
shoot from the hip subjective observations
 Contractors Struggled to Meet Acceptance Limits
 In many instances acceptable material was rejected and
rejectable material was accepted
 Post 1958
 1958 AASHTO Road Test Collected “Real Time” Test Data
 Material and Construction Properties 8

9. Evolution of Specifications (con’t)
AASHTO Road Test Outcomes
Specifications Must Recognize Total Variability of
Materials and Construction Properties (Standard
Deviation)
Specifications Must Apply Reasonable Risk to the Buyer
and Seller
Seller’s Risk (α) should not exceed 5.0% (FHWA)
9

10. 10
Are Risks
Determine Variability
of Material or
Construction Property
Modify Spec limits,
Sample Size
and/or lot Size
Finalize Specifications
No
YesAcceptable
Yes
Specification
Development
Process
Propose Spec Limits
Examine Seller’s Risk

11. Measuring Risk
11

12. Tools
Standard Deviation
Is the measure of dispersion of a set of data from its mean. It measures
the absolute variability of a distribution; the higher the dispersion or spread of
data about it’s mean, the greater is the standard deviation.
12
= S
= s

13. Total Variability
Variability = variability + variability + variability
(sampling) (testing) (mat./const.)
S2
QC/QA = S2
s + S2
t + S2
m/c

14. Measuring Risk (con’t)
Transportation Related Material Properties Are:
 Symmetrically/Normally Distributed About the Mean
Mean
Test ResultsTest Results
14

15. Measuring Risk (con’t)No.ofSamples
15
Mean

16. Material Distribution Sellers Risk (α)No.ofSamples
-3s -2s -1s +1s +2s +3s
68%
95%
99.7%
16%
16
PWL = 68% α = 32%
NTS

17. Material Distribution Sellers Risk (α)No.ofSamples
-3s -2s -1s +1s +2s +3s
68%
95%
99.7%
α = 5%
2.5%
17
PWL = 95%
Target

18. Material Distribution Sellers Risk (α)No.ofSamples
-3s -2s -1s +1s +2s +3s
68%
95%
99.7% 18
PWL = 99.7%
Target
α = 0.3%

19. Sellers Risk (α)No.ofSamples
-3s -2s -1s +1s +2s +3s
68%
95%
99% 19
TargetMean
PWL= ? α = ?

20. Determining Sellers Risk α
Mix Property: Binder Content
Pooled Standard Deviation (S): 0.20
Proposed Specification Tolerance: ± 0.4% (2 X S)
Target Binder Content: 5.0%
 Mean = 5.0
 USL = Upper Specification limit (5.4)
 LSL = Lower Specification Limit (4.6)
20

21. Determining Sellers Risk α
Percent Under Curve
(USL - x̄ )/S= Z score
(LSL - x̄ )/S= Z score
Z = 1.96 = 47.50
47.5 X 2 = 95
100 – 95 = 5.0% Risk
21

22. Seller’s Risk Evaluation Examples
DOT Proposed Binder Content Tolerance
Current Tolerance: ± 0.5%
Proposed Changing Tolerance: ± 0.3%
Evaluate the Risk Associated with the Proposed Change
 DOT
 Industry
 Academia
22

23. Risk Evaluation Examples
What is the Variability for Binder Content?
 Based on Statewide Pooled Data from QC/QA Projects
 Population Standard Deviation (S) = 0.20
23

24. 24
Seller’s Risk
(With 0.3% Tolerance)
S
(n=1)
x̄ PWL
Risk
(α)
0.20 5.0 86 14%
Binder Content
1.7 3.7 4.7 5.0 5.3 6.3 7.3
Upper limitLower limit
Target
5.0%
16%7%7%
NTS
86%
When the contractor provides a
product using all the care and skill
normally exercised within the
industry, they will fail to meet the
specified acceptance requirements
14% of the time.

25. 25
Sellers Risk
(With 0.3% Tolerance)
S
(n=2)*
x̄ PWL
Risk
(α)
0.14 5.0 96 4%
Binder Content
1.7 3.7 4.7 5.0 5.3 6.3 7.3
Upper LimitLower Limit
Target
5.0%
16%2%2%
NTS
96%
*Avg. of Two randomly seleted
Independent Samples within a
Sub Lot
15

26. Total Variability
Variability = variability + variability + variability
(sampling) (testing) (mat./const.)
S2
QC/QA = S2
s + S2
t + S2
m/c

27. Balancing Risk and CostAgencyand/orContractorRisk
DirectCost($)
1 2 3
Number of Test Samples (n)

28. 28
Sellers Risk
(With 0.3% Tolerance)
S
(n=2)* x̄ PWL
Risk
(α)
0.14 5.0 96 4%
Binder Content
1.7 3.7 4.7 5.0 5.3 6.3 7.3
Upper LimitLower Limit
Target
5.0%
16%2%2%
96%
*Averageof Two IndependentSamples
forStandardSpec Projects (minimum)
15NTS

29. 3
29
Sellers Risk
(With 0.4% Tolerance)
S
(n=1)
x̄ PWL
Risk
(α)
0.20 5.0 95 5%
in e ontent
3.8 4.2 4.6 5.0 5.4 5.8 6.2
Upper LimitLower Limit
Target
5.0%
16%2.5% 1.72.5%
Use with QC/QA Specifications
(Lot = X sublots)

30. QC/QA Proposal 2018
Type A HMA for Job Mix Formula Verification and Production Start Up
Quality characteristic
Test
method
Requirement
Asphalt binder content (%) AASHTO T
308
Method A
JMF ± 0.35
30
HQ Construction has observed that Contractors have been
achieving 1.04 – 1.05 pay factors from the old QC/QA specs
where tolerance is “± 0.40.

31. 31
Skewed Tolerance’s
Upper LimitLower Limit
Target
1
+5%
NTS

32. 32
Seller’s Risk
( it Skewed Tolerance)
S
(n=1)
PWL
Risk
(α)
0.20 5.0 92 8%
Binder Content
e i ito e i it
Target
5.0%
1
1
49%
x̄
NTS

33. 33
Seller’s Risk
( it Skewed Tolerance)
S
(n=1)
PWL
Risk
(α)
0.20 5.1 95 5%
Binder Content
4.7 5.1 5.5
Upper Limito e i it
e n
16%
47.5%47.5%
x̄et ue
NTS

34. 34
Additional Binder Effect on
Volumetrics
Air Voids
- 4.0 +
5.0% Design Binder Content
16%
Additional Binder (+0.1%)
Percent Defective

35. Evaluating Risk Examples (con’t)
Local Agency Relative Density Specification
Local Agency 92 – 97 % using single core (n=1)
Contractors’ Could Not Meet Minimum Density Specifications
Specification was Evaluated to Determine Contractors Risk
Specification was Modified to:
 Assigned proper level of contractor risk without compromising pavement
performance
35

36. Risk Evaluation Examples
What is the Variation for Relative Density?
 Sample Standard Deviation (s) = 1.84
 Based on <30 observations from projects
36

37. Relative Density Specifications
37
Relative Density Pay Factor
97.1 0r Higher (Over-asphalted mix) 90% Pay Factor
92-97% (Ideal) 100% Pay Factor
89 – 91.9 (Marginal Air Voids) 85% Pay Factor
88.9 Or Less Reject (RQL)
Pay Factors
For all asphalt concrete pavement subject to acceptance testing, the
finished asphalt concrete pavements that do not conform to the
specified relative compaction requirements will be paid for using the
following pay factors:

38. 9
38
Seller’s Risk
(With 2.5% Tolerance)
S
(n=1)
x̄ PWL
Risk
(α)
1.84 94.5 82 18%
Relative Density
92.0 94.5 97.0
Upper Limit
Lower Limit
Target
16%9%9%
NTS
82%
When the contractor provides a product
using all the care and skill normally
exercised within the industry, they will
fail to meet the specified acceptance
Requirements 18% of the time

39. How to Lower Sellers Risk
What are the Options
• Change Specification Tolerances
• Increase Number of Observations
39

40. 40
Seller’s Risk
(With 4.0% Tolerance)
S
(n=1) x̄ PWL
Risk
(α)
1.84 94.5 97 3.0%
Relative Density
90.5 94.5 98.5
Upper Limit
Lower Limit
Target
16%1.5%1.5%
NTS
98%
Change spec band from
± 2.5% to ±4.0%

41. Relative Density Specifications
41
Relative Density Pay Factor
97.1 0r Higher (Over-asphalted mix) 90% Pay Factor
92-97% (Ideal) 100% Pay Factor
89 – 91.9 (Marginal Air Voids) 85% Pay Factor
88.9 Or Less Reject (RQL)
Pay Factors
For all asphalt concrete pavement subject to acceptance testing, the
finished asphalt concrete pavements that do not conform to the
specified relative compaction requirements will be paid for using the
following pay factors:

42. How to Lower Sellers Risk
What are the Options
• Change Specification Tolerances
• Increase Number of Observations
42

43. 43
Seller’s Risk
(With 2.5% Tolerance)
S
(n=3)*
x̄ PWL
Risk
(α)
1.05 94.5 98 2%
Relative Density
92.0 94.5 97.0
Upper LimitLower Limit
Target
16%1%1%
NTS
98%
*Avg. of Three Independent Samples

44. Risk vs Number of Observations (n)
The myth of the Single Representative Sample
44

45. Hang in There!
45

46. The myth of the Single Representative Sample:
“The idea persists that a test on a single sample shows
the "true" quality of the material, and that if any test result
is not within some limit, there is something wrong with the
material, construction, sampling or testing. Thus, terms
such as investigational, check, and referee samples are
in common use to either confirm or document these
"failures.“ Nature dislikes identities; variation is the rule.
Therefore, any acceptance or process control sampling
must account for variability of materials or construction.
Multiple sampling accomplishes this objective”
FHWA - NHI Course No. 13442
46

47. Risk Vs Number of TestAgencyand/orContractorRisk
1 2 3 4 5 6 7
Number of Test Samples (n)
Best Practice:
Never make a decision to accept or reject
material based on a single observation!
47

48. FHWA Peer Review Team
Recommendation
For other items without pay factors (non critical sieves, SE, etc.) it
is recommended that if one test falls outside the specification
limit then another test will be taken. If the specification limit is
met on the subsequent test, production continues without any
penalties.
If the second consecutive test falls outside the specification limit,
production will cease until the contractor demonstrates that the
specification limit can be met.
48

49. 49
Acceptance of Binder Content
(Single Observation)
Asphalt Content
16%
NTS
.
USLLSL
AQL
Accept or Reject?

50. 50
Buyer’s Risk
(Single Observation)
Asphalt Content
16%
NTS
.
USLLSL
AQL RQL
Material Good or Bad?

51. 51
Buyer’s Risk
(Single Observation)
Asphalt Content
16%
NTS
AQL Population
.
RQL Population
Good or Bad?

52. 52
Buyer’s Risk
(Population Defined with Additional Test)
Asphalt Content
16%
NTS
AQL Population
.
RQL Population
.
.. .. .. .
.

53. 53
Buyer’s Risk
(Population Defined with Additional Tests)
Asphalt Content
16%
NTS
AQL Population
.
RQL Population
.
.. .. .. .
.
Good Material?

54. 54
Buyers Risk (β)
4.5 4.6 5.0 5.4
16%
Buyers
Risk (β)
NTS
RQL
α
USLLSL
50%
Binder Content
19%
31%
Based on a single observation there is a
31% chance of accepting RQL product
thinkin th t it’s t of AQ o uct
(population)
Binder Content
S
(n=1)
x̄ Test
.20 4.5 4.6

55. 55
Buyers Risk (β)
4.5 4.6 5.0 5.4
16%
Buyers
Risk (β)
NTS
RQL
α
USLLSL
50%
26%
24%
Binder Content
S
(n=2)
x̄ Test
.14 4.5 4.6
Based on a single observation there is a
24% chance of accepting RQL product
thinkin th t it’s t of AQ o uct
(population)
Binder Content

56. Quiz Question
Is it ever acceptable to accept or reject
material based on a single test result
56

57. 57

58. Questions
58