1. Determining the Heterogeneity
of Reference Materials
Thomas Bagley, Dr. Cliff Stanley, Dr. John Murimboh
Depts. of Earth & Environmental Science and Chemistry
Acadia University
Geoscientists commonly use geochemistry to address geological and environmental
problems. The geological samples are sent to the lab, and the results indicate the
major or trace element compositions. But how do we know that the results are true
to the sample?
Geoscientists send reference material with the batch of geological
samples. Reference materials have an accepted concentration, so the quality of the
analysis can be assessed from the result. My thesis concerns the heterogeneity of
reference materials – how reproducible are the results from reference materials?
Current practice assumes they are homogeneous, and attributes any variation to lab
error.
2. 2
Acknowledgements
• ACME Analytical Labs, Vancouver
• Bureau Veritas, Perth
• Acadia Faculty Research
Funding
• Canada Summer Jobs Funding
•MAC Student Research Funding
•SEG Student Research Funding
Participating Laboratories &
CRM Manufacturers
Grants
• African Mineral Standards,
Johannesburg Geostats Proprietary,
Vancouver
• CDN Resource Laboratories
• Ore Research/Exploration, Melbourne
• Rocklabs, Auckland
3. Background
• Certified Reference Materials (CRMs):
– Pulverized rock samples
– With accepted element concentrations
– With accepted standard deviations
– Used to monitor analytical quality
Certified reference materials are pulverized rock or soil samples. They are useful in
quality assessment/quality control (QAQC) because they are attributed with an
accepted concentration & error.
5. Background
• How are CRMs used ?
– Analyzed in a batch of samples
• CRM measured concentrations are
compared with accepted concentrations to
monitor accuracy
– CRM measured concentrations are compared
with each other to monitor precision
– These provide an assessment of data quality
for the batch
Certified Reference Materials (CRMs) are used by the mining/resource and
environmental industries to monitor analytical quality. Certified reference materials
are sent with a batch of samples for analysis. To assess accuracy, the measured
concentration can be compared to the accepted concentration. To measure
precision, the results can be compared against themselves.
6. Control Chart
This is an example of a traditional control chart. Each CRM is plotted in the order
that it was analyzed, from left to right.
The concentration is represented on the y axis. The control chart has
tolerances above and below the mean (1, 2 and 3 SD). Samples are unlikely to
have concentrations outside 2SD from the mean, and very unlikely to have
concentrations outside 3SD from the mean. In this chart there is an acceptable
variation in the results from an analysis. A sample beyond 2SD would probably not
be an outlier, but multiple samples beyond 2 SD would be anomalous. There also
aren’t any obvious patterns. Patterns can be cause for concern – they can indicate
problems like contamination. If one of two crushers is contaminated then the results
on a control chart would alternate high – low.
7. Background
• How are CRMs prepared ?
– Pulverized
– Homogenized
– Sub-sampled
– Chemically analyzed
(in multi-lab round robin)
– Statistically analyzed
(to remove outliers)
•The rock or soil sample is pulverized in a ball mill.
•The sample are blended. homogenized
•Sub samples are taken from the population which has been ‘homogenized’.
•The sub samples are analyzed at external labs in a ‘round robin’ analysis.
•Statistical processes are applied to remove outliers from the round robin: t-test (iso
5725), z-test, iso/iec 43-1, iso 3207..
•http://upload.wikimedia.org/wikipedia/commons/c/c2/Ball_mill.gif
Image of a ball mill from google images
http://bestservices.co.in/images/equipment/ball-mill-dry-type.jpg
8. Problem
• The variance observed in CRM concentrations is
not all laboratory error
• CRMs are heterogeneous, and thus exhibit
sampling error too
• But how heterogeneous are CRMs ?
-Despite the best efforts of the manufacturer, the homogenization is imperfect – the
CRMs are heterogeneous
-Also, segregation can occur during shipping – graded bedding
Therefore, the mining industry cannot use CRMs to monitor analytical quality
unless the magnitudes of their heterogeneity are known.
So, how heterogeneous are CRMs?
10. Background
• What contributes to the total variance of a CRM ?
– Sampling Error
– Laboratory Error
– Inter-Laboratory Error
2222
InterLabSampTot
Variance (gesture toward a sigma) is the standard deviation squared, it represents
the spread of the data. Variation is high when replicate values differ significantly in
concentration. Variance describes the magnitude of error. There are three sources
of variation in a certification/round-robin:
•Sampling error is the heterogeneity of the sample – this is relevant to all elements.
Gold is especially prone to high sampling error (nuggets..)
•Laboratory error is introduced during the analysis – variation in reagent quality,
internal pressure in the mass spec, flame flicker..
•Inter lab error is introduced when the labs in the round robin produce different
distributions.
11. Background
Outlying sample
Analysis of
larger samples ?Outlying lab
0.45
0.4
0.5
0.55
Here is our round-robin again:
•The red labs produced results that differ from the other labs, they’re outlying labs.
(different means)
•The purple results from lab 14 differ substantially from most of lab 14’s results, and
are outlying samples.
Statistical procedures are used to distinguish outliers in this sort of scenario, to
improve the quality of the certification.
•Also notice that the blue lab was much more precise than the other labs, this can
happen if they analyze using a larger sample mass.
12. Background
• Inter-laboratory error and outliers are removed
using inferential statistical tests
• Observed variation is now only the sum of lab
error and sample heterogeneity
• Standard practice assumes that CRMs are
homogenous; suggesting that observed
variation is only lab error (NOT TRUE!)
222
LabSampTot
By removing the outliers we also remove the inter-lab error.
•Since inter-lab error has been removed, the error equation becomes: (gesture
toward the equation on ppt)
•This is the point where it has generally been assumed that the reference material is
homogeneous and that the total variation is lab error. THIS IS NOT TRUE, BUT IT
IS STANDARD PRACTICE.
13. Objective
• Need to determine the magnitude of
sampling error in CRMs
To improve the quality assessment methods, we must be able to determine the
magnitude of the sampling error.
14. Strategy
• Employ Dr. Stanley’s method to measure CRM
lab and sampling errors simultaneously
– Measure CRM concentrations in small and large
samples
– Solve using 3 equations / 3 unknowns
2
.
2
.
22
.
2
,
22
.
2
,
SmSampSmLgSampLg
LabSmSampSmTot
LabLgSampLgTot
MM
There is a statistical approach that was developed by my supervisor, Dr Cliff
Stanley that’s separates sample and lab error.
•It is known that the sample mass is proportional to sampling error. Assuming lab
error to be constant then the sample error can be determined.
•In this way, sampling error and lab error can both be determined.
15. Methods – Analysis of CRMs
• Aqua Regia/ICP-MS for trace elements
• 9 * 2.00 g samples;
• 36 * 0.50 g samples;
• Data quality assessment samples
• Calculate large and small CRM sampling
errors and laboratory error
2
.
2
.
,
,
SmTotSm
LgTotLg
M
M
22
.
2
. LabSmSampLgSamp
The lab procedure was developed with Dr John Murimboh. This is the summary of
the procedure that we use for each batch of samples. There are large and small
samples, measured in replicate to estimate the variance. The procedure blank +
control sample are for QAQC purposes
Using this procedure we estimate each error:
•Small sampling error
•Large sampling error
•Lab error
16. Sources of Error in Co
0
2
4
6
8
10
12
14
16
2P
1M
4E
7F
12A
GS50
20B
BAS-1
P4B
BL-10
P6
QUA-1(1)
QUA-1(2)
%RSD
Small Sampling Error
Large Sampling Error
Analytical Error
Results
%RelativeStandardDeviation
The method allows us to determine the sampling error for both the large and small
samples, and the analytical error. These graphs illustrates the relative proportions of
analytical and sampling error for large and small samples. The small samples have
much more sampling error! No surprise there! Samples that have no bars plotted
did not produce valid estimates of sampling error and analytical error (to be
discussed in detail later on)
17. Fundamental Sampling Constant
• Unique to pulverized material
• Relates sample size to sampling error
(inversely proportional)
• For a given sample size, sampling error is
known
ΨσMσM SmSampSmLgSampLg 2
,
2
,
More background:
In the last graphs we saw that the sampling error was smaller for large samples. It
turns out that the variance is inversely proportional to sample mass. That is, they
multiply to define a constant, the fundamental sampling constant (FSC). The FSC
can be used to compare homogeneity, and it allows us to CALCULATE THE
ERROR AT ANY SAMPLE MASS – very practical information for QAQC.
18. 18
Results
Variance Ratio Plot of Co in Reference Materials
0.1
1
10
2P
1M
4E
7F
12A
GS50
20B
BAS-1
P4B
BL-10
P6
QUA-1(1)
QUA-1(2)
Reference Material Batch
LargeVariance/SmallVariance
02
Lab σ
0
and
0
2
Sm.Samp
2
Lg.Samp
σ
σ
The ratio of the total variances determines if the sampling and analytical errors can
be decomposed. The lower tolerance is equal to the mass ratio (Ms/ML = 0.25 in
this case), the upper tolerance is 1: the large total variance equalling the small total
variance. Above the upper tolerance, negative sampling errors result and below the
lower tolerance we get negative analytical errors.
19. 19
Discussion
• The large & small sampling, and
laboratory variances exhibit error
• Sampling and Laboratory error
measurements are dependent on
adequate total variance estimates
• How do we achieve adequate estimates of
the total variance?
The cause of failed sampling & analytical error estimates is likely to be a low
standard error on the total variance (caused by too few replicates or an ineffective
sampling strategy). How can we achieve the best estimates?
20. 20
Discussion
• Standard error is dependent on the
sampling & analytical variances, and the
number of samples
• It is best to estimate total variances with
the same standard error
1n
2
s
1n
2
sSE
Lg
2
Lab
Lg
2
Lg.Samps2
Lg.Tot
1n
2
s
1n
2
sSE
Sm
2
Lab
Sm
2
Sm.Samps2
Sm.Tot
Standard error on the total variance is a function of the sampling & analytical error,
and the number of samples. The best strategy to maximize the precision of the
analytical and sampling error estimates is to make the standard error of the large
total variance equal to the standard error on the small total variance (Stanley &
Smee 2007).
21. 21
Discussion
Analytical variance is equal to the small sampling variance in this diagram, the large
sampling variance is ¼ of the small sampling variance for a sample mass 4
times the size (Stanley 2007).
1) Small samples need to be analyzed more times (as expected) to achieve the
same standard error
22. 22
Discussion
2) The optimal sampling strategy uses the same standard error on the variance for
both large and small samples; 36 and 15 in this case (we would have achieved
more consistent results with fewer invalid estimates had we used 15 large samples
instead of 9)
23. 23
Discussion
• To determine the optimal sampling
strategy (equal standard errors)
1
)(
)1n()1(
n 2
S
2
L
κλ
λ
2
Lg.Samp
2
Lab
Sm
Lg
s
s
,
M
M
λκ
The previous example was for fixed sampling and analytical errors
The sampling strategy changes depending on the proportions of sampling and
analytical error. This system models the number of small to large samples that it
takes to achieve an equal standard error on the total variances.
24. 24
Discussion
4
M
M
Sm
Lg
The optimal proportion of small to large samples to analyze based on the % large
sampling variance (of the large total variance). The orange line is the same case
that we examined previously (50% sm. Sampling error), not that 15 large samples to
approximately 36 small samples.
This diagram also illustrates that the sampling strategy for a reference material
exhibiting no sampling error is an equal proportion of large and small samples, a
case where there is only analytical error (slope = 1, blue line, 0.001%). The
sampling strategy for a reference material that exhibits only sampling error (a
perfect geochemical analysis!) would use a proportion of 1:16 large to small
samples (slope of 16), when using a large sample mass 4 times the small sample
mass.
25. 25
Discussion
8
M
M
Sm
Lg
By using samples with twice the difference (1:8 instead of 1:4 by mass), the number
of large samples is greatly decreased – only 8 would need to be analyzed for 36
small samples.
26. Conclusions
• Reference material heterogeneity it is
not zero
• Fundamental sampling constants can be
used to estimate sampling error at
different sample masses
• CRM manufacturers should provide
fundamental sampling constants with their
accepted values
• Analytical procedures should be designed
to optimize standard error on the variance
27. 27
Future work
• Investigate the controls on sampling
strategy, to maximize precision
• Develop QAQC methods that
accommodate reference material
heterogeneity
28. 28
References
• Stanley, C.R. 2007. The Fundamental Relationship
Between Sample Mass and Sampling Variance in Real
Geological Samples and Corresponding Statistical
Models. Exploration and Mining Geology, 16: 109-123.
• Stanley, C.R., and Smee, B.W. 2007. Strategies for
Reducing Sampling Errors in Exploration and Resource
Definition Drilling Programmes for Gold Deposits.
Geochemistry: Exploration, Environment, Analysis, 7: 1-
12.
• Stanley, C.R, O'Driscoll, N., and Ranjan, P. 2010.
Determining the magnitude of true analytical error in
geochemical analysis. Geochemistry: Exploration,
Environment, Analysis, 10: 355–364
