The amount of environmental data is increasing, and the data would be valuable to the society if they are delivered to the right processes at the right time. In the seminar, we show examples of available data, how they are produced and processed, and how the data can be used in new innovative applications. This presentation is part of the Environmental Data for Applications Seminar held on the 23rd of September 2015. The seminar was organised by the MMEA (Measurement, Measuring and Environmental Assessment) research programme under the Cleen Ltd (SHOK). The presentations are based on the research results related to environmental data interoperability. The participants included key players and partners in the field of environmental monitoring in Finland. More info at www.mmea.fi

1. VARIOGRAM-DERIVED MEASURES FOR
QC PURPOSES
Markku Ohenoja
Control Engineering group
University of Oulu
1
10/15/2015Faculty of Technology / Control Engineering / Markku Ohenoja

2. 15.10.2015
2
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
Petersen, L., Minkkinen, P. & Esbensen, K.H. 2005, "Representative sampling for reliable data analysis: Theory
of Sampling", Chemometrics and Intelligent Laboratory Systems, vol. 77, no. 1–2, pp. 261-277.
Time
Meas.
https://s-media-cache-
ak0.pinimg.com/236x/64/46/7f/
64467fa3382ac08d567d36b6aef05
13b.jpg

3. BACKGROUND
• All measurements retain some amount of uncertainty, but also
sampling errors may affect on the result
• Utilization of different measurements collected with very
different sampling rates requires evaluation of their
information content
• Environmental measurements are often periodic, sparsely
collected and from various sources
• Variographical analysis used for evaluating sampling errors and
information content of the measurement
15.10.2015
3
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

4. OUTLINE
• What is Variogram and how it is calculated?
• Variogram-derived measures
• Examples within MMEA
15.10.2015
4
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

5. VARIOGRAM
• Tool for empirical estimation of sampling errors incl. analytical
error
• Enables optimizing the sampling strategy with respect to
variance of the sampling error and number of samples takes
• Provides an estimate of the standard error of the lot mean and
the minimum possible error (MPE) of sampling
15.10.2015
5
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
Semi-variogram
Chrono-variogram
Variographical
analysis
Geostatistics
Kriging
Variography Chronostatistics

6. VARIOGRAM
• Collection of the data
• At least 30 samples with systematic sampling
• 1/5 smaller sampling interval than routine samples
• Flowrate/sample weight should be included
• Calculation of the heterogeneity of the data
• Calculation of the experimental variogram v(j)
• Relationship between the samples and the lag distance j
• Estimation of the intercept v(0) (=MPE)
• Graphically, separate experiment…
• Auxiliary functions for comparing sampling strategies
• Point-to-point calculation, algebraic modeling…
15.10.2015
6
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
ℎ 𝑛 =
𝑎 𝑛 − 𝑎 𝐿
𝑎 𝐿
∙
𝑀 𝑛
𝑀 𝑛
𝑣 𝑗 =
1
2(𝑁 − 𝑗)
ℎ 𝑛+𝑗 − ℎ𝑗
2
𝑁/2
𝑛=1
≈

7. VARIOGRAM
15.10.2015
7
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
0 10 20 30
0
5
10
15
20
25
30
Variogram of 24h averaged online data
Sampling interval (days)
Relativestandarddeviationofthesamplingerror(%)
0 10 20 30
0
5
10
15
20
25
30
Variogram of daily sample
Sampling interval (days)
Relativestandarddeviationofthesamplingerror(%)
Variogram
Systematic sampling
Random sampling
Variogram
Systematic sampling
Random sampling
σ2,σ,2σ,...

8. VARIOGRAM
15.10.2015
8
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
0 10 20 30
0
5
10
15
20
25
30
Variogram of 24h averaged online data
Sampling interval (days)
Relativestandarddeviationofthesamplingerror(%)
0 10 20 30
0
5
10
15
20
25
30
Variogram of daily sample
Sampling interval (days)
Relativestandarddeviationofthesamplingerror(%)
Variogram
Systematic sampling
Random sampling
Variogram
Systematic sampling
Random sampling
3x

9. INDICES
15.10.2015
9
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
• Variogram-based indices applied for QC and PAT purposes
• Standard error of the mean
• MPE/σProcess
• v(1)/σProcess

10. INDICES
15.10.2015
10
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
• Variogram-based indices applied for QC and PAT purposes
• Standard error of the mean
• MPE/σProcess
• v(1)/σProcess
Process stability
measure
Bisgaard & Kulahci, Quality
Engineering, 17(2), 2005
Drift estimation
Paakkunainen et al.,
Chemometrics and Intelligent
Laboratory Systems, 88(1), 2007
Fault diagnosis
Kouadri et al., ISA Transactions,
51(3), 2012 Temporal uncertainty
propagation
Jalbert et al., Journal of
Hydrology, 397(1-2), 2011
DQOs for control
charts
Minnit & Pitard, Journal of SAIMM,
108(2), 2008

11. STANDARD ERROR OF THE MEAN
• Variance estimate of the sampling attained from variogram
• Standard error of the mean calculated based on variance
estimate and number of samples collected during a selected
time frame
• Recursive calculation possible for online measurements 
moving average and its confidence intervals from the selected
time frame
15.10.2015
11
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

12. STANDARD ERROR OF THE MEAN
15.10.2015
12
Month 2M 3M 4M 5M HalfYear Year All
0
0.5
1
1.5
2
2.5
3
2M
,%
Time frame for the lot mean
Online 17h average
Online 12h average
Online 8h average
Online 6h average
Online 4h average
Online data
Month 2M 3M 4M 5M HalfYear Year All
0
5
10
15
20
25
30
35
40
2M
,%
Time frame for the lot mean
Laboratory
Calibrated online
Raw online x 10
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

13. STANDARD ERROR OF THE MEAN
15.10.2015TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen
Sukuniminen-Sukuniminen
13
23-Nov-2009 12-Jan-2010 03-Mar-2010 22-Apr-2010 11-Jun-2010 31-Jul-2010 19-Sep-2010 08-Nov-2010 28-Dec-2010 16-Feb-2011
0
20
40
60
31-Dec-2010
7.341 7.3415 7.342 7.3425 7.343 7.3435 7.344 7.3445 7.345 7.3455
x 10
5
10
15
20
25
Lot mean and 2
M
(%) for Three day average
7.341 7.3415 7.342 7.3425 7.343 7.3435 7.344 7.3445 7.345 7.3455
x 10
5
0
10
20
30
23-Nov-2009 12-Jan-2010 03-Mar-2010 22-Apr-2010 11-Jun-2010 31-Jul-2010 19-Sep-2010 08-Nov-2010 28-Dec-2010 16-Feb-2011
5
10
15
20
25
30
Lot mean and confidence intervals for Three day average

14. DATA COMPARISON
• Multiple measurement sources with different sampling rates
• Data harmonization and comparison
• Based on MPE
• Comparable averaging of the dense data around sparse samples,
• Variographical analysis for whole averaged dense data mimicking
more densely collected laboratory measurements
• Information content evaluation based on v(1)/σProcess
15.10.2015
14
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

15. WHAT SPARSE CANNOT SEE?
15.10.2015
15
0 5 10 15 20 25
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
Variograms for collective samples
Sampling interval
Variance
Variogram, Sparse meas.
Variogram, Av. dense meas.
0 5 10 15 20 25
0
0.1
0.2
Variogram of sparse measurement
Variance
Sampling interval
0 200 400 600 800 1000
0
0.1
0.2
Variogram of averaged dense measurement
Sampling interval
Variance
Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi

16. WHEN DENSE IS NOT REPRESENTATIVE?
15.10.2015TIEDEKUNTA TIEDEKUNTA / osasto osasto osaston osasto / Etuniminen
Sukuniminen-Sukuniminen
16
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013
0
10
20
30
40
Meas.
Time series
Dense meas.
Sparse meas.
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013
0
0.5
1
1.5
es
/
P
Index
Dense meas.
Sparse meas.
26-May-2013 05-Jul-2013 14-Aug-2013 23-Sep-2013 02-Nov-2013 12-Dec-2013
-1
-0.5
0
0.5
1
Substracted index
Index

17. SUMMARY
15.10.2015Faculty of Technology / Control Engineering / Markku Ohenoja
markku.ohenoja@oulu.fi
17
• Variogram can be utilized for
1. Sampling error estimation
2. Sampling optimization
3. Moving average and confidence interval calculation
4. Information content evaluation
• Recursive calculation enables e.g. monitoring, filtering,
decision making
• Information content evaluation allows comparison of
measurement sources