This paper explores the performance of two commonly used air quality models: dispersion models and land-use regression models. Both models are widely used in air pollution epidemiological studies and in health impact assessment studies. In this work, we looked at how the choice of the validation dataset impacts the performance of air quality models and the insights gleaned from their validation. We also looked at whether the spatial resolution for the models' setup impacts the performance of air quality models and the insights gleaned from their validation. We saw that R-squared almost halved when the air quality models' estimates were made at the centroid of the 100x100m grid in which the validation point fell, instead of at the exact location of the validation point. We also saw that the different validation datasets give very different insights. Dispersion models and land-use regression models are widely used in air pollution epidemiological studies and in health impact assessment studies. As such the performance of these air quality models have implications on the ability of epidemiological studies to pick up associations between the exposures and the health outcomes of interest, and the ability of health impact assessment studies to quantify the impacts accurately. This work demonstrated the value of validating modeled air quality data against various datasets to obtain a better understanding of the performance of models and the value of reporting these validation results. Also, the work suggested that the spatial resolution of the models’ estimates has a significant influence on the validity at the application point. These results should be considered when air quality models are used to assign human exposures and study the health effects/impacts of these exposures. Significant work is still needed to improve the performance of air quality models and their ability to pick up the variations of air pollution levels and especially the higher and more variable levels that are related to traffic. Significant work is also still needed to account for the factors that underlie this variation in epidemiological and health impact assessment studies, especially time activity patterns of the exposed populations

1. 1
Center for Advancing Research
in Transportation Emissions,
Energy and Health
U.S. Department of Transportation University
Transportation Centers Program
HaneenKhreis, PhD
www.carteeh.org

2. 2
“The Impact of
Different
Validation
Datasets on Air
Quality Modeling
Performance”

3. 3
•Not possible to make sufficient air pollution exposure
measurements for epidemiological and health impact
assessments  143,472 children
•Many studies rely on air pollution modeling
•Commonly used models include:
• Land use regression (LUR)
• Atmospheric dispersion (AD)
Background

4. 4
LUR modeling
• Repeated measurements
(passive samplers) at N
sites
• Average measurements
over longer term and
adjust for temporal
variations
• Regression model to
combine measurements
with GIS-based predictors
• Apply model to non-
measured locations

5. 5
LUR modeling
• BUILDINGS  Local land use, Area/number of buildings, m2/N(umber)
• TRAFLOAD  Local road network, Total traffic load of all roads in a buffer (sum of
(traffic intensity*length of all segments)), Veh. Day-1 m
• NATURAL  Semi-natural and forested areas, m2
• HEAVYTRAFMAJOR  Heavy-duty traffic intensity on nearest major road, Veh. Day-1

6. 6
AD modeling

7. 7
•Models are only validated using one validation dataset
•Their estimates at select receptor points are
sometimes generalized to larger areas
•This may lead to unsatisfactory validation and/or
inaccurate insights about the models’ performance
and suitability for large-scale application
Background

8. 8
•Objective 1  explore the effect of different
validation datasets on the validation results of two
commonly used air quality models
•Objective 2  explore the effect of the model
estimates’ spatial resolution on the models’ validity at
different locations
Objectives

9. 9
Study area

10. 10
Annual (2009) NO2 and NOx
LUR model
Validation against 4
different datasets
Spatial resolution
analysis
Estimate at exact location
of validation point
Estimates at centroid of
100x100m grid in which
validation point fell
AD model
Validation against 4
different datasets
Spatial resolution
analysis
Estimates at exact
locations of validation
point
Estimates at centroid of
100x100m grid in which
validation point fell
Methods

11. 11
Measurement
campaign and
dataset (n = 126)
Pollutants
measured Measurement device
Year and time
interval for final
dataset
Locations and purpose of
measurements
ESCAPE diffusion
tubes (n=41) NO2 and NOx Ogawa badges 2009
(annualized)
At the façade of homes of
study subjects as the primary
objective of the ESCAPE
project was to characterize
residential exposures and
associated health
CBMDC diffusion
tubes (n=29) NO2 “Diffusion tubes” 2009
(annualized)
Three sites were not close to
main road whilst the rest
were kerbside sites at 0.5-5m
from the nearest road,
monitoring undertaken to
review and assess air quality
progress
de Hoogh diffusion
tubes (n=48) NO2 Palmes tubes
Four 2-week
periods during
2007-2008
Close to the front door of 48
homes of study subjects from
the Born in Bradford cohort
to characterize their
residential exposures and
compare with future ESCAPE
work
CBMDC fixed-site
monitoring (n=8) NO2
Automatic urban network
chemiluminescence
2009
(annualized)
Two sites were classified as
urban background whilst the
rest were kerbside sites at
1.5-2 m from the nearest
road, monitoring undertaken
to review and assess air
quality progress
Methods

12. 12
Annual (2009) NO2 and NOx
LUR model
Validation against 4
different datasets
Spatial resolution
analysis
Estimate at exact location
of validation point
Estimates at centroid of
100x100m grid in which
validation point fell
AD model
Validation against 4
different datasets
Spatial resolution
analysis
Estimates at exact
locations of validation
point
Estimates at centroid of
100x100m grid in which
validation point fell
Methods

13. 13
Results: validation against different datasets
Models combination Validation dataset
ESCAPE
NOx
diffusion
tubes
(n=41)
ESCAPE
NO2
diffusion
tubes
(n=41)
CBMDC
NO2
diffusion
tubes
(n=29)
De Hoogh
NO2
diffusion
tubes
(n=48)
CBMDC NO2
fixed-site
monitoring
(n=8)
ADmodel
COPERT dispersion
model NOx at points
(varying background)
R2
= 0.30
COPERT dispersion
model NO2 at points
(varying background)
R2
= 0.33 R2
= 0.20 R2
= 0.59 R2
= 0.24
LURmodel
NOx LUR estimates
at points
R2
= 0.58
NO2 LUR estimates
at points
R2
= 0.54 R2
= 0.21 R2
= 0.61
R2
= 0.38
(r= 0.62)

14. 14
AD vs. LUR annual average
NO2/NOx Estimates (µg/m3) at
 46,452 specified output
points centering each 100m x
100m grid across 40 * 33 km
Results: spatial
resolution of
estimates

15. 15
Results: spatial resolution of estimates
Validation dataset
ESCAPE NOx
diffusion tubes
(n=41)
ESCAPE NO2
diffusion tubes
(n=41)
CBMDC NO2
diffusion tubes
(n=29)
de Hoogh NO2
diffusion tubes
(n=48)
CBMDC
NO2
fixed-site
monitorin
g (n=8)
LURmodels
NOx LUR
estimates at
points
R2= 0.58
NOx LUR
estimates at
raster
R2= 0.35
NO2 LUR
estimates at
points
R2= 0.54 R2= 0.21 R2= 0.61
R2= 0.38
(r= 0.62)
NO2 LUR
estimates at
raster
R2= 0.31 R2= 0.06 R2= 0.32 R2= 0.38
(r=- 0.61)
-23%

16. 16
Results: spatial resolution of estimates
Validation dataset
ESCAPE NOx
diffusion tubes
(n=41)
ESCAPE NO2
diffusion tubes
(n=41)
CBMDC NO2
diffusion tubes
(n=29)
de Hoogh NO2
diffusion tubes
(n=48)
CBMDC
NO2
fixed-site
monitorin
g (n=8)
LURmodels
NOx LUR
estimates at
points
R2= 0.58
NOx LUR
estimates at
raster
R2= 0.35
NO2 LUR
estimates at
points
R2= 0.54 R2= 0.21 R2= 0.61
R2= 0.38
(r= 0.62)
NO2 LUR
estimates at
raster
R2= 0.31 R2= 0.06 R2= 0.32 R2= 0.38
(r=- 0.61)
-23% -15% -29%

17. 17
• LUR and AD models validated against four different datasets
Summary and discussion

18. 18
• LUR and AD models validated against four different datasets
• LUR and AD model estimates made at different spatial resolution and
validated against four different datasets
Summary and discussion

19. 19
• LUR and AD models validated against four different datasets
• LUR and AD model estimates made at different spatial resolution and
validated against four different datasets
• The validation metrics varied substantially (R2 0.20 – 0.61) based on:
• which model was used
• which validation dataset was used
• whether exposure estimates were made at exact validation point or at
centroid of containing grid
Summary and discussion

20. 20
• LUR and AD models validated against four different datasets
• LUR and AD model estimates made at different spatial resolution and
validated against four different datasets
• The validation metrics varied substantially (R2 0.20 – 0.61) based on:
• which model was used
• which validation dataset was used
• whether exposure estimates were made at exact validation point or at
centroid of containing grid
• Validation results based on actual points’ locations were generally
much better than at a grid level
Summary and discussion

21. 21
• There is value of validating modeled air quality data against various
datasets
Conclusions

22. 22
• There is value of validating modeled air quality data against various
datasets
• The spatial resolution of the models’ estimates has a significant
influence on the validity at the application point (even at 100m level)
Conclusions

23. 23
• There is value of validating modeled air quality data against various
datasets
• The spatial resolution of the models’ estimates has a significant
influence on the validity at the application point (even at 100m level)
• Have implications for epidemiological studies disregarding time-activity
patterns or using location proxies
Conclusions

24. 24
• There is value of validating modeled air quality data against various
datasets
• The spatial resolution of the models’ estimates has a significant
influence on the validity at the application point (even at 100m level)
• Have implications for epidemiological studies disregarding time-activity
patterns or using location proxies
• Have implications for health impact assessments where estimates of air
quality models at select points are extrapolated to larger areas/populations
Conclusions

25. 25
• There is value of validating modeled air quality data against various
datasets
• The spatial resolution of the models’ estimates has a significant
influence on the validity at the application point (even at 100m level)
• Have implications for epidemiological studies disregarding time-activity
patterns or using location proxies
• Have implications for health impact assessments where estimates of air
quality models at select points are extrapolated to larger areas/populations
• Can improve understanding of the most influential
uncertainties/errors across full-chain health impact assessment
Conclusions

26. 26

27. 27
Thank you!
Haneen Khreis
H-khreis@tti.tamu.edu
@HaneenKhreis