Quantifying Error in Training Data for Mapping and Monitoring the Earth System - A Workshop on “Quantifying Error in Training Data for Mapping and Monitoring the Earth System” was held on January 8-9, 2019 at Clark University, with support from Omidyar Network’s Property Rights Initiative, now PlaceFund.

1. ORNL is managed by UT-Battelle, LLC
for the US Department of Energy
Challenges to Large Scale Mapping: can
data geometry help?
Presented at
Clark University 2019
Dalton Lunga
AI and Machine Learning Scientist
Geographic Data Sciences Group
lungadd@ornl.gov

2. 22
Area: 1,345,006 sq km
Images: 20,000
Data size:
90TB
Total 7.68 trillion px
Builtup: 51 billion px
Area: 1,262,155 sq km
Images: 4800
Data size:
18TB
Builtup: 62 billion px
Total 32 trillion px
Data volume challenge
only 2% of imagery covers built-up area … we are searching for a needle in a haystack!

3. 33
From 1m NAIP imagery From <1m WV and aerial
imagery
Data Variety: Heterogeneity of sensors
… introduces sample data bias …

4. 44
Long Beach, LA

5. 55
Humanitarian impact: mission support for 2017
hurricane season

6. 66
Generating high resolution solar panel maps for
US Cities
San Francisco

7. 77
Humanitarian impact: high resolution settlement
maps to support polio vaccination in Nigeria
Kano Borno
EnuguLagos

8. 88
Site A Site B Site C Site D
Challenges for automation: data distribution
shifts

9. 99
Data bias from measurement distortions

10. 1010
Limitations for automated classifiers

11. 1111
Exploiting geometry of data

12. 1212
Understanding human labeling induced errors

13. 1313
Shifted labels—binary model output
3 pixels 6 pixels 12 pixels 15 pixels
The level of degraded quality of training annotation for building mapping

14. 1414
False negative—distance model output
10
%
20% 40% 60
%
The level of degraded quality of training annotation for building mapping

15. 1515
False positive—binary model output
10
%
20% 40% 60
%
The level of degraded quality of training annotation for building mapping

16. 1616
Thank you
Dalton Lunga
lungadd@ornl.gov
Budhu Bhaduri
bhaduribl@ornl.gov
Contact Details

17. 1717
Active learning to increase sample set
• Human-in-the-loop: Can we incorporate more
representative/meaningful samples into the training
set, with less human efforts with the most valuable
domain knowledge encodedà Machine guided and
human assisted labeling

18. 1818
Data Bias from Training Data Sampling