Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Tarabalka_IGARSS.pdf
1. Best Merge Region Growing
with Integrated Probabilistic Classification
for Hyperspectral Imagery
Yuliya Tarabalka and James C. Tilton
NASA Goddard Space Flight Center,
Mail Code 606.3, Greenbelt, MD 20771, USA
e-mail: yuliya.tarabalka@nasa.gov
July 28, 2011
2. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Outline
1 Introduction
2 Proposed spectral-spatial classification scheme
3 Conclusions and perspectives
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 2
3. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hyperspectral image
Every pixel contains a detailed spectrum (>100 spectral bands)
+ More information per pixel → increasing capability to distinguish
objects
− Dimensionality increases → image analysis becomes more complex
⇓
Advanced algorithms are required!
λ pixel vector xi
samples
x1 x2 x3 xi xi
x2i
Tens or xn
hundreds of
bands lines wavelength λ
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 3
4. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Supervised classification problem
AVIRIS image Ground-truth Task
Spatial resolution: 20m/pix
Spectral resolution: 200 bands data
16 classes: corn-no till, corn-min till, corn, soybeans-no till, soybeans-min
till, soybeans-clean till, alfalfa, grass/pasture, grass/trees,
grass/pasture-mowed, hay-windrowed, oats, wheat, woods,
bldg-grass-tree-drives, stone-steel towers
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 4
5. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Classification approaches
Only spectral information
Pixelwise approach
Spectrum of each pixel is analyzed
SVM and kernel-based methods
→ good classification accuracies
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 5
6. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Classification approaches
Only spectral information
Pixelwise approach
Spectrum of each pixel is analyzed
SVM and kernel-based methods
→ good classification accuracies
Spectral + spatial information
Info about spatial structures is included
Because neighboring pixels are related
How to extract spatial information?
How to combine spectral and spatial
information?
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 5
7. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Our previous research
Segment a hyperspectral image into homogeneous regions
Each region = adaptive neighborhood for all the pixels within the region
Spectral info + segmentation map → classify image
1 1 1 1 1 1
1 1 1 2 2 2
Segmentation
1 1 2 2 2 2
map
(3 spatial regions) 1 1 2 2 2 2
1 1 3 3 3 2
1 3 3 3 3 3
3 3 3 3 3 3
Combination of Result of
Pixelwise segmentation and spectral-spatial
classification map pixelwise classification
(dark blue, white classification results (classification
and light grey (majority vote within map after majority
classes) 3 spatial regions) vote)
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 6
8. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Our previous research
Segment a hyperspectral image into homogeneous regions
Each region = adaptive neighborhood for all the pixels within the region
Spectral info + segmentation map → classify image
1 1 1 1 1 1
1 1 1 2 2 2
Segmentation
1 1 2 2 2 2
map
(3 spatial regions) 1 1 2 2 2 2
1 1 3 3 3 2
1 3 3 3 3 3
3 3 3 3 3 3
Combination of Result of
Pixelwise segmentation and spectral-spatial
classification map pixelwise classification
(dark blue, white classification results (classification
and light grey (majority vote within map after majority
classes) 3 spatial regions) vote)
Unsupervised segmentation: dependence on the chosen measure of
homogeneity
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 6
9. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Our previous research: Marker-controlled segmentation
Probabilistic pixelwise Markers = the Marker-
SVM classification most reliably controlled region
classified pixels growing
Classification map Probability map
⇒ ⇒
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 7
10. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Our previous research: Marker-controlled segmentation
Probabilistic pixelwise Markers = the Marker-
SVM classification most reliably controlled region
classified pixels growing
Classification map Probability map
⇒ ⇒
Drawback: strong dependence on the performance of the selected
probabilistic classifier
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 7
11. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Objective
Perform segmentation and classification concurrently
→ best merge region growing with integrated classification
⇑ ⇑
Dissimilarity criterion? Convergence criterion?
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 8
12. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Input
Hyperspectral image
Preliminary
probabilistic
classification
Each pixel =
B-band hyperspectral image one region
X = {xj ∈ RB , j = 1, 2, ..., n}
While
not converged
B ∼ 100
Find min(DC) between
all pairs of spatially
adjacent regions (SAR)
Merge all pairs of SAR
with DC = min(DC).
Classify new regions
Spectral-spatial
classification map
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 9
13. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Preliminary probabilistic classification
Kernel-based SVM classifier* → well suited for Hyperspectral image
hyperspectral images Preliminary
probabilistic
classification
Output: Each pixel =
one region
• classification map • for each pixel xj : While
not converged
L = {Lj , j = 1, ..., n}
a vector of K class Find min(DC) between
all pairs of spatially
probabilities adjacent regions (SAR)
Merge all pairs of SAR
{P (Lj = k|xj ), with DC = min(DC).
Classify new regions
k = 1, ..., K}
Spectral-spatial
classification map
*C. Chang and C. Lin, "LIBSVM: A library for Support Vector Machines," Software available at
http://www.csie.ntu.edu.tw/∼cjlin/libsvm, 2011.
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 10
14. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
1 Each pixel xi = one region Ri Hyperspectral image
preliminary class label L(Ri )
Preliminary
class probabilities probabilistic
classification
{Pk (Ri ) = P (L(Ri ) = k|Ri ), k = 1, ..., K} Each pixel =
one region
While
not converged
1
2
3 Find min(DC) between
all pairs of spatially
adjacent regions (SAR)
4
5
6 Merge all pairs of SAR
with DC = min(DC).
Classify new regions
7
8
9 Spectral-spatial
10
11
12
classification map
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 11
15. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
2 Calculate Dissimilarity Criterion (DC) between
Hyperspectral image
spatially adjacent regions
Preliminary
DC = function of region statistical, geometrical probabilistic
classification
and classification features Each pixel =
one region
1
?
2
3
While
not converged
Find min(DC) between
all pairs of spatially
4
5
6
adjacent regions (SAR)
Merge all pairs of SAR
with DC = min(DC).
Classify new regions
7
8
9
Spectral-spatial
classification map
10
11
?
12
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 12
16. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
2 Calculate Dissimilarity Criterion
between adjacent regions:
Compute Spectral Angle Mapper Compute SAM between region
mean vectors SAM(ui, uj)
between the region mean vectors
ui and uj no yes
L(Ri) = L(Rj) = k’
ui · uj
SAM(ui , uj ) = arccos( )
ui 2 uj 2 no (card(Ri) > M) & yes DC = (2 – max(Pk’(Ri),
(card(Rj) > M) Pk’(Rj)))SAM(ui, uj)
If adjacent regions have equal class DC = ∞
labels → they belong more likely to
DC = (2 – min(PL(Rj)(Ri),
the same region: PL(Ri)(Rj)))SAM(ui, uj)
DC = (2−max(Pk (Ri ), Pk (Rj ))) ·
SAM(ui , uj )
DC
If two large regions are assigned to
different classes → they cannot be
merged together
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 13
17. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
2 Calculate Dissimilarity Criterion
between adjacent regions:
Compute Spectral Angle Mapper Compute SAM between region
mean vectors SAM(ui, uj)
between the region mean vectors
ui and uj no yes
L(Ri) = L(Rj) = k’
ui · uj
SAM(ui , uj ) = arccos( )
ui 2 uj 2 no (card(Ri) > M) & yes DC = (2 – max(Pk’(Ri),
(card(Rj) > M) Pk’(Rj)))SAM(ui, uj)
If adjacent regions have equal class DC = ∞
labels → they belong more likely to
DC = (2 – min(PL(Rj)(Ri),
the same region: PL(Ri)(Rj)))SAM(ui, uj)
DC = (2−max(Pk (Ri ), Pk (Rj ))) ·
SAM(ui , uj )
DC
If two large regions are assigned to
different classes → they cannot be
merged together
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 13
18. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
2 Calculate Dissimilarity Criterion
between adjacent regions:
Compute Spectral Angle Mapper Compute SAM between region
mean vectors SAM(ui, uj)
between the region mean vectors
ui and uj no yes
L(Ri) = L(Rj) = k’
ui · uj
SAM(ui , uj ) = arccos( )
ui 2 uj 2 no (card(Ri) > M) & yes DC = (2 – max(Pk’(Ri),
(card(Rj) > M) Pk’(Rj)))SAM(ui, uj)
If adjacent regions have equal class DC = ∞
labels → they belong more likely to
DC = (2 – min(PL(Rj)(Ri),
the same region: PL(Ri)(Rj)))SAM(ui, uj)
DC = (2−max(Pk (Ri ), Pk (Rj ))) ·
SAM(ui , uj )
DC
If two large regions are assigned to
different classes → they cannot be
merged together
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 13
19. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
2 Calculate Dissimilarity Criterion
between adjacent regions:
Compute Spectral Angle Mapper Compute SAM between region
mean vectors SAM(ui, uj)
between the region mean vectors
ui and uj no yes
L(Ri) = L(Rj) = k’
ui · uj
SAM(ui , uj ) = arccos( )
ui 2 uj 2 no (card(Ri) > M) & yes DC = (2 – max(Pk’(Ri),
(card(Rj) > M) Pk’(Rj)))SAM(ui, uj)
If adjacent regions have equal class DC = ∞
labels → they belong more likely to
DC = (2 – min(PL(Rj)(Ri),
the same region: PL(Ri)(Rj)))SAM(ui, uj)
DC = (2−max(Pk (Ri ), Pk (Rj ))) ·
SAM(ui , uj )
DC
If two large regions are assigned to
different classes → they cannot be
merged together
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 13
20. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
2 Calculate Dissimilarity Criterion
between adjacent regions:
Compute Spectral Angle Mapper Compute SAM between region
mean vectors SAM(ui, uj)
between the region mean vectors
ui and uj no yes
L(Ri) = L(Rj) = k’
ui · uj
SAM(ui , uj ) = arccos( )
ui 2 uj 2 no (card(Ri) > M) & yes DC = (2 – max(Pk’(Ri),
(card(Rj) > M) Pk’(Rj)))SAM(ui, uj)
If adjacent regions have equal class
labels → they belong more likely to DC = ∞
the same region
DC = (2 – min(PL(Rj)(Ri),
If two large regions are assigned to PL(Ri)(Rj)))SAM(ui, uj)
different classes → they cannot be
merged together DC
If two regions have different class
labels → DC between them is
penalized by
(2 − min(PL(Rj ) (Ri ), PL(Ri ) (Rj )))
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 14
21. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
Hyperspectral image
3 Find the smallest dissimilarity criterion DCmin Preliminary
probabilistic
classification
Each pixel =
one region
1
2
3 While
not converged
Find min(DC) between
4
5
6 all pairs of spatially
adjacent regions (SAR)
Merge all pairs of SAR
with DC = min(DC).
7
8
9
Classify new regions
DCmin Spectral-spatial
10
11
12
classification map
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 15
22. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
4 Merge all pairs of spatially adjacent regions Hyperspectral image
with DC = DCmin . Preliminary
probabilistic
classification
Each pixel =
For each new region Rnew = Ri + Rj : one region
While
not converged
Pk (Ri )car d(Ri ) + Pk (Rj )car d(Rj ) Find min(DC) between
Pk (Rnew ) = all pairs of spatially
adjacent regions (SAR)
car d(Rnew )
Merge all pairs of SAR
with DC = min(DC).
Classify new regions
L(Rnew ) = arg max{Pk (Rnew )}
k Spectral-spatial
classification map
All the pixels in Rnew get a definite class label.
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 16
23. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
4 Merge all pairs of spatially adjacent regions
with DC = DCmin .
1
2
3
For each new region Rnew = Ri + Rj :
4
5
6
Pk (Ri )car d(Ri ) + Pk (Rj )car d(Rj )
Rnew
8
9
Pk (Rnew ) =
car d(Rnew )
11
12
L(Rnew ) = arg max{Pk (Rnew )}
k
All the pixels in Rnew get a definite class label.
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 16
24. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Hierarchical step-wise optimization with classification
Hyperspectral image
Preliminary
2 Calculate Dissimilarity Criterion between probabilistic
classification
adjacent regions
Each pixel =
one region
3 Find the smallest dissimilarity criterion DCmin While
not converged
Find min(DC) between
all pairs of spatially
4 Merge all pairs of spatially adjacent regions adjacent regions (SAR)
with DC = DCmin Merge all pairs of SAR
with DC = min(DC).
Classify new regions
5 Stop if all n pixels get a definite class label. Spectral-spatial
classification map
If not converged, go to step 2
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 17
25. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Classification maps
SVM Proposed HSwC method
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 18
27. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Conclusions and perspectives
Conclusions
1 New spectral-spatial classification method for hyperspectral images
was proposed
2 New dissimilarity criterion between image regions was defined
3 The proposed method:
improves classification accuracies
provides classification maps with homogeneous regions
Perspectives
Explore further the choice of:
optimal representative features for segmentation regions
dissimilarity measures between regions
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 20
28. Introduction
Proposed spectral-spatial classification scheme
Conclusions and perspectives
Thank you for your attention!
Yuliya Tarabalka and James C. Tilton (yuliya.tarabalka@nasa.gov) Best merge region growing with integrated classification for HS data 21
29. Best Merge Region Growing
with Integrated Probabilistic Classification
for Hyperspectral Imagery
Yuliya Tarabalka and James C. Tilton
NASA Goddard Space Flight Center,
Mail Code 606.3, Greenbelt, MD 20771, USA
e-mail: yuliya.tarabalka@nasa.gov
July 28, 2011