2. Motivation
 Necessity of combining physical models and
statistical techniques in unmixing
 Methods of integration of scientific models &
statistical algorithms not always obvious
– Enable machine learning techniques to produce scientifically
meaningful results in unsupervised settings
 Meaningfulinformation not always readily
accessible or easily readable
– Representation = Simplification

3. Outline
 Background
– Mineral spectral signatures in VNIR
 Meaningful features for mineral identification
 Data challenges for planetary data
 Data processing pipeline
 Validation: expert assessment
 Comparison with state-of-the-art
 Conclusions and future work

4. Mineral spectral signatures
 Each mineral has
a distinct spectral
shape (signature)
 Discriminative
information mostly
in absorption
band positions
and shapes
 Difference can be
subtle
 Can create
parameters for
discrimination

5. Mineral spectral signatures
 Each mineral has
a distinct spectral
shape (signature)
 Discriminative
information mostly
in absorption
band positions
and shapes
 Difference can be
subtle
 Can create
parameters for
discrimination

6. Spectral features for minerals
 Use splines
 Select knots so that

7. Spectral features for minerals
 Use splines
 Select knots so that
– Reconstruction insensitive to
artifacts

8. Spectral features for minerals
 Use splines
 Select knots so that
– Reconstruction insensitive to
artifacts
– Reconstruction with higher
sensitivity in diagnostic areas

9. Spectral features for minerals
 Use splines
 Select knots so that
– Reconstruction insensitive to
artifacts
– Reconstruction with higher
sensitivity in diagnostic areas
– Reconstruction sharper
(green) for vibrational bands
and smoother (red) for
electronic transition bands
 B-splinecoefficients as
feature vector

10. hyperspectral data challenges
 Cloud has curved
boundaries and is non
convex
 Some dimensions
uninformative
 No apparent clusters, high
density
 Noise creates outliers
 Most unique spectra =
extreme points or “corners”
or “image endmembers”

11. Objectives for spectral unmixing
 Separation of spectral pixels in families (image
segmentation)
 Sensitivity to subtle changes in spectral absorption
positions and shapes (mineral sub-families)
 Sensitivity to small (spatial) outcrops
 Robustness with respect to noise
 Useful visualization

12. Pipeline
1 2
1
Preprocessing
1
2
3 2
4
1
5
Dimensionality 2
reduction
3
3
4 4
4
Clustering
3
Unmixing
Pruning
1 2 1 2
5 1 1

13. Pipeline
1 2
1
Preprocessing
1
2
3 2
4
1
5
Dimensionality 2
reduction
3
3
4 4
4
Clustering
3
Unmixing
Pruning
1 2 1 2
5 1 1

14. Operation modes
100 pixels
50 pixels
 Two capabilities:
– Select areas based on parameter maps (user version)
– Divide the image in sections (pipeline version)
 Operate on each area independently

15. Pipeline
1 2
1
Preprocessing
1
2
3 2
4
1
5
Dimensionality 2
reduction
3
3
4 4
4
Clustering
3
Unmixing
Pruning
1 2 1 2
5 1 1

16. Dimensionality reduction: issues
 Intrinsic dimensionality of data is low: benefit from
dimensionality reduction.
 From movie:
– Need nonlinear transform.
– Need to preserve local geometry
– Need to highlight natural clusters
 Reduce dimensionality to 2 – 3 for visualization

17. Dimensionality reduction
High-D Feature Space
Low-D Space
x1 , . . . , xn(known) as vertices y1 , . . . , yn (unknown) as
of a graph
vertices of a graph
Edge weights proportional to Edge weights fixed
spectral dissimilarities and spatial
adjacency
x1 x3
y1 y3
x2 y2
yn
xn

18. Dimensionality reduction
High-D Feature Space
Low-D Space
Small distance = small distance
p12 q12
x1 x3
y1 y3
x2 y2
yn
xn

19. Dimensionality reduction
High-D Feature Space
Low-D Space
Med/big distance = bigger distance
p13 q13
x1 x3
y1 y3
x2 y2
yn
xn

20. Dimensionality reduction
High-D Feature Space
Low-D Space
P = {pij } Q = {qij }
x1 x3
y1 y3
x2 y2
yn
xn

21. Correl. Neighbor Embedding (Parente 2011)
High-D Space
Low-D Space
cij = αij,spatial · cij,spect
dij = 1 − cij
exp(−d2 /2σi2 )
ij
pij = 2 2
k=l exp(−dkl /2σ )
k
pij (xi , xj )
 Minimize relative entropy
D = argmin pij (xi , xj ) log
yi ,yj
i,j
qij (yi , yj )
 Solve by gradient descent
∂D(P ||Q) = 4 κij (yi − yj )(pij − qij )
∂yi j
 Variation on t-Stochastic Neighbor Embedding (Van der
Maaten et al. 2008)

22. Pipeline
1 2
1
Preprocessing
1
2
3 2
4
1
5
Dimensionality 2
reduction
3
3
4 4
4
Clustering
3
Unmixing
Pruning
1 2 1 2
5 1 1

23. Graph partitioning as clustering
 Cluster points in the transformed
space to take advantage of separated
sections
 The geometry is nonlinear: need
clustering on curved structure
 Consider the set of vertices yi of the
graph and the edge weights qij (yi , yj )
 Clustering is equivalent to partitioning
graph into disjoint subsets.
– can be done by spectral clustering
because CNE creates several
connected components

24. Image segmentation
Original Segmentation
Clustering = mineral image
map
family mapping =
image
segmentation

25. Pipeline
1 2
1
Preprocessing
1
2
3 2
4
1
5
Dimensionality 2
reduction
3
3
4 4
4
Clustering
3
Unmixing
Pruning
1 2 1 2
5 1 1

26. Local endmember detection
1
2
4
3
 The clusters in the original space are roughly convex
 Locally to a cluster can assume linear mixing
 approximate the data cloud with a conic or convex
combination of a small number of “endmembers”
 Have a way to extrapolate endmembers if the data
does not support clear detections

27. Robust Nonneg. Matrix Factorization
minimize ϕ(Y − W H) + 2
λ||DW ||F
subject to W,H ≥ 0, 1T H = 1 T
W ∈ Rm×k , H ∈ Rk×n
 k is the number of local endmembers
 ϕ is a robust estimator
 D imposes smoothness and corrects MNF “problems”
 Solve with alternating projected gradient
 Zymnis 2009, Parente 2009, Parente 2011

28. Pipeline
1 2
1
Preprocessing
1
2
3 2
4
1
5
Dimensionality 2
reduction
3
3
4 4
4
Clustering
3
Unmixing
Pruning
1 2 1 2
5 1 1

29. Spectral pruning
Features for Features for
spectrum 1
spectrum 2
Cross-correlate
 Spectrum 1 is any local
feature vectors
endmember candidate
 Spectrum 2 is either a
local endmember or an
estimate of the baricenter
of the cloud
 If the score is higher than
a threshold Spectrum 1 is
pruned
Score

30. Validation
 Martian image analysis lacks ground truth
 Simulation of the complete hyperspectral image
formation process (Parente et al. 2010)
– Soil mixing, atmosphere, instrument response, noise
 Comparison with manual expert assessment
– the expert can extract the complete spectral variability (3E12)
– the expert can only extract partial spectral variability (94F6)
– The expert cannot extract spectral variability
 Self-consistency: comparison with state-of the
art (partial)

31. Validation: 3E12
 Different mineral
families evident
from RGB
 Low noise
 Good spectral
variability

32. Validation: 3E12
Automatically retrieved Manually selected spectra
spectra over the whole over the whole scene
scene

33. Validation: 94F6
R=band 233, G=band 78, B=band 13
R=D2300, G=OLINDEX, B=BD2210

34. 94F6 manual retrieval
Regions of Interest (ROIʼs)
Spectra from ROIʼs

35. 94F6
 Several more spectral families
detected by the algorithm
 Letʼs zoom in!

36. 94F6 automated

37. 94F6 automated
 2.205 µm

38. 94F6 automated
 2.205µm
 2.2913 µm

39. 94F6 automated
 2.205µm
 2.2913 µm
 2.3046 µm

40. 94F6 automated
 2.205µm
 2.2913 µm
 2.3046 µm
 2.3244 µm

41. 94F6 automated
 2.205µm
 2.2913 µm
 2.3046 µm
 2.3244 µm
 2.4038 µm

42. 94F6 automated
 2.205µm
 2.2913 µm
 2.3046 µm
 2.3244 µm
 2.4038 µm
 2.5229 µm

43. 94F6 automated
 2.205µm
 2.2913 µm
 2.3046 µm
 2.3244 µm
 2.4038 µm
 2.5229 µm
 2.5295 µm

44. 94F6 automated
 2.205µm
 2.2913 µm
 2.3046 µm
 2.3244 µm
 2.4038 µm
Carbonate !!
 2.5229 µm
 2.5295 µm

45. Some difficult data:199C7

46. 199C7 automated

47. 199C7 automated
 2.04 µm
 2.29 µm, 2.30 µm,
2.31 µm
 2.52 µm, 2.53 µm

48. Comparison with state of the art
 Current unmixing algorithms:
– require convexity
– developed for earth
 environmental conditions are known
 ground truth is available
– donʼt consider impulsive noise
– some require linear assumptions
 Nonlinear unmixing not yet mature
 Not able to discriminate subtle spectral differences

49. Comparison with other algorithms
The proposed algorithm is The SMACC algorithm is
insensitive to noise and extremely sensitive to noise
picks up more surface
components

50. B141 Mawrth Vallis
ENVI SMACC
endmembers
Proposed
approach

51. ABCB: Nili Fossae
Endmembers
Proposed from VCA
approach

52. More algorithms
(a) Proposed Algorithm
(b) N-FINDR
(c) PPI
(d) SMACC
(e) SISAL

53. Conclusions
 Presented a novel method for unmixing
 The algorithm effectively captures the image spectral
variability, down to subtle differences, is robust to noise and
outperforms current state-of-the-art algorithms
 Can be applied to any hyperspectral dataset
 Produces segmentation and endmember maps
 We proposed this technique to the CRISM and M3 teams
as the “official” data summarization tool for their processing
pipelines.

54. Future work
 Include a physical unmixing layer: use radiative
transfer theory
 Provide mechanism to tag “virtual” endmembers
 Complete validation process with expert feedback

55. References
 L. van deer Maaten and G. Hinton, (2008). Visualizing data using t-
SNE, Journal of Machine Learning, 9, pp. 2579-2605.
 A. Ng, M. Jordan and Y. Weiss, (2001). On spectral clustering:
Analysis and an algorithm, NIPS.
 M. Parente , J.T. Clark, A. Brown and J.L. Bishop (2010). End-to-
end simulation of the image generation process for CRISM
spectrometer data, IEEE Transactions on Geoscience and Remote
Sensing.
 M. Parente, (2011). Summarization of hyperspectral images:
application to Mars, IEEE Transactions on Geoscience and
Remote Sensing, (in review).
 M. Parente, J. L. Bishop and J. F. Bell III, (2009), Spectral unmixing
and anomaly detection for mineral identification in Pancam images
of Gusev soils, Icarus, Vol 203, N. 2, p. 421-436.

56. Questions?

57. Publications based on project
 Parente M. and A. Plaza (2010), Survey of geometric and statistical unmixing algorithms for
hyperspectral images, IEEE 2nd WHISPERS (Workshop on hyperspectral image and signal
processing: evolution of remote sensing) Conf. June 14-16, Reykjavyk, Iceland (invited keynote
presentation for special session on “Geometric vs. statistical unmixing algorithms”).
 M. Parente Spectral unmixing using nonnegative basis learning: comparison of geometrical and
statistical endmember extraction algorithms. (invited paper) Space Exploration Technologies,
edited by Wolfgang Fink Proc. of SPIE Vol. 6960, 69600P, (2008). doi: 10.1117/12.777895
 M. Parente Exploratory data analysis of planetary datasets – new development, (invited talk) Jet
Propulsion Laboratory, Pasadena CA, December 4 2008.
 Parente M., Clark J.T., Brown A.J., and Bishop J.L.. (2009). Simulation of the image generation
process for CRISM spectrometer data. IEEE WHISPERS (Workshop on hyperspectral image
and signal processing: evolution of remote sensing) Conf. Aug 26-28 Grenoble, France. (Best
paper award)
 Bishop J. L., Noe Dobrea E. Z., McKeown N. K., Parente M., Ehlmann B. L., Michalski J. R.,
Milliken R. E., Poulet F., Swayze G. A., Mustard J. F., Murchie S. L., and Bibring J.-., P. (2008)
Phyllosilicate diversity and past aqueous activity revealed at Mawrth Vallis, Mars. Science 321,
DOI: 10.1126/science.1159699, pp. 830-833.
 Parente, M. and J.L. Bishop, (2010). Extracting endmember spectra from CRISM images:
comparison of new Direx image transform technique with MNF, Lunar Planet Science Conf, XLI
abstr. #2633.

58. Backup slides

59. MRO-CRISM: VNIR Spectra Can Characterize
Small Deposits on Mars
Examples of surface features at different CRISM spatial
resolutions
• Global Mode: 70 channels
• Targeted Mode: 544 channels
OMEGA CRISM multispectral survey (100-200 CRISM targeted hyperspectral
(300-1000 m/pixel, 13 nm/ch.) m/pix, 70 ch.) discovers small (15-38 m/pixel, 6.55 nm/ch)
discovers large deposits deposits
characterizes deposits

60. CRISM Noise sources
1. Vertical striping due to
miscalibration of pixel sensors
(red arrows).
2. Pixels with elevated bias or
abnormal dark ("bad" pixels)
create stripe segments (cyan)
 Both artifacts create spikes
in the spectral domain
60/40

61. Noise removal with CIRRUS
Original
Cleaned
Original
 CIRRUS (CRISM Iterative
Recognition and Removal
of Unwanted Spiking)
Cleaned
(Parente 2008)
 CIRRUS currently in use in
CRISM processing pipeline

62. Comparison with PCA
Proposed approach (3D)
PCA (first 3 PCs)
 Natural clusters well  Natural clusters not
separated
evident
 Between-clusters,  similar points can
different spectra
differ in norm
 Within-cluster, similar  1st PC illumination
spectra
gradient

63. Comparison with other techniques
Proposed approach (3D)
PCA (first 3 PCs)
LLE (3D)
 Natural clusters well  Natural clusters not  Natural clusters not
separated
evident
evident
 Between-clusters,  similar points can  Some endmembers
different spectra
differ in norm
evident
 Within-cluster, similar  1st PC illumination  Clustering particularly
spectra
gradient
hard

64. Graph partitioning as clustering

65. Graph partitioning as clustering

66. Graph partitioning as clustering

67. Graph partitioning as clustering

68. Clustering for case study

69. Clustering performance comparison
Original Proposed K-means in K-means with Hierarchical in Hierarchical in
image
approach
original correlation in original space
3-D space
space
original space

70. K-Eigenvector Clustering
(Ng et al. 2001)
1. Construct matrix of normalized weights Aʼ
2. Decomposition: Find the eigenvectors of Aʼ
corresponding to the k largest eigenvalues.
These form the the columns of the new matrix X.
3. Form the matrix Y
– Renormalize each of Xʼs rows to have unit length
– Y |
– Treat each row of Y as a point in
3. Cluster into k clusters via k-means
4. Final Cluster Assignment
– Assign point to cluster j iff row i of Y was assigned to cluster j
k can be found by maximum spread between eigenvalues

71. Validation
 This software is undergoing extensive validation
ID Solicitation
aimed at confirming that the proposed method
can be used pervasively and reliably in the
summarization of the whole CRISM database.
 The validation process starts with requesting
Processing
from the community image IDʼs with manually
selected endmembers.
 An automated pipeline is in place that sends
back via email the spectra retrieved by the
Feedback
algorithm to each author of manual analysis.
 Upon receiving feedback on dissimilarities and
quality of the detections the pipeline will
Validation
calculate validation statistics and will send them
statistics to the team for review.
 After validation the production stage will begin.