7. A Impossibility Theorem for Clustering
◦ Jon Kleinberg, NIPS 2002
Measures of Clustering Quality: A Working Set
of Axioms for Clustering
◦ M.Ackerman and S.Ben-David, NIPS 2008
Characterization of Linkage-based
Clustering.
◦ M.Ackerman and S.Ben-David, COLT 2010
8. X:
+
d : X × X →R ,d ( x, x ) = 0(∀x ∈ X )
X,d )
( clustering
( X,d,k )
F
clustering
€ C = F ( X,d,k )
€ ⎛ € ⎞
{C1,C2 ,Ck }⎜ Ci = X,1 ≤ k ≤ X ⎟
⎝ i ⎠
9. general clustering function F
Input: F ( X,d )
⎛ ⎞
Output:
C = {C1,C2 ,Ck }⎜ Ci = X,1 ≤ k ≤ X ⎟
⎝ i ⎠
€
k-clustering function F
€
Input: F ( X,d,k ), (1 ≤ k ≤ X )
⎛ ⎞
Output:
= {C1,C2 ,Ck }⎜ Ci = X,1 ≤ k ≤ X ⎟
C
⎝ i ⎠
€
€
10.
Properties of Clustering Functions
A taxonomy of k-clustering fucntions
11. iso. invariance
scale invariance
order invariance
outer consistency
cluster
inner consistency
cluster
k-rich
k
inner rich
richness
outer rich
threshold rich
locality
clustering
refinement-confined
k clustering
12. clustering
φ : X → X ʹ′
x, y ∈ X,d ( x, y ) = dʹ′(φ (x), φ (y))
F ( X,d,k ),F ( X ʹ′, dʹ′,k ) : isomorphic(∀k)
x, y : same → φ (x), φ (y) : same
20. cluster
cluster clustering
cluster cluster
dʹ′ : (C,d ) − consistent
x, y : same → dʹ′( x, y ) ≤ d ( x, y )
x, y : different → dʹ′( x, y ) ≥ d ( x, y )
24. cluster clustering
cluster
dʹ′ : (C,d) − innterconsistent
x, y : same → dʹ′( x, y ) ≤ d ( x, y )
x, y : different → dʹ′( x, y ) = d ( x, y )
25. clustering
any : X1, X 2 X k
X ʹ′ = { X1, X 2 X k }
→∃d : F ( X ʹ′,d,k ) = { X1, X 2 X k }
26. clustering
※ clustering
any : (X1,d1 ),(X 2 ,d2 )(X k ,dk )
⎛ k ⎞
→∃d : F ⎜ X i , d,k ⎟ = { X1, X 2 X k }
ˆ ˆ
⎝ i=1 ⎠
ˆ
d : entends − d (i ≤ k)
i
27. clustering
※ clustering
( X,d), X = { X1, X 2 X k }
ˆ ˆ (
→∃d : d ( a,b) = d ( a,b) a ∈ X i ,b ∈ X j ,i ≠ j )
⎛ k ⎞
F ⎜ X i , d,k ⎟ = { X1, X 2 X k }
ˆ
⎝ i=1 ⎠
28. clustering
∃a < b
x, y : same →d(x, y) ≤ a,
x, y : different →d(x, y) ≥ b
F ( X,d, C ) = C
29. k≦k’ F(X,d,k’) F(X,d,k)
refinement
1 ≤ k ≤ kʹ′ ≤ X ,
O( F ( X,d,k')) ≥ O( F ( X,d,k ))
€
33. single linkage clustering
stop condition
Consistency + Richness: only link if distance is less than r
◦ clustering
cluster
Consistency + SI: stop when you have k connected
components
◦ clustering /
clustering
Richness + SI: if x is the diameter of the graph, only add
edges with weight βx
◦ cluster /
clustering
34.
Properties of Clustering Functions
A taxonomy of k-clustering fucntions
40.
Properties of Clustering Functions
A taxonomy of k-clustering fucntions
41. Invariance properties
Consistency properties
(C,d) − nice var iant
[ ] [
P F ( X, dʹ′, C ) = C ≥ P F ( X,d, C ) = C ]
Richness properties
∀ε > 0
€
∃d : P ( F ( X,d,k ) = C ) ≥1 − ε
Locality
[ ]
P F ( X ʹ′,d / X ʹ′, Cʹ′ ) = Cʹ′
€
=
[
P Cʹ′ ⊆ C F ( X,d, j ) = CandC / X ʹ′isak − clustering ]
[ ]
P ∃C1,C2 Ck s.t.Ci = X ʹ′ F ( X,d, j ) = C ≠ 0
