Eurolan 2005 Pedersen

Language Independent Methods of Clustering Similar Contexts (with applications) Ted Pedersen University of Minnesota, Duluth http://www.d.umn.edu/~tpederse [email_address]

The Problem ,[object Object],[object Object],[object Object],[object Object],[object Object]

Language Independent Methods ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Outline (Tutorial) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Outline (Practical Session) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

SenseClusters ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Many thanks… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Practical Session ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Headed and Headless Contexts ,[object Object],[object Object],[object Object],[object Object]

Headed Contexts (input) ,[object Object],[object Object],[object Object],[object Object],[object Object]

Headed Contexts (output) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Headless Contexts (input) ,[object Object],[object Object],[object Object],[object Object],[object Object]

Headless Contexts (output) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Applications ,[object Object],[object Object],[object Object],[object Object],[object Object]

Applications ,[object Object],[object Object],[object Object],[object Object]

Underlying Premise… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Identifying Lexical Features Measures of Association and Tests of Significance

What are features? ,[object Object],[object Object],[object Object]

Where do features come from? ,[object Object],[object Object],[object Object],[object Object]

Feature Selection ,[object Object],[object Object],[object Object],[object Object],[object Object]

Lexical Features ,[object Object],[object Object],[object Object],[object Object],[object Object]

Bigrams ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Co-occurrences ,[object Object],[object Object],[object Object],[object Object],[object Object]

Bigrams and Co-occurrences ,[object Object],[object Object],[object Object],[object Object],[object Object]

“ occur together more often than expected by chance…” ,[object Object],[object Object],[object Object],[object Object],[object Object]

2x2 Contingency Table 100,000 300 !Artificial 400 100 Artificial !Intelligence Intelligence

2x2 Contingency Table 100,000 99,700 300 99,600 99,400 200 !Artificial 400 300 100 Artificial !Intelligence Intelligence

2x2 Contingency Table 100,000 99,700 300 99,600 99,400.0 99,301.2 200.0 298.8 !Artificial 400 300.0 398.8 100.0 000.12 Artificial !Intelligence Intelligence

Interpreting the Scores… ,[object Object],[object Object]

Interpreting the Scores… ,[object Object],[object Object],[object Object]

Measures of Association ,[object Object],[object Object],[object Object]

Measures Supported in NSP ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

NSP ,[object Object],[object Object],[object Object],[object Object],[object Object]

Summary ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Related Work ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Context Representations First and Second Order Methods

Once features selected… ,[object Object],[object Object],[object Object]

First Order Representation ,[object Object],[object Object],[object Object]

Contexts ,[object Object],[object Object],[object Object],[object Object]

Unigram Feature Set ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

First Order Vectors of Unigrams 1 0 1 0 1 C4 0 0 0 0 0 C3 1 1 0 1 0 C2 1 1 1 1 1 C1 child magic curse black island

Bigram Feature Set ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

First Order Vectors of Bigrams 1 0 1 1 0 C4 0 1 1 0 0 C3 1 0 0 0 1 C2 1 0 0 1 1 C1 voodoo child serious error military might island curse black magic

First Order Vectors ,[object Object],[object Object],[object Object],[object Object],[object Object]

Second Order Representation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Word by Word Matrix 120.0 0 69.4 0 0 voodoo 0 89.2 0 21.2 0 serious 0 54.9 100.3 0 0 military 73.2 0 0 189.2 0 island 43.2 0 0 0 123.5 black child error might curse magic

Word by Word Matrix ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

There was an island curse of black magic cast by that voodoo child. 120.0 0 69.4 0 0 voodoo 73.2 0 0 189.2 0 island 43.2 0 0 0 123.5 black child error might curse magic

Second Order Representation ,[object Object],[object Object]

There was an island curse of black magic cast by that voodoo child. 78.8 0 24.4 63.1 41.2 C1 child error might curse magic

First versus Second Order ,[object Object],[object Object],[object Object],[object Object]

Second Order Co-Occurrences ,[object Object],[object Object]

Second Order Co-occurrences ,[object Object],[object Object],[object Object],[object Object],[object Object]

Summary ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Related Work ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Dimensionality Reduction Singular Value Decomposition

Motivation ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Many Methods ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Effect of SVD ,[object Object],[object Object]

Effect of SVD ,[object Object],[object Object],[object Object]

How can SVD be used? ,[object Object],[object Object],[object Object],[object Object],[object Object]

Word by Word Matrix 4 2 0 0 0 3 0 1 box 0 1 2 2 1 2 0 0 memory 0 0 0 1 0 0 2 0 organ 0 2 0 3 2 0 0 0 debt 0 1 0 3 1 0 0 2 linux 0 1 0 3 2 0 0 0 sales 3 0 2 2 0 3 0 0 lab 1 0 2 0 0 1 2 0 petri 0 1 0 0 2 0 0 1 disk 1 0 2 0 0 0 3 0 body 0 0 0 3 1 0 0 2 pc plasma graphics tissue data ibm cells blood apple

Singular Value Decomposition A=UDV’

U -.52 .39 -.48 .02 .09 .41 -.09 .40 -.30 .08 .31 .43 -.26 -.39 -.6 .20 .00 -.00 -.00 -.02 -.01 .00 -.02 -.00 -.07 -.3 .14 -.49 -.07 .30 .25 .56 -.01 .08 .05 -.01 .24 -.08 .11 .46 .08 .03 -.04 .72 .09 -.31 -.01 .37 -.07 .01 -.21 -.31 -.34 -.45 -.68 .29 .00 .05 .83 .17 -.02 .25 -.45 .08 .03 .20 -.22 .31 -.60 .39 .13 .35 -.01 -.04 -.44 .08 .44 .59 -.49 .05 -.02 .63 .02 -.09 .52 -.2 .09 .35

D 0.00 0.00 0.00 0.66 1.26 2.30 2.52 3.25 3.99 6.36 9.19

V -.20 .22 -.07 -.10 -.87 -.07 -.06 .17 .19 -.26 .04 .03 .17 -.32 .02 .13 -.26 -.17 .06 -.04 .86 .50 -.58 .12 .09 -.18 -.27 -.18 -.12 -.47 .11 -.03 .12 .31 -.32 -.04 .64 -.45 -.14 -.23 .28 .07 -.23 -.62 -.59 .05 .02 -.12 .15 .11 .25 -.71 -.31 -.04 .08 .29 -.05 .05 .20 -.51 .09 -.03 .12 .31 -.01 .02 -.45 -.32 .50 .27 .49 -.02 .08 .21 -.06 .08 -.09 .52 -.45 -.01 .63 .03 -.12 -.31 .71 -.13 .39 -.12 .12 .15 .37 .07 .58 -.41 .15 .17 -.30 -.32 -.27 -.39 .11 .44 .25 .03 -.02 .26 .23 .39 .57 -.37 .04 .03 -.12 -.31 -.05 -.05 .04 .28 -.04 .08 .21

Word by Word Matrix After SVD 1.1 1.0 .98 1.7 .86 .72 .85 .77 memory .00 .00 .17 1.2 .77 .00 .84 .00 organ .00 1.5 .00 3.2 2.1 .00 .00 1.2 debt .13 1.1 .03 2.7 1.7 .16 .00 .96 linux .41 .85 .35 2.2 1.3 .39 .15 .73 sales 2.3 .18 2.5 1.7 .35 2.0 1.7 .21 lab 1.4 .00 1.5 .49 .00 1.2 1.1 .00 germ .00 .91 .00 2.1 1.3 .01 .00 .76 disk 1.5 .00 1.6 .33 .00 1.3 1.2 .00 body .09 .86 .01 2.0 1.3 .11 .00 .73 pc plasma graphics tissue data ibm cells blood apple

Second Order Representation ,[object Object],[object Object],[object Object],[object Object],1.0 .72 memory .00 .00 organ .13 1.1 .03 2.7 1.7 .16 .00 .96 linux .00 .91 .00 2.1 1.3 .01 .00 .76 disk Plasma graphics tissue data ibm cells blood apple

Clustering Methods Agglomerative and Partitional

Many many methods… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

General Methodology ,[object Object],[object Object],[object Object],[object Object]

Agglomerative Clustering ,[object Object],[object Object],[object Object]

Measuring Similarity ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Agglomerative Clustering ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Average Link Clustering 1 2 4 S3 1 2 4 S3 0 2 S4 0 3 S2 2 3 S1 S4 S2 S1 0 S4 0 S2 S1S3 S4 S2 S1S3 S4 S1S3S2 S4 S1S3S2

Partitional Methods ,[object Object],[object Object],[object Object],[object Object],[object Object]

Partitional Methods ,[object Object],[object Object]

Results of Clustering ,[object Object],[object Object],[object Object],[object Object]

Label Types ,[object Object],[object Object]

Evaluation Techniques Comparison to gold standard data

Evaluation ,[object Object],[object Object],[object Object],[object Object]

Evaluation ,[object Object],[object Object],[object Object]

Baseline Algorithm ,[object Object],[object Object]

Baseline Performance ,[object Object],170 55 35 80 Totals 170 55 35 80 C3 0 0 0 0 C2 0 0 0 0 C1 Totals S3 S2 S1 170 80 35 55 Totals 170 80 35 55 C3 0 0 0 0 C2 0 0 0 0 C1 Totals S1 S2 S3

Evaluation ,[object Object],[object Object],[object Object],[object Object],[object Object],170 55 35 80 Totals 65 10 5 50 C3 60 40 0 20 C2 45 5 30 10 C1 Totals S3 S2 S1

Evaluation ,[object Object],[object Object],[object Object],170 80 55 35 Totals 65 50 10 5 C3 60 20 40 0 C2 45 10 5 30 C1 Totals S1 S3 S2

Analysis ,[object Object],[object Object],[object Object],[object Object]

Practical Session Experiments with SenseClusters

Experimental Data ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Creating Experimental Data ,[object Object],[object Object],[object Object],[object Object],[object Object]

Name Conflation Data ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Clustering Contexts ,[object Object],[object Object],[object Object],[object Object]

Headed Clustering ,[object Object],[object Object],[object Object]

Headless Contexts ,[object Object],[object Object]

If you after all these matrices you crave knowledge based resources… Read on…

WordNet-Similarity ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Many thanks! ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Vector measure ,[object Object],[object Object],[object Object],[object Object],[object Object]

Many other measures ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Thank you! ,[object Object],[object Object],[object Object]

Eurolan 2005 Pedersen

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (6)

Ähnlich wie Eurolan 2005 Pedersen

Ähnlich wie Eurolan 2005 Pedersen (20)

Mehr von University of Minnesota, Duluth

Mehr von University of Minnesota, Duluth (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Eurolan 2005 Pedersen