1. Similarity measurement:Folksonomyvs.LSA Preliminary Results

2. The Tripartite structure of tagging Folksonomy is a set of triples < user , tag, object> A folksonomy is a tuple F :=(U, T, R, Y) where U, T, and R are finite sets, whose elements are called users, tags and resources. Y is a ternary relation between user, tags and resources.

3. Del.icio.us Tag distribution Tag distribution Log-log Tag distribution After crawling the delicious.com site, the total number tags (tokens) obtained was 7,528,528, among which the number of types was 188,964. All the tags are stemmed using the Porter Stemmer and the total number of stemmed tags ended up to be 174,887.

4. LSA Processing Workflow in R tm = textmatrix(‘dir/‘) tm = lw_logtf(tm) * gw_idf(tm) space = lsa(tm, dims=dimcalc_share()) as.textmatrix(tm)

5. LSA corpus preparing A total number of 17,085 web pages were crawled and were later parsed to remove all the HTML markups. Stemming and Stop-word removal The processed corpus:14,993,620 tokens, 259,464 types of words. Only words with frequency more than 100 were kept to be entered into a word-by-document matrix. There were 1047 words with frequency more than 100. The text length kept in the corpus is from ranges from 51 words per document to 4009 words per document . Therefore, the resulting term-document matrix have 3465 columns (documents) and 1082 rows (words).

6. LSA document length distribution The text length kept in the corpus is from ranges from 51 words per document to 4009 words per document .

7. Three similarity measurements Tag Co-coccurrence counts Tag vector cosine similarity LSA

8. Similarity Measurement Tag Co-occurrence Counts: 1)simple count: how many times two tags are used by the same user to annotate the same resource 2)normalized count: Jaccard IndexThe co-occurrence counts of tag A and tag B divided by the joint frequency of A and B.

9. Distribution of Tag co-occurrence Counts (simple counts)

10. Distribution of Tag co-occurrence Counts (normalized)

11. Measurement 2: Cosine Similarity Based on the co-occurrence vector of each tag with every other tag. Since there are normalized and unnormalized tag co-occurrence counts, we then ended up with two X and Y are the co-occurrence vectors of two distinct tags.

12. Distribution of Tag Cosine Similarity

13. Distribution of Tag Cosine Similarity based on normalized Tag Co-occurrence counts

14. Results The pair-wise Pearson correlation and Spearman correlation among 5 measurements [Tag-cooccurrence count, Tag cosine similarity, LSA, normalized Tag-cooccurrence count, Tag cosine similarity based on normalized tag-tag cooccurrence matrix]

15. Correlation

21. Qualitative Insight – ‘linguistics’ Top 10 “linguistics” related words according to 5 measurements

22. Correlation between two measurements:normalized tag co-occurrence counts vs. normalized tag cosine P= 0.7547