This document summarizes various tools for visualizing and analyzing activity data. It discusses tools for data wrangling like Google Refine and DataWrangler. It also covers visualization libraries and platforms like Many Eyes, Matplotlib for time series data, and Graphviz for graphs and networks. Statistical analysis in R and graphics libraries like Protovis and Processing are also mentioned. The document provides links to examples of analyzing hierarchical data, text processing with Unix tools, and visualizing trends and autocorrelation in time series data.

1. Visualising Activity Data Tony Hirst Dept of Communication and Systems, The Open University Scattered puzzle pieces next to solved fragment by HoriaVarlan

2. Today’s link shortener is bit.ly Read: [ jlKwGq ] as: http://bit.ly/jlKwGq

3. Visual Analysisvs.Presentation Graphics

6. data preparation

8. ScraperWiki [ aGhJtK ]

9. Search and replace… …add regular expressions and you have search and replace “on steroids”

10. Google Refine [ aq1jUE ] Example: walkthrough (@jenit) [ awGQPT ] Example: merging two tables by column [ pWK3C0 ]

11. DataWrangler [ gmE3yz ]

12. Data has shape and structure

13. Hierarchical Data

14. Many Eyes [ qY5786 ] Treemaps

19. plot srcfile using ($1):(column(focusCar) -$2) with lines title "VET", srcfileusing ($1):(column(focusCar) -$3) with lines title "WEB", srcfileusing ($1):(column(focusCar) -$4) with lines title "HAM", srcfileusing ($1):(column(focusCar) -$5) with lines title "BUT", srcfileusing ($1):(column(focusCar) -$6) with lines title "ALO", srcfileusing ($1):(column(focusCar) -$7) with lines title "MAS", srcfileusing ($1):(column(focusCar) -$8) with lines title "SCH", srcfileusing ($1):(column(focusCar) -$9) with lines title "ROS", …

20. Or heatmaps in R: [ qXmPgs ]

21. Text processing with Unix tools[ m5tz63 ] [ lOVySX ] Count number of lines in a file: wc-l L2sample.csv View first few lines in a file: head L2sample.csv or head -n 4 L2sample.csv View last few lines in a file: tail L2sample.csv or tail -n 15 L2sample.csv Sample contiguous rows from start or end of file: head -n 1 L2sample.csv > headers.csv tail -n 20 L2sample.csv > subSample.csv cat headers.csvsubSample.csv > subSampleWithHeaders.csv Sample contiguous rows from middle of file: head -n 15 L2sample.csv | tail -n 6 > middleSample.csv Split large file into smaller files: split -l 15 L2sample.csv subSamples Search for lines containing a term: grepmendeley L2sample.csv grepEBSCO L2sample.csv > rowsContainingEBSCO.csv

22. More text processing tricks Extract columns: cut -f 3 L2sample.csv cut -f 1,2,14,17 L2sample.csv > columnSample.csv Sort data in a column: cut -f 40 L2sample.csv | sort Identify distinct entries in a column: cut -f 40 L2sample.csv | sort | uniq Count how many times each distinct term appears in a column: cut -f 40 L2sample.csv | sort | uniq –c Sort can also sort by column (-k), reverse order (-r): cut -f 40 L2_2011-04.csv | sort | uniq -c | sort -k 1 -r > uniqueSID.csv

23. [ dAdIo3 ]

25. Time series data

26. aka “seasonal subseries” [ j3HODr ]

28. matplotlib Trends [ qSIcrV ] #time series data in d #first difference fd=np.diff(d) Autocorrelation

29. Graphs and Networks

30. Graphviz digraph test { CSV [shape=box] KML [shape=box] JSON [shape=box] XML [shape=box] RDF [shape=box] HTML [shape=box] GoogleSpreadsheet[shape=Msquare] RDFTripleStore [shape=Msquare] "[SPARQL]" [shape=diamond] "[YQL]" [shape=diamond] "[GoogleVizDataAPI]" [shape=diamond] "<GoogleGadgets>" [shape=doubleoctagon] "<GoogleVizDataCharts>" [shape=doubleoctagon] "<GoogleMaps>" [shape=doubleoctagon] "<GoogleEarth>" [shape=doubleoctagon] "<JQueryCharts_etc>" [shape=doubleoctagon] "[SPARQL]"->RDF; "[SPARQL]"->XML; "[SPARQL]"->CSV; "[SPARQL]"->JSON; JSON-> "<JQueryCharts_etc>"; CSV->"{GoogleRefine}" CSV->ScraperWiki JSON->ScraperWiki "[YQL]"->ScraperWiki ScraperWiki->CSV HTML->ScraperWiki HTML->"[YQL]" "[SPARQL]"->"[YQL]" "{GoogleRefine}"->CSV [style=dashed] CSV->"<Gephi>" [style=dashed] "<Gephi>"->CSV [style=dashed] RDF->"[YQL]” }

31. Gephi

32. [ nKoB4b]

33. [ nKoB4b]

38. Statistical Graphs

39. R

41. Graphics Libraries

42. Protovis

43. Processing

45. I hope that’s beenouseful.info….?