Did you have 20 opened Chrome tabs and feel frustrated about research? This is a problem more than organising tabs. In this talk, I will introduce how to combine human and machine intelligence (yes, not machine alone) to address this challenge.
DATA SUMMIT 24 Building Real-Time Pipelines With FLaNK
Making Sense of (Big) Data with Visual Analytics
1. Making Sense of (Big) Data
with Visual Analytics
Dr Kai Xu
Associate Professor in Data Analytics
Middlesex University, London, UK
k.xu@mdx.ac.uk https://kaixu.me
2. Outline
• What is Sensemaking
• Why do we need Visual Analytics
• Demo – SAVI: Social Analytics Visualisation
• Demo – SenseMap: A ‘Map’ for Sensemaking
3. What is Sensemaking?
• Making sense of data
• Collecting, understanding, analysing, reasoning, and
making decisions
• It is something we do everyday:
• Plan a holiday, buy a house, understand an illness, …
• Defence, policing, investment, medical diagnosis, …
• How is it different from data analysis?
• The task is usually not well defined
4. Example: what is the best camera for about £500?
What is the best
camera for £500?
Pixel number
Sensor size
Image quality
chromatic
aberration?!
Noise
reduction
What does
experts say?
Online reviews
What does my
friend say? Smart phone
Compact
Full frame?
Micro 4/3?Sony RX100
Nikon D750Samsung
Galaxy S7
What are the
price?
How do I
compare? Panasonic
LX100
Form factor
Models
Camera Lens
Aperture
5. This is usually what it looks like after one hour
• What is relevant and what is not?
• Where is the information about image quality?
• How to compare the models?
• Where did I left off two days ago?
• How do I explain to my wife?
8. Why IBM Watson or AlphaGo can’t do it
• Watson is good at:
• Natural language processing, e.g., understand the Jeopardy! Questions
• Find the (relevant) fact quickly
• However, the £500 camera task is
• Every personal, Watson need all the information about me and understand it
• No ‘best’ answer, so can’t just search it
• For AlphaGo, the Go game is very complex and difficult, but
• The goal and rules are very well defined, and the results are easily measurable
• However, the £500 camera task is ill defined and not easily measurable
• How many people have the knowledge and resource to build a deep neural network,
collect all the training data, and then train and tune it, just to find a camera?
9. Who is the best chess player in the world?
• Deep Blue, was in 1997
• Currently, probably a human-machine
team
• And the two people on the team are not
even professional chess players
• The power of integrating the
complementary strength of human and
machine
10. Visual Analytics = Human + Computing Intelligence
Visualisation
Data
Analysis
Interaction
Information Retrieval
Machine Learning
Data Mining
Information Visualisation
Scientific Visualisation
Computer Graphics
Human-Computer Interaction
Cognitive Psychology
Perception
12. Example - SAVI: Social Analytics Visualisation
• IEEE Visual Analytics Science & Technology (VAST) Challenge
• Provide dataset and analysis tasks
• Entry: visual analytics systems
• Leading research groups and companies
• VAST Challenge 2014 – Mini Challenge 3
• Data: tweets
• Task: detect and describe a crime
17. The Final Findings
• Still a long time before AI can do such
sensemaking
• Difficult for human, too: almost impossible
without the tool
• Human leads, the tool supports
• The tool does not provide answer,
• Reveal pattern, help with organisation and
reasoning, and many more
18. A ‘Map’ of Sensemaking
• Sensemaking is kind of like exploring a maze …
• What may be helpful is something like this …
19. SenseMap – A ‘Map’ for Online Sensemaking
Browser enhancement
History
Map
Knowledge
Map
22. Takeaway Messages
• Sensemaking is how people understand, reason, and make decisions
with data
• It is important to Big Data, but there is limited support available
• Visual Analytics combines data visualisation with analytics
• A promising approach for sensemaking support
More details about SAVI and SenseMap: http://vis4sense.github.io/
Editor's Notes
A Sense Making model:
Using literature review as an example
from search & filter and then jump to hypothesis & presentation.
For big data, most focus on the early stages of sense making: we have storage, search, and analysis; Relatively very little on the further stages
This is the focus of our research: Not all the stages at the same time yet; Mostly in the defence and intelligence domain.
Various funding sources: EPSRC, dstl, UK Government, EU
No, we were not involved in the NSA
One example