This is your third research methods lecture. This week we look at Symbol Surfing Project, its setup, purpose, method, data collection and analysis.

1. New Media Research Methods
Part 1 – How research
methods relate to the
research question
Part 2- Qualitative and
Quantitative
Part 3 – Data collection and
analysis – Case Study
Example
Part 4 – Presentation and
analysis

2. Recap from last week
• Qualitative and Quantitative Research
Methods
• Interviews and surveys
• Types of questions
• Designing questionnaires and interviews
• Good practice

3. Project team: Nick Weldin, Karen Bunning
and
Gosia Kwiatkowska
Case Study – Symbol Surfing Project
Funded by Esmee Fairbairn Trust

4. Symbol Surfing Project - Background
• Ubiquity of computer
technology and the WWW
– access and participation
problems
• Accessibility issues –
keyboard/mouse
• Match between the
interface and the individual
capacity of the user

5. Symbol Surfing Project - Background
• The role of the user/the
role of the supporter
• Interactions:
student/supporter;
student/computer +
supporter/interface/com
puter
• Teacher initiating role to
align the student –
operational engagement

6. Symbol Surfing Project - Background
• Communication –
symbols, signs, pictures
e.g. Makaton, Widgits
etc
• Use user experience
(using symbols) but for
different purpose –
control the computer

7. Symbol Surfing Project - Background
• Benefits – low cost, using
existing skills, control, etc
• Exploring the use of freely
available symbol
recognition software for
PMLD community
• Accessing and controlling
computers by PMLD

8. Research questions
• What are the ways in which individuals are
able to use and engage with a simple symbol
recognition system to access media on a
computer?
• What role is played by the supporter during
user activity with this system?

9. Symbol Surfing Project - Method
• Design – an action research, ethics, consent
• Settings – specialised collage (2), secondary
part of special school (2), residential setting
(1). Supporters all knew the participants and
understood their communication e.g.
gestures, facial expressions etc
• Sample – five people with PMLD, age range 15
– 28, difficult to engage, English – first
language,

10. Symbol Surfing Project – Data
Collection
• Four visits, monthly intervals,
• Symbol surfing software was installed
• Two video cameras were used – one to capture the
screen and the other to capture the engagement

11. Symbol Surfing Project – Data
Collection
• Environment – familiar
• Researchers role –
marginal participants –
guiding the supporter if
required
• Field notes were
recorded
• Length of the sessions –
varied

12. Symbol Surfing Project – Data
Collection
• The formats : model, withdraw support, observe
• Number of cards used – varied
• Video footage – combined and displayed side by
side
• Sampling of video – 60 sec at 2 min intervals
• Data transcribed through repeated viewings
1 3 5 7 9 11 13 15
17

13. Symbol Surfing Project – Data
Collection
• Transcriptions template – actions participant/
supporter/ researcher
• Actions:
– vocal,
– non vocal communication behaviours,
– computer related actions,
– contextual information

14. Symbol Surfing Project –Data Analysis
• Coding framework – structural linguistics – to
capture user attempts at manipulating the
symbol tools, but also recording user
responses to the activity and the facilitation
role performed by the supporter
• Initiating moves, response moves, pre-
initiating moves and self expression

15. Symbol Surfing Project –Data Analysis
• Refining the coding framework – viewing
video and observing behaviours
• Definition for each category with examples

16. Coding Framework Used

19. • Transcriptions
reviewed with the
video footage
• Code annotations
applied to transcripts
Symbol Surfing Project –Data Analysis

20. Symbol Surfing Project –Data Analysis
• Frequency of occurrence for each code per
trial was summarised in a prepared excel
spreadsheet
• An overall mean scores were calculated – to
explore user engagement across trials

22. Symbol Surfing Project - Reliability
• Two researchers reviewing each transcripts
and assigning the code via a process of
consensus, which informed the definition of
each code.
• Also independent coding on 10% of data –
Cohen’s Kappa Coefficient

23. Symbol Surfing Project - Findings
• Individual use of and engagement with symbol
recognition system
• Levels of ‘self expression’
• ‘Basic’ and ‘Functional actions’ with symbols
varied – factors: alertness and physical demands
of manipulating the symbol, attention
• Role of supporter during user activity- move
types: ‘instruction’, ‘query’, ‘technical’, feedback’
– facilitating engagement

24. The following slides are for you to
look at independently as they cover
material covered by Tony and myself
in previous lectures.
For next week please be ready to
discuss your research question and
your proposal.

25. Emotional Tools - Recap
• Emotional Text Analysis – locating emotion in
what users have to say
• Facial Recognition – observing emotional
responses to experiences
• EEG – observing brainwaves (relaxed and
attentive states)
• GSR – skin conductance
• Blending of the above with eye tracking

26. Emotional Analysis of The School Plan
Nice
Pleasant
Fun
Active
Nasty
Unpleasant
Sad
Passive
High Imagery
Low Imagery
Sample 1
Total words 27201
Sentence Length 18.72
Sentences 1453
Periods 614
Question Marks 466
% Nice 2.06
% Pleasant 4.38
% Fun 7.80
% Active 4.43
% Nasty 1.74
% Unpleasant 1.52
% Sad 4.42
% Passive 13.87
% High Imagery 2.57
% Low Imagery 39.27
Emotional text recognition using Whissell’s Dictionary of Affective Language

27. Blair’s Speech on War in Iraq
0
5
10
15
20
25
Nice
Pleasant
Fun
Active
NastyUnpleasant
Sad
Passive
Series1
Nice 2.04
Pleasant 3.53
Fun 3.79
Active 4.02
Nasty 4.3
Unpleasant 4.53
Sad 4.53
Passive 19.62

28. Analyzing User Feedback
• What is your goal?
– “I’m not sure. I am a little confused at this point as to what I am
supposed to be doing, It’s not obvious.”
• What did you expect when you did that?
– “I was hoping that if I clicked this icon I would be able to move to the
help page, but I am surprised that didn’t work.”
• Can you tell me what you were thinking?
– “I’m not sure. I think I’m supposed to be moving through this section
of the maze, but there is no help and I’m getting lost.”
• How did you feel about that process?
– “In a word confused. It looked good to begin with, but I am lost and
feel like giving up.”

29. Analyzing User Feedback
0
2
4
6
8
10
12
14
Nice
Pleasant
Fun
Active
NastyUnpleasant
Sad
Passive
Series1
Nice 4.3
Pleasant 2.69
Fun 2.15
Active 3.76
Nasty 1.08
Unpleasant 5.91
Sad 1.61
Passive 11.83

30. http://compling.org/emotion_in_text_cgi/DAL_app/index.php

31. match tools to your questions
• For questions on factual
clarification
• Use polls and surveys
• For questions on opinion
• Use surveys, interviews, focus
groups
• For questions on experience
• Interviews, focus groups,
observations, user testing, and
ethnographies
• For questions on concepts
• surveys, interviews,
ethnographies, and user testing
• For questions on emotions
• Surveys, interviews, focus groups,
and observations

32. Correlations
Interview, Questionnaire & Focus Group

33. References
• Bunning, K., Kwiatkowska, G., Weldin, N. (2012) Journal
of Assistive Technology, V24 n4 p259-270
• May T (1997) Social research: issues, methods, progress
Buckingham: OUP
• Rugg, D. (1941) Experiments in wording question. II
Public Opinion Quarterly, 5:91-92.
• Silverman D (1997) Qualitative research: theory, method,
practice London: Sage
• Silverman D (2004) Doing Qualitative Research: A
Practical Handbook London: Sage