The study aimed to determine how the inclusion of volunteered geographic information (VGI) alongside professional geographic information (PGI) impacts user judgements and system acceptance. 101 participants evaluated a travel website containing different combinations of VGI and PGI data. Presenting VGI increased perceptions of currency, usefulness, credibility and authority. Telling users about the VGI modestly improved authority, usefulness and satisfaction. The results suggest VGI can enhance existing systems without negatively impacting user perceptions if applied appropriately.
7. Study Aims
What We Are Trying To Work Out
• The extent to which the content of the information
within the mashup affects the users’ judgements on
the mashup.
• The extent to which the user judgements on the
information influence the overall usability and
system acceptance of the mashup.
• The extent to which the facets of the users’
judgements which may be harnessed to optimise
the design of future mashups combining both VGI
and PGI information.
8. Study Rational
What We Are Really Getting At
Presenting VGI Telling Participants
Alongside PGI in a Their Mashup
mashup Contains VGI
The Utility of VGI, The degree to which
what it may offer to current perceptions of
users that PGI does information from
not amateur volunteers
The Style of VGI, a influences perception
different way of of neogeography
presenting information
16/01/2012
9. Study Rational
Dependable Variables
User Judgements of Online Information System Acceptance Model
Rieh (2002) Maguire (1998)
Quality Authority Usability
Good Trustworthy Usefulness
Accurate Credible Clear
Current Reliable Efficient
Useful Official Satisfaction
Important Authority
11. Methodology
Selection of a Participant Community
• Need to critically evaluate information
• Can be bounded in as a simple user group
• Make use of PGI as well as generating and utilising
VGI
• Critically assess information for risk management
• BUT be normal people doing normal things.
• Selection Criteria
• Age 18-65
• Full time wheelchair user
• Non cognitive or Sensory Disabled
• Non-house bound
• Other (regular ethics)
12. Methodology
Two Parts of the Study
• Part 1
• Observation (6 wheelchair users)
• VGI: Generate Data Set
• Literature Review
• PGI: Pool data
• Part 2
• Mono Methods
• Experimental
• Online (101 participants)
• Four Groups – with a unique combination of
contents of the map and information about
the map.
13. Methodology
Fixed Variables and Groups
Information Presented
In Map
PGI PGI + VGI
Participant Told PGI Group 1 Group 3
What Map PGI + VGI Group 2 Group 4
Contained
15. Study Rational
Dependable Variables
User Judgements of Online Information System Acceptance Model
Rieh (2002) Maguire (1998)
Quality Authority Usability
Good Trustworthy Usefulness
Accurate Credible Clear
Current Reliable Efficient
Useful Official Satisfaction
Important Authority
23. Part 2
Assumption Testing
User Judgements of Online Information System Acceptance Model
Rieh (2002) Maguire (1998)
Quality Authority Usability
Good Trustworthy Usefulness
Accurate Credible Clear
Current Reliable Efficient
Useful Official Satisfaction
Important Authority
24. Part 2
Assumption Testing: Factor Analysis
User Judgements of Online Information System Acceptance Model
Rieh (2002) Maguire (1998)
Quality & Authority Usability
Good Trustworthy Usefulness
Accurate Credible Clear
Current Reliable Efficient
Useful Official Satisfaction
Important Authority
25. Part 2
Assumption Testing: Collinearity
User Judgements of Online Information System Acceptance Model
Rieh (2002) Maguire (1998)
Quality & Authority Usability
Good Trustworthy Usefulness
Accurate Credible Clear
Current Reliable Efficient
Useful Official Satisfaction
Important Authority
30. Results
Quality and Authority
• Presenting VGI alongside PGI
• Most notable influence is on currency, as
perceived by the inclusion of VGI within the
data set and should be considered
• Importance, Credibility, Usefulness and
Authority are all positively influenced, but
how much consideration needed is
debatable
• Telling People their mashup contains VGI
alongside PGI
• Most notable influence is on Authority
• Usefulness and Credibility worth considering
• The importance of there variables and effect
of being told is limited
31. Results
System Acceptance
• Information Presented in Mashup
• 303 Participants (Estimation)
• F (3, 297) = 4.67
• p = .003 > .00851
• Wilks’ Lambda = .96
• ηp2 = .045
• Information Told to Participants
• 404 Participants (Estimation)
• F (3, 398) = 4.38
• p = .005 > .00851
• Wilks’ Lambda = .97
• ηp2 = .032
• No Statistically Significant Interaction
16/01/2012
33. Results
System Acceptance
• Presenting VGI alongside PGI
• Most notable influence is on usefulness
• May result from more info = more utility
• Telling People their mashup contains VGI
alongside PGI
• Most notable influence is on slightly
increased levels of satisfaction
• May be due to idea of social interaction?
35. Conclusions
From Research
• Presenting VGI Alongside PGI
• Reinforces the mantra of right data for right
task
• Does VGI cover more ground than PGI?
• Judgements of Quality & Authority increased
along with Usability
• No negative Influence!
• Telling Users They Are Using PGI
• Personally held biases have a measurable
effect on Quality & Authority and overall
Usability
• Effects are relatively minimal and attention
may be best given to other factors
• No negative influence!
36. Conclusions
For Design
• VGI did not change the game dramatically
• Though currency is a very interesting outcome
• In every case VGI enhanced the user
perceptions and usability
• VGI is not necessarily a game changer in
normal situations, but instead, if applied
correctly is polish.
• Wont make a bad system good
• But has potential to enhance what is already
good, making it more satisfying.
• Little point polishing low end mashups, but high
end a small amount of polish is VERY important
Hinweis der Redaktion
Our new home at the Loughborough Design School
Brief overview of the background research in my PhD so you understand why the research I will be talking about is important
VGI = Volunteered Geographic Information, PGI = Professional Geographic InformationThese are the main user groups that emerged from the interviewsEach user group perceives the other one differently, unique relationshipsEach one perceived VGI and PGI differentlyThis is based on user requirements AND existing beliefs.HAPPYNESS: We can’t please all the people all the time
Some photos of my data gathering with focus groups, participatory observations and diary studies.I didn’t capsize but I did manage to sink my kayak in the middle of the river!
Focus Group Graph = Impact of Information on Outcomes; #refs madeEarliest stages of kayaking activity rely on external informationLater stages rely on internal information (water gauges)Lots of VGI used in planning, therefore sharing experiences importantNo mention about volunteering the info – GAP!!!!VGI best for fast changing, subjective areasPGI best for static, objective features.HAPPYNESS: The info only helps make you happy when it helps you achieve your activity
Current Research
What we are trying to find out
These are the real influences we are measuring in the study
Two theories used as the backbone to this researchDID NOT mix them!
But we can’t just research any user group, we need on which fits our requirementsCLICKSo as you might have guessed by now, wheelchair users were selected!Compatible to kayakers, but different enough to understand the wider issues of use of VGI
Study in two parts…
Does adding VGI to PGI increase the quality, accuracy and usability perceptions of the mashup?Does telling people there is VGI in the PGI increase quality, accuracy and usability perceptions?
So we analyses the data, compiled it and presented it through a website FREE TRAVELLER
After using the website, participants given a questionnaire derived from the two base theories of this research4 questions (2 positive, 2 negative) per dependable variable. Wording of questions based on suggested key works as laid out by the frameworks.Then we run Multivariate Analysis of Variance to understand the influence of the fixed variables on the groups of judgements. (MANOVA)
That was the route we took over 6 hours!
So I went out in London with a group of wheelchair users and created my own VGI data set.Each wheelchair user had a diary and we recorded the good and the bad points about public transport in London
Done by literature review of all current transport data about London and disabled access. Gaining ‘best case scenario’ of what is currently presented professionally.Along with VGI, two data sets were created of a demonstrable quality above that commonly found online today.
Remember that website that we built?
And here is an example of the data with presented the participants withAt the end of using the maps a 32 question Likert scale (5 point) was presented to test the perceptions of Quality, Authority and Usability
Before MANOVA could be applied to survey data, assumptions needed to be tested.Initially three categories of data were presumed, in line with the theories
Confirmatory Factor Analysis demonstrated these two groups, not three
Multicollinearity and singularity testing (Spearman rho) demonstrated high correlation between some of the factors. These were removed from the data set to allow MANOVA to meet all of its assumptions. Removed these as we are interested in not just if variables are significant, but also their effect size.From this the frameworks of assessing the user reaction were reached.All other assumptions of the data required to run MANOVA were met.
All those demonstrated here are positively influenced by the VGI stimuliBare in mind, this graph is designed as an easily understandable overview of the data. Each dependable variable is demonstrating the Partial Eta Squared for the lowest N value to achieve significance. Realistic would be more skewed and misleading in one graph. Done this was due to the use of Sample Size Estimation which biases the data when taken to the Nth degree, so this graph is more realistic to a researchable sample size.0.01 is a small effect, 0.09 a medium effect and 0.25 a large effect
Graph to show currencyThe most dramatic. Shown to demonstrate the kind of relations were working with.
0.01 is a small effect, 0.09 a medium effect and 0.25 a large effectBare in mind that the effect sizes for usefulness on the Q&A and SA are similar.
While the extent to which this study can represent the exact nature of VGI is debatable, this study has demonstrated some important underlying phenomenon
While the extent to which this study can represent the exact nature of VGI is debatable, this study has demonstrated some important underlying phenomenon