A presentation at Museums and the Web 2009.
Morag Hutcheson, Parks Canada, Canada
Tamara Tarasoff, Parks Canada, Canada
Christophe Rhin, Camineo, France
Handheld GPS-triggered location-based devices seem ideal for outdoor heritage sites. In this age of economic strain, they may be able to attract new audiences and draw existing audiences to return more often. But what do we really know about how these devices impact visitor experience and learning? Which segments of our audience do they actually appeal to? Can heritage institutions with limited financial and human resources develop these types of projects sustainably? To find out, between November 2007 and July 2008, Parks Canada staff developed and launched three handheld GPS-triggered tours. The process of developing the tours was monitored closely throughout the 8-month development period, and feedback gathered from team members and other staff. Then during the two-month pilot period, over 1000 visitors used the devices; regular feedback was provided by both users and staff.
The project had many positive outcomes, including recommendations for streamlining the development process and product delivery, a better understanding of the target audience, and suggestions for improving the usability and effectiveness of the devices.
Session: Location-Aware Services [Technology]
2. Parks Canada Agency
Who We Are and What We Do
•We are a federal agency mandated to protect and present nationally
significant examples of Canada's natural and cultural heritage
•We manage 42 national parks, 158 national historic sites, 3 national
marine conservation areas from coast to coast to coast
3. Why GPS-triggered Location-based Technology?
Many perceived benefits
• No physical infrastructure
required, works in remote
outdoor settings
• Could appeal to existing and
reach new audiences (Banff
New Media Institute Research,
2007)
• Could contribute to learning
and meaningful visitor
experience (WebPark, Dias
2007)
4. Should Parks Canada Invest in this Technology?
Two supporting questions
1. Can Parks Canada staff
develop this type of
technology project
using their own
resources and
expertise?
2. How do GPS-triggered,
location-based devices
affect visitor learning
and experience in
outdoor settings?
5. Content Developed by Staff? Yes!
• 150 days of site staff time to develop the first project; half this time
needed to develop subsequent trails
• Staff needed: part-time project manager and interpreter; GIS
specialist, subject matter experts, visitor services, IT, others
6. User Evaluation: Research Questions
1. How do visitors use the device?
2. How does this technology affect the
experience of visitors on the trail?
3. What do visitors learn from it?
4. Would visitors rent a device like this if
it were available, here or in other parks
or sites?
5. Which segments of the audience are
most positive towards this
technology?
7. Methodology
253 395
102 192
12 10
278 384
Data collected between July 8th
(at SH) and July 14th
(at KJ) to Sept. 15th
, 2008
Tally Sheets
Questionnaires
Observation Diaries
Data Logs
n (SH) n (KJ)
8. 1. How Was It Used?
Data logs show actual use
•Red = Pushed POIs
•Green = User actions
(low speed)
Many users explore
content at start and at
each pushed point of
interest.
9. 1. How Was It Used?
Typical use of the device
Visitors typically DO
• Read content out loud to other group members
• Interact with each other and the surroundings
• Access 2nd
level content (select ‘More’, Quiz,
Audio or Video button)
Visitors typically DO NOT
• Access content that is not attached to a POI
• Argue over the device
• Get distracted by the technology itself
10. 2. How Was the Experience?
Experience Statements - Agreement Scores
85%
67%
94%
78%
82%
93%
0%
20%
40%
60%
80%
100%
Made hiking experience
more enjoyable
The device was easy to
use
The device was a fun
activity to do with kids
Signal Hill Kejimkujik
11. 2. How Did It Change the Experience?
Before and After Mean Retrospective Scores
3.76
4.00
4.00
4.13
4.50
4.62
4.25
4.54
1.0 2.0 3.0 4.0 5.0
Enthusiasm for
activities like this
one (SH)
Enthusiasm for
activities like this
one (KJ)
Interest in learning
more about SH
Interest in learning
more about KJ
Signal Hill Kejimkujik
(Before)
(Before)
(Before)
(Before)
(After)
(After)
(After)
(After)
12. 3. What Was Learned?
Learning Statements - Agreement Scores
85%
91% 88% 87%89% 86%
0%
20%
40%
60%
80%
100%
Learned more about the
site
Better understanding of
landscape & natural
processes
Noticed something they
would not have noticed
on their own
Signal Hill Kejimkujik
13. 3. What Was Learned?
Signal Hill - Changes in Learning
2.53
2.73
2.93 2.95
4.46 4.37 4.43
4.18
1.0
2.0
3.0
4.0
5.0
Awareness: people
lived here for 300
years
Awareness: natural
features of
landscape
Understanding:
importance of SH
for defense
Understanding: use
of SH for
communications
Before the HGD Experience After the HGD experience
14. 4. How Much Would They Pay?
Price sensitivity for a 1/2-day rental?
83%
0%
14%
75%
4%
21%
100%
0%
20%
40%
60%
80%
100%
Free $4-$6 $7-$10 $11-$15
Signal Hill Kejimkujik
• Majority would pay ($4 - $6) for a
1/2-day rental
• Little appetite for prices exceeding $6
15. 5. Which Users Reacted Most Positively?
They all did.
• There is no category that reacted negatively to the technology.
• The technology clearly has a broad appeal.
16. Recommendations and Lessons Learned
•Focus on the content, not the
technology
•Tailor the content to your audience
•Focus on pushed/main content
•Not a one-time investment
•Start small – but start!
•Try out a device early
•Create, test on location, adjust,
test…
17. What’s Next?
•Facilitate adoption by Parks
Canada sites
•More user research
Questions? More info?
tamara.tarasoff@pc.gc.ca
christophe.rhin@camineo.com
18. Costs to develop the first and subsequent
Explora projects
Item
Year 1: Cost for first
project
Year 2-6: Costs for
sustaining project
Amortized cost per year
over 6 years
Software license and support TBD TBD TBD
First project set up fee TBD 0 TBD
12 PDAs and accessories 4600 4000 1433
Equipment (laptop, digital
camera, etc.) 0-3000 0 0-500
Satellite reradiator 900 0 150
Storage rack and charging
station 200 0 33
Travel for team members 0-1000 0-5000 0-1000
Translation 2500 12500 2500
Copyright fees 500 2500 500
Illustrations, AV 850 4250 850
GIS data 200 0 34
Promotional material: signage,
flyers 2600 1000 600
Total $12,350-16,350 $24,250-29,250 $6,100-7,600
Costs to develop first and subsequent Explora projects
Assumptions: Scale and nature of project is similar to Explora pilot; assumes site will update and add content
each year
19. Time required to develop the first and
subsequent Explora projects
Function Days for First Project Days for Subsequent Project
Project Manager 20-30 10-15
Interpreter 80-100 30-50
GIS Specialist 12 4
Subject Matter Experts 8 8
Visitor Services 8 4
CIO 4 4
Operations/Facilities 8 2
Total 140-170 days 62-87 days
Explora Project: Time Required to Develop First and Subsequent Projects
Assumptions: Scale and nature of project is similar to Explora pilot; excludes time for support
from Explora support team
Hinweis der Redaktion
I work in the New Media Team in Gatineau. We do research into new technologies so that we can provide advice and support to the field. It is the field – that is, those working at each park, historic site and nmca, that determines how to allocate their resources for visitor programming. Location-based technologies were identified as one promising technology.
Why did we choose this technology?
At first glance, the technology is very appealing. It works in remote outdoor area, and does not require physical infrastructure
A 2007 study on Tracklines, a pilot that took place in Banff National Park, showed that visitors were favourable towards this technology.
Research on WebPark, which took place in Switzerland and Holland in 2001-2004, showed that hikers using this technology spent more time on trails and retained more information than hikers with paper guides only
Could use existing content, could be repurposed for other platforms
We had 3 main questions that we hoped the pilot project would answer. Today, I will focus primarily on the third question so that I can tell you about the findings of our visitor evaluation. But before I do that, let me show you what we developed and tell you what we learned about developing this kind of project.
Three trails at two sites: Kejimkujik NP & NHS (Nova Scotia) and Signal Hill NHS (Newfoundland)
Content produced by staff
Staff mentored by outside technology expert
Devices provided to users for free, July 8-Sept. 15
Device independent solution; off-the-shelf devices
As for the second question, I can tell you that we did determine that our staff, when supported by an outside expert, can indeed develop this type of project. We selected two sites for the pilot project: Keji and Signal Hill. Development began in November 2007. In July 2008, the project was launched with the public. We learned that indeed Parks Canada Staff can develop this type of project with their own resources and expertise. Outside technical support is required to develop a first project but this need is reduced with subsequent project
For the remainder of this presentation, I will explain how we evaluated the visitor response to the device and what our findings were.
We had five main research questions, as you can see on the slide.
Between July 8 and September 15, 2008, over 1500 people used the devices on three trails at Signal Hill and Kejimkujik. Our methodology combined qualitative and quantitative research. In total, 22 observations and 294 surveys were administered, and we also completed tally sheets for all uses (648). In addition, we analysed the data logs; these logs were records of every user’s position every 15 seconds, as well as records of the location and time of every user action.
All user actions and positions were logged by the devices. This map from Kejimkujik shows the location of the pushed POIs, in red, as well as the location of user actions, in green. The data logs showed that users stopped at locations of pushed POIs, and also explored content on the device at these locations. This tells us that these areas are where users will interact with content.
Users typically use the device by stopping when the device “chimes” at most (if not, all) activation points, interacting with the pushed content (reading, doing a quiz, watching a video, listening to audio), reading content aloud to their group, accessing the “more” or second-level pulled content, then resuming their journey along the trail until the device “chimes” again. Throughout the trail, few users accessed items that were not prompted, but quizzes and audio/video content linked to pushed content were very popular.
There is very strong evidence that users enjoyed using the HGD, and that it added to their experiences on the trails, making it more enjoyable. A large number of users gave the device top marks for being a fun activity to do with the kids, and for the likelihood of using a similar device on other trails and at other sites. The non-content functions of the HGD (directions, location features) were highly enjoyable, as well as the content or experiential functions (being informative, providing educational content). Dislikes focused on functional elements that were barriers to using the HGD optimally (things like screen legibility, menu access and technological glitches in navigation).
More enthusiastic about these types of activities
• More interested in learning about the site
The HGD positively contributed to the experience at the parks for most users. After using the device, users reported greater interest in learning about their sites, and gave high scores for the device helping them notice things they would otherwise not have noticed. A direct assessment of how the HGD changed the behavior of visitors was not possible because no comparable information (such as whether there was greater group interaction and engagement due to use of the HGD) was gathered from visitors not using an HGD prior to the HGD pilot program.
The majority of users were able to identify main messages, which varied by site. In Kejimkujik, most users identified changes in nature and their importance as the main message, while Signal Hill users felt the focus was on cultural and military history. Given the plethora of possible content that could be shared with users and the amount of time required to consume that content, the identification and execution of main message content needs to be efficient and effective.
The majority of visitors learned something new, learned more about the site, helped increase understanding of the landscape and natural processes at the site, and helped them notice something they would not have noticed on their own. These learnings varied by sites. In Kejimkujik, the users recalled learning about the Rattlesnake Plantain, Mi’Kmaq culture, and the Beech Canker. In Signal Hill, learning focused on military history, wildlife and natural history.
Over 85% of visitors who piloted Explora agreed that the device helped them notice things on the trail they would have otherwise missed.
Significant increases in awareness and understanding
Awareness and understanding of cultural and natural aspects of the trail increased significantly after using the device.
Determining the extent to which visitors benefited from the use of the interactive features of the HGD is difficult because of the challenges linking measurable HGD experience to questionnaire responses. However, suggestions are that text, quizzes, audio and video materials were utilized and liked. We cannot say with certainty that users’ educational experience was (or was not) impacted by use of the interactive media, nor can we compare how learning varied between users and non-users.
94% of users would use a similar device on other trails here or at other parks or sites.
Those who participated in the evaluation were very eager to use HGD devices at other locations, both within the site on alternate trails and at other Parks Canada sites.
Visitors at both sites very strongly agreed with the statement “I would use a similar device on other trails here or in other parks or sites”. In fact, the single most frequent recommendation for improving the HGD experience at Kejimkujik was making more trails available on the HGD.
The preference is to pay between 4 and 6 dollars for a half-day rental of the device, but the price-point will ultimately need to be determined by Parks Canada.
Users at both sites and from every demographic enjoyed and learned from their HGD experience.
All HGD users tended to describe the same positive, educational HGD experience, regardless of age, gender, place of residence, level of technological experience, and group size.
Several recommendations can be made In order to maximize the impact that using the HGD has on visitors’ Parks Canada site-related learning and experiences.
Tailor the HGD Content to the HGD User – Visitors may benefit more from the HGD if its content is targeted to their experience with the site or with their primary area of interest.
Rely on Pushed Content –Visitors preferred being presented with material at the activation points either directly, as in the pushed content, or indirectly, as with the more, quiz, and audio/video material that was linked to pushed pages.
Include More Trails at each Site –. By expanding the number of trails that the HGD can be used on, the learning and enjoyment can also be expanded.
Adjust the Device Hardware to Make it More User-Friendly - Some visitors found that carrying the device using the lanyard was cumbersome, and that the activation noise was annoying.
Secure a common Agency-wide license and support for location-based technology use
Develop toolkit and training to facilitate the development of location-based tours
Communications with parks and sites
Research price sensitivity of sites and parks
Explore repurposing of location-based content for other platforms (e.g. web) and “grow” with the changes in technology
Slide to be hidden but to be kept in case anyone wants the info.