PISA-VET launch_El Iza Mohamedou_19 March 2024.pptx
Encouraging Active Participant Engagement in the Evaluation of Online Conferencing
1. Encouraging Active
Participant
Engagement in the
Evaluation of Online
Conferencing
Angela Murphy
Amy Antonio
Shirley Reushle
2. Online conferences offer
convenient professional
development opportunities
without the travel and
expense of face to face.
Most conference evaluations
focus on participant
satisfaction and reactions and
do not evaluate the learning
that actually occurs (Anderson
& Anderson, 2010).
This study adapted the new
learning methodology (Chapman
et al., 2007) to evaluate the
impact of an online
conference holistically and
interactively.
3. The Follow the Sun Online Learning Futures Festival ran non-stop for 48 hours,
over six shifts of eight hours each, with consecutive handovers between
Australia (University of Southern Queensland), United Kingdom (Leicester
University) and Canada (Athabasca University).
4. The aim of the conference was to bring together university staff and students
from a range of disciplines across the world to share ideas and explore
knowledge development.
750 participants from over 35 countries registered for the conference.
5. The non-stop nature
of the event aimed to
mirror a 24-hour
digital society and
the 21st century
learner who wants to
be engaged with
other learners
any time,
anywhere.
Image source: http://www.rgbstock.com/bigphoto/niXkXUu/World+Time+1
6. Objectives & Approach:
1. Identify participation trends during conference sessions. Analytics
2. Explore networking and interactions trends. Twitter, chat and social media
3. Real-time insights into participant
Real time evaluation questions
perceptions and new learnings.
4. Identify and track new learning and
intention to action.
Summative survey evaluation
5. Presenter experiences and
future support requirements.
Presenters’ survey
6. Inform the conduct of
future events. Focus group with organisers
7. “Please tell us about
any new learning
you have
experienced or any
new questions that
have occurred to
you as a result of
this conference?”
(Chapman et al., 2007)
Image source: http://www.soil-net.com/
8. Festival analytics
3% 4% 81%
6 7 9
5% 5
4
7%
3
22%
Attended two
sessions
Most online conference
participants attend only 51%
one or two sessions Attended only
one session
9. Twitter and social media
conversations
Conversational
Tweets that included an
observation or expressed an
opinion “Interesting conversation
“Nominal Group Technique about the future of the
Session @ #fts12. Join us!” world”
Promotional Informative
Tweets that pointed to
Tweets that promoted an
resources with an
upcoming presentation
accompanying link “Recordings from
#fts12 now available @
http://t.co/smWYn413”
Instructional #FTS12 Tweets could
Tweets with a directive to
perform an activity be categorised
“Click latecomers link
to join! into four groups
http://t.co/5iTLCwN”
10. Real-time evaluation questions
Most found that the sessions
inspired ideas for daily
Please indicate your agreement with the sessions
practice that they intend to
facilitated new ideas and learning : Total Sample (n=60)
action
11. Summative survey
evaluation for participants
I was actively listening, asking questions and/or
38%
communicating
I listened closely to the sessions but did not ask
28%
questions
I listened occasionally while working or multi-
21%
tasking
I dropped in and out of sessions 6%
I tried to listen and multitask but didnt manage
3%
it very well
Online conference
None of these (please specify) 4% participants listen and
participate actively
Q8. Please indicate which of the following are the closest to the way in which you
participated in the sessions. (Select one) (n=116) during sessions
12. Summative survey
evaluation for presenters
“It's more difficult to keep participants engaged because you can't
rely on eye contact, body language or movement around the room to
help maintain interest. You also have to keep talking (most of the
time) as the participant isn't sure what is happening during a pause.
You also have to keep faith that the technology will work for both
yourself & the participants!”
“It was just hard to keep going with the
presentation and keep a tab on what was going on
with the social media at the same time.”
“It was the first time I had talked 'at' participants for
30 min, rather than seeking feedback periodically.
Although I thought it went well, I did find that it was Presenters found
hard to present for that long without seeking some
kind of feedback along the way..” online presentations to
be challenging
13. Focus group for
conference organisers
Using the same old
tech advertising
didn’t reach new
audiences within
disciplines
Risk
Collaboration Time consuming and
technical to set up ~
difficult within other
Reputational impact Global responsibilities
and reach much reach
greater than could
be achieved through
other means
14. Immediate
Feedback XII Limited
Responses
Identify
issues as they
occur III Time
consuming
Address
issues
Create promptly Difficult to
opportunities
organise
for engagement
15. Difficulties in setting aside dedicated time
above work commitments is the primary barrier
to attending online conferences
What barriers have you experienced in attempting to attend online events in
general? Select those that are the most applicable to you) (n=172)
16. The combination of synchronous and
asynchronous features is considered to be
the ideal online conference format
An event with synchronous and asynchronous features, e.g.
pre-recorded video accompanied by questions in real time
34%
Pre-recorded mini webinars that can be accessed at any time 17%
Mini webinars of one or two hours that include options to post
questions and communicate with other delegates
16%
Shorter sessions, e g , 30 minutes 12%
Full online conference over two or more days 7%
Full online conference over a day 6%
Mini webinars of no longer than half a day that include options
to post questions and communicate with other delegates
5%
Other preferences 3%