Building an engagement toolkit (Information Online 2019)

BUILDING AN
ENGAGEMENT TOOLKIT
Dr Kate Davis
Digital Life Lab | USQ
@katiedavis
Kathleen Smeaton
Library Services | UQ
@kathleensme

INTRODUCTION
and some housekeeping

HI, HELLO, HOW ARE YOU?
meet the facilitators

DR KATE DAVIS | @KATIEDAVIS
senior research fellow | digital life lab |
university of southern queensland | katedavis.info |
kate.davis@usq.edu.au
what i do:
i’m an interdisciplinary human experience researcher; a social scientist interested in
understanding how people experience information as part of their everyday lives. i am
also interested in how people experience online learning in higher education, and
applied i do research related to applied topics in information studies.
what i’m excited about:
transforming online learning at usq. implementing the digital literacy framework we are
developing for the university. having an impact on the experience of teaching and
learning at usq.

KATHLEEN SMEATON | @KATHLEENSME
project coordinator | uql |
university of queensland | k.smeaton@library.uq.edu.au
what i do:
i'm a librarian who has also worked as an academic and learning designer;
i'm interested in the role of libraries in lifelong learning
digital disruption, digital divide(s), digital inclusion, information literacy, all
things open

LYNDELLE GUNTON | @LYNDELLEG
liaison librarian | library services
university of southern queensland | lyndelle.gunton@usq.edu.au
what i do:
i support staff and students participating in teaching, learning and
research in business and commerce disciplines. i collaborate with
academics to design and deliver online and offline educational
experiences that develop information and digital literacies and enhance
learning outcomes
information experience, digital literacies, design for engaged online
learning, academic integrity, social media, using evidence to inform
practice

A QUICK NOTE ON WHERE TO FIND STUFF
online
readings, slides, references and tools will be available on kate’s
website: katedavis.info
don’t stress about noting references – it’s all linked online
in your hands
we’ll be handing out slides and activity materials throughout the day
and you’ve each got a folder to hold your materials

ALIA Information Online 2019 – Five Senses of GLAMR Session
10.30am-12.00pm Wednesday 13th February in the State Room
Pre-Registration required : https://membership.alia.org.au/events/event/the-five-senses-of-glamr
by 7pm Tuesday evening, via the website

getting to know each other
ACTIVITY: LET’S BREAK
THE ICE

STEP 1: TAKE AN M&M
take an m&m from the bowl on your table
(don’t eat it. yet. J)

STEP 2: SHARE SOMETHING ABOUT
YOURSELF
according to the colour of your m&m, please feel welcome to share
the following information about yourself:
• brown – share a hobby you enjoy outside work
• yellow – describe your commute to and from work
• red – describe your favourite breakfast food
• orange – share what you did on saturday
• green – share the first live performance or concert you ever
attended
• blue – share one thing you are excited about this week – be
specific! (you can’t just say ‘the conference’!)
tables
10 minutes

WORKSHOP OVERVIEW
what we’ll get up to today

WORKSHOP TIMINGS
SESSION TIME CONTENT
session 1 9.15am – 10.40am introduction, scene setting and
problem definition
morning tea 10.40am – 11am
session 2 11am – 1.30pm • frameworks
• designing a research strategy
lunch 1.30pm – 2.15pm
session 3 2pm – 3.10pm collecting and analysing data
afternoon tea 3.10pm – 3.30pm
session 4 3.30pm – 5pm • collecting and analysing data
• applying what we’ve learned

OUR GOALS
we want you to walk away with an understanding of
• how to design a program of research, investigation or evaluation
• some qualitative research approaches and when to use them
• frameworks for addressing problems, program / service design,
and evaluation
• how to collect data that provides rich insights into human
experience (specifically, through interviewing)
• how to analyse qualitative data

WHAT WE WON’T ACHIEVE TODAY
• you won’t be an expert in anything we cover (sorry – come see us
about doing a phd!)
• you won’t learn about the nitty gritty of any of the frameworks (we
could spend a whole day on each of them)

WHAT WE WILL ACHIEVE TODAY
• expose you to frameworks, methods and approaches you can
investigate further (and we’ll give you tips on where to start)… today
will be a ’taster’
• give you a recipe for constructing a research strategy
• inspire your curiosity (and hopefully build a desire for further
learning)
• help you to see how research can help you address problems you
are facing and engage your customers
• identify barriers to operationalising this stuff and how you can
overcome them

WE HAVE A REALLY
AMBITIOUS AGENDA
TODAY
BUT A FUN ONE( )

GROUND RULES
• work fast
• done, not perfect
• collaborative
• voices for everyone
• no spectators
• when the time is up, it’s up! come back to the group quickly
• shout out your questions as we go through
• did we say work fast?

WHAT HAPPENS IN THE
WORKSHOP STAYS IN
THE WORKSHOP

WHY WE’RE TALKING ABOUT
QUALITATIVE RESEARCH TOOLS
as an engagement toolkit

UNDERSTANDING PROBLEMS
• unless we really understand the problems we face, how can we
hope to address them?

SCENARIO
a significant proportion of students in course x are submitting work
that includes plagiarised material or ideas that are not appropriately
attributed.
you are the librarian that works with the relevant subject area. the
course coordinator comes to you for help.
what do you do?

SCENARIO
a significant proportion of students in course x are submitting work
that includes plagiarised material or ideas that are not appropriately
attributed.
you are the librarian that works with the relevant subject area. the
course coordinator comes to you for help. what do you do?
step 1: find out why the students are engaging in this practice.

SCENARIO
when the course coordinators unpacked what was happening in this
instance, here’s what they learnt:
1. the majority of students who were demonstrating academic
dishonesty were international students, from the same country,
and the same region
2. they had:
opoor language skills
ono experience with assessment that asked them to formulate their own
ideas and arguments (assessment was all based on exams that relied on
rote learning and had a low pass grade)
ono experience with an education system that asked them to attribute
material (‘copying’ was seen as a respectful practice in their culture’)
oa deep seated fear of approaching the teacher (a cultural issue)

SCENARIO
ultimately:
• they did not understand the content because their spoken and
written english was so poor
• they didn’t know copying was the wrong thing to do
• once they did know, they struggled to change their practice
because of the language barriers

SCENARIO
• what other causes might there be?

SCENARIO
other reasons:
• difficult content that is not well understood
• poor teaching
• confusing assessment guidelines
• a lack of understanding of common citation practices
• illness (theirs, or someone else’s)
• an intention to cheat because they wanted to see if they could get
away with it

PEOPLE, EXPERIENCE AND COMPLEXITY
• people are complex. to understand them, their needs and their
experiences, we need to talk to them

QUALITATIVE RESEARCH ALLOWS US TO…
• understand problems
• understand people’s needs, behaviours, experiences
• elicit stories
• tell evocative stories that illustrate our impact

CASE STUDY 1
24 hour library opening and spaces research

24 HOUR LIBRARY OPENING AND SPACES
RESEARCH
why do this research?
• to find out how the service was working
• to find out the value of the service to students
• to find out the 'worth' of the service to the library
• to see what else might come up…
we needed an overarching research question so we asked…
• are they doing what we think they are doing?
so then we could ask…
• are we delivering the 'right' service?
• are we investing in the 'right' things?

24 HOUR LIBRARY OPENING AND SPACES
RESEARCH
how?
• interviews with staff
• interviews with students
what did we find?
• happy with the service (in fact want more!)
• spaces are critical
which made us think about...
• technology and spaces

CASE STUDY 2
digital literacy framework development:
a consultation process

WHAT WE NEEDED TO DO
• develop a digital literacy framework for the university
• increase likelihood of acceptance by engaging early and often
• two phases of consultation

PHASE 1: PURPOSE
• gather an evidence base to inform the development of the draft framework.
• aimed to address the following questions:
o how do members of the USQ community understand the concept of ‘digital literacy’? does
it align with the draft definition developed by the advisory group?
o what do USQ community members need from a digital literacy framework?
o what will make the framework meaningful and useful to the different stakeholder groups?
o how should the framework contents be organised to ensure maximum utility for
stakeholder groups?
• secondary aims:
o starting a strategic conversation about digital literacy across the institution
o socialising work done to date
o improving likelihood of the framework being accepted by USQ community members by
ensuring they feel consulted and included in the development process.

PHASE 1: CONSULTATION ACTIVITIES
• input from both staff and students by a number of mechanisms:
ofocus groups (staff and students)
odiscussion at key meetings (staff only)
oquestionnaire (staff only)

PHASE 1: ANALYSIS
• thematic analysis of qualitative data from focus groups and
questionnaire
• descriptive analysis of qualitative data to pull out the various skills
and knowledge that are part of digital literacy
• descriptive analysis of quantitative data around definition
acceptance, extent to which staff are confident with technology etc

PHASE 2: PURPOSE
• socialise and gain feedback on the draft framework

PHASE 2: CONSULTATION ACTIVITIES
• call for written feedback from staff and students
ostructured feedback form
ounstructured, open responses via email
• interviews with select participants:
okey stakeholders
oparticipants in the Phase 1 staff questionnaire who identified a specific
interest in digital literacy

ACTIVITY: PROBLEM
STORMING
exploring your problems and how research can help

GUIDELINES
what problems are your customers facing that you want to
address?
example > students in course x are not displaying academic integrity
what problems are you facing in your practice?
example > you aren’t seeing strong usage of
electronic collections and you want to know
what to do

STEP 1: WRITE DOWN YOUR PROBLEMS
working individually, write down your problems:
• customer problems on orange coloured post it
• your practice problems on pink coloured post its
• one problem per post it
individually
5 minutes

STEP 2: SHARE YOUR PROBLEMS WITH YOUR
GROUP
• run your table through your problems
• keep it brief, but ask and answer clarifying questions if necessary
• one minute per person! (two at most!)
tables
10 minutes

STEP 3: WHAT’S YOUR BIGGEST PROBLEM?
• choose your biggest / worst / hardest / most difficult / most
important problem and share it with the room
• hang onto that post it – you’ll need it later!
together
10 minutes

STOP! COFFEE TIME!
see you back here at 11am

FRAMEWORKS
for designing and evaluating programs, products and services
(and research strategies, too)

IN THIS SECTION
• research plays a role in many frameworks for professional
practice. we’re going to explore two:
oevidence based practice
odesign thinking
oand a hybrid of the two
• how to design a research study – what you need to consider
• we’re realists. we know it’s not always possible to adopt these
frameworks or undertake research in practice. we’ll talk about the
barriers and how you can get around them

““
DESIGN THINKING
design thinking is a human centred and collaborative approach to
problem solving that is creative, iterative and practical.
(Brown, 2008)

THE PROCESS
• discovery
• problem definition
• ideation
• prototyping
• testing
discovery
definition
ideationprototyping
testing

DESIGN PROCESSES ARE KINDA MESSY
(Newman, D. 2010)
uncertainty / patterns / insights clarity / focus
research concept / prototype design

evidence based!
practice!EVIDENCE BASED PRACTICE

DESIGN THINKING
design !
thinking !

design !
thinking !
& EBP!
DESIGN THINKING
AND
EVIDENCE BASED
PRACTICE

FRAMEWORKS
• give us a structure to work with when we are trying to solve
problems
• what is key: the centrality of research in the process

RESEARCH PROCESS
Establish a
Research Question
Identify Approach
Conduct Literature
Review
Determine Data
Types
Determine Data
Collection &
Analysis
Procedure
Identify Source of
Data
Obtain Ethics
Approval
(if necessary)
Data Collection Analysis of Data
Formulate
Conclusion and
Recommendation
phase 1: initiate the study
phase 2: plan the study
phase 3: collect the data
phase 4: analyse the data
Phase 5: write up

KEY QUESTIONS TO ANSWER
1. what is your problem?
2. what is your research question?
3. what type of data do you need to answer the question?
4. does any of the data already exist? where? how can you get access?
5. what method/s will you use to collect the data?
6. what will your data collection instrument look like? (what will your
interview questions be? what will your survey questions be?)
7. who will you collect data from? who are your participants? what is your
sampling strategy?
8. how will you recruit participants?
9. what ethical considerations are there? how will you mitigate them? do
you need to apply for ethical approval?
10. how will you analyse the data?
11. how will you ensure quality?
12. how will you report on your findings / outcomes? and why?

ACTIVITY: BARRIERS
speed boat / cloud game

STEP 1: WHAT’S WEIGHING YOU DOWN?
• what are your anchors? what’s stopping your cloud from floating
freely in the sky?
• one anchor per post it
tables
10 minutes

STEP 2: WHAT’S YOUR HEAVIEST ANCHOR?
• What are the most significant / common / heaviest / most
impactful barriers / anchors for your table?
• choose a top three
tables
4 minutes

STEP 3: SHARE
• share the one most significant anchor your table came up with
together
10 minutes

STEP 4: HOW CAN YOU CUT THE ANCHORS?
• working with your three most significant anchors, look at each of
the anchors in turn and brainstorm ways that you could cut the
anchors, or even start to incrementally reel them up
• three minutes per anchor
• use a different coloured post it than you used for
your anchors
• one idea per post it
tables
9 minutes

STEP 3: SHARE
• from each table, we want to hear one anchor and three strategies
for reeling it in
together
10 minutes

IN THIS SECTION
• we’ll talk about what qualitative research is
• we’ll look at the nuts and bolts of collecting qualitative data,
including
oan overview of qualitative approaches
oa discussion about participants
oa deep dive into interviewing
oa discussion of focus groups
oa discussion about combining data collection strategies
• we’ll talk about analysing qualitative data
• we’ll design a research strategy

““
qualitative research involves disciplined inquiry that examines
people’s lives, experiences and behaviours, and the stories and
meanings individuals ascribe to them. it can also investigate
organisational functioning, relationships between individuals and
groups, and social environments… it may bring new insights into the
experiences of individuals, groups or communities, or into issues
such as environmental change, public policies and planning.
qualitative research may also have quantitative elements or aspects.
WHAT IS QUALITATIVE RESEARCH?
NHMRC (2014). Chapter 3.1: Qualitative methods. In NHMRC (2015). National Statement on Ethical Conduct in
Human Research. Retrieved April 12, 2016 from https://www.nhmrc.gov.au/book/chapter-3-1-qualitative-
method

““
NATURE OF QUALITATIVE RESEARCH
qualitative research is concerned with '...developing explanations of social phenomena...'
1. the world in which we live
2. why things are the way they are
3. concerned with social aspects of our world
4. seeks to answer questions about
1. why people behave the way they do
2. how opinions and attitudes are formed
3. how people are affected by the events that go on around them
4. how and why cultures have developed in the way they have
5. the differences between social groups
5. qualitative questions:
1. how
2. why
3. what
University of Surrey (n.d.). Introduction to research. Retrieved September 4, 2016 from
http://libweb.surrey.ac.uk/library/skills/Introduction%20to%20Research%20and%20Managing%20Information%20Leicest
er/page_53.htm

““
qualitative researchers typically rely on four methods for gathering
information: (a) participating in the setting, (b) observing directly, (c)
interviewing in depth, and (d) analyzing documents and material
culture.
WHAT IS QUALITATIVE RESEARCH?
Chapter 4 Data collection methods in Marshall, C. and Rossman, G. (2014). Designing Qualitative Research.
Sage. pp. 97-150.

PARTICIPANTS: WHO, HOW
MANY, WHERE

PURPOSIVE | PURPOSEFUL SAMPLING
• participants selected according to specific criteria related to the
research question
opurposive
oconvenience
osnowball
otheoretical

SAMPLE SIZE
• depends on
omethodology guidelines
ospecific data collection method
oplanned analysis methods
oresearch question
odata quality
opractical constraints like time and budget
• often determined by theoretical saturation
• for PhD studies, there might be 20 to 40 participants in an interview
study

FINDING PARTICIPANTS
• participants don’t just fall into your lap!
• having robust recruitment strategies is critical
• recruitment protocols:
oposts to elists
oflyers in community centres, libraries, doctors’ surgeries, other relevant
locations
osocial media through
• the researcher’s own social media accounts
• a designated account for the project
• accounts of affiliated groups

EXAMPLES: TWITTER
• Are you a social media mum? Participate in research exploring how new mums use social media. Info:
katedavis.info/?page_id=5 pls RT!
• Participate in a PhD study exploring the information experience of new mums in social media. Info: katedavis.info/research pls
RT!
• Calling all new mums! If you use social media regularly, we need you! Participate in PhD research. Info katedavis.info/research
pls RT!
• Do u know a new mum who uses social media? Let them know abt this opp to participate in research! Info
katedavis.info/research
• Are you a tweeting, facebooking, blog reading new mum? We need you! Participate in research! Info katedavis.info/research
pls RT!
• Hey mummy bloggers! We’re looking for social media mums to participate in research. Info katedavis.info/research pls RT!
• Are you a blogging mum or a reader of mummy blogs? Participate in a PhD study. More info: katedavis.info/research pls RT
• Do you tweet between nappy changes and Facebook between feeds? We need you to participate in our study!
katedavis.info/research
• New mums! Do u use Twitter, Facebook, blogs, YouTube or other SM tools? Participate in research katedavis.info/research

EXAMPLE: FACEBOOK
Are you a social media mum? Is your youngest child under two
years? Participate in my PhD study exploring how new mums use
social media. Participation involves completing an online survey and
an interview. More information at katedavis.info/?page_id=5 pls
Please feel free to tell your friends!

!
!
PARTICIPATE!IN!RESEARCH!
Information!for!Prospective!Participants!
The$following$research$activity$has$been$reviewed$via$QUT$arrangements$for$the$conduct$of$research$involving$human$participation.$
If$you$choose$to$participate,$you$will$be$provided$with$more$detailed$participant$information,$including$who$you$can$contact$if$you$have$any$concerns.$
Information!practice!in!social!media:!the!experience!of!new!mums!
!
Research$Team$Contacts!
Principal!Researcher:! Kate!Davis,!PhD!student,!Science!and!Engineering!Faculty,!Queensland!University!of!Technology,!
k3.davis@qut.edu.au!
Associate!Researchers:! • Professor!Helen!Partridge,!Principal!Supervisor,!Science!and!Engineering!Faculty,!Queensland!
University!of!Technology,!h.partridge@qut.edu.au!
• Dr!Hilary!Hughes,!Associate!Supervisor,!Faculty!of!Education,!Queensland!University!of!
Technology,!h.hughes@qut.edu.au!
What$is$the$purpose$of$the$research?!
The!purpose!of!this!research!is!to!investigate!how!new!mums!use!social!media,!and!in!particular,!their!information!practices!in!
social!media.!
Are$you$looking$for$people$like$me?$
If!you!are!a!new!mum!(with!your!oldest!child!under!the!age!of!two!years)!and!you!regularly!use!social!media!(like!Facebook,!
Twitter,!YouTube,!blogs,!wikis!and!parenting!websites),!we’d!really!like!to!hear!from!you!!
What$will$you$ask$me$to$do?$
Your!participation!will!involve!
• a!questionnaire!
• an!interview!that!will!take!approximately!30!to!60!minutes,!in!which!you'll!be!asked!to!talk!about!your!use!of!social!
media.!The!interview!will!be!held!in!a!location!and!at!a!time!convenient!to!you.!The!interview!will!be!audio!recorded.!
You!are!welcome!to!bring!your!child/children!along!with!you!if!you!wish!
• a! period! of! social! media! observation! following! your! first! interview,! in! which! the! researcher! will! follow! you! in! your!
preferred!social!media!channels,!including!Twitter,!Facebook!and!your!personal!or!group!blog/s!
• a!second!audio!recorded!interview!at!the!end!of!the!observation!experience,!conducted!via!phone!or!an!application!like!
Skype.!This!will!be!an!opportunity!for!you!to!talk!further!about!your!social!media!experiences.!
Are$there$any$risks$for$me$in$taking$part?$
The!research!team!does!not!believe!there!are!any!risks!beyond!normal!dayStoSday!living!associated!with!your!participation!in!this!
research.!!
Are$there$any$benefits$for$me$in$taking$part?$
It! is! expected! that! this! project! will! not! benefit! you! directly.! However,! it! may! benefit! social! media,! information! literacy! and!
information! practice! researchers.! It! may! also! benefit! organisations! (like! health! care! agencies)! by! providing! them! with! an!
understanding!of!how!new!mums!use!social!media.!This!in!turn!may!benefit!other!new!mums.!
I$am$interested$–$what$should$I$do$next?$
If!you!have!any!questions!or!would!like!to!participate!in!this!study,!please!visit!our!website!or!contact!the!principal!researcher!
for!details!on!how!to!proceed.!!
Kate!Davis!
katedavis.info/research!
k3.davis@qut.edu.au!
0411!073!094!
You!will!be!provided!with!further!information!to!ensure!that!your!decision!and!consent!to!participate!is!fully!informed.!
Thank$You!! QUT!Ethics!Approval!Number:! !
!

QUALITATIVE DATA
COLLECTION METHODS
a super brief overview

COMMON QUALITATIVE METHODS
• interviews
• oral histories | life stories | narrative inquiry
• focus groups
• observation
o online
o offline
• field notes
• questionnaires and surveys
• action research
• document analysis

SPECIALISED QUALITATIVE METHODS
• reflective journals
• archival research
• historical analysis
• video | photography
• psychological testing

METHOD ADVANTAGES DISADVANTAGES
Surveys
• Anonymous completion possible
• Can administer to groups of
• people at the same time
• Can be efficient and cost
effective
• Forced choices may miss certain
responses from participants
• Wording may bias responses
• Impersonal
• Limited opportunity for qualitative
responses
Interviews
(individual/ in-
depth)
• Can build rapport with participant
• Can prompt to get additional
information
• Can get breadth or depth of
information
• Time consuming
• Expensive
• Interviewing styles and wording may
affect responses
Focus groups
• Can get common impressions
quickly
• Can be an efficient way to get
breadth and depth of information
in a short time frame
• Need experienced facilitator
• Can be di cult and costly to schedule
a group of 6–8 people
• Time consuming to analyze
responses
Observation
• Can observe the phenomenon as
it occurs
• Difficult to interpret observed
behaviors
• May influence behaviors of
participants
• May be expensive and time
consuming to record each individual
event
CDC. (n.d.) Selecting data collection methods.
https://www.cdc.gov/std/Program/pupestd/Selecting%20Data%20Collection%20Methods.pdf

INTERVIEWS
• yield rich and detailed data about how people experience the
world
• a conversation between researcher and participant
• can be done face-to-face, over the phone, or via video call
• generally recorded and then transcribed
• guided by an interview discussion guide, which sets out the
procedure for the interview

““
• usually yield richest data, details, new insights
• permit face-to-face contact with respondents
• provide opportunity to explore topics in depth
• allow interviewer to experience the affective as well as cognitive aspects of
responses
• allow interviewer to explain or help clarify questions, increasing the likelihood
of useful responses
• allow interviewer to be flexible in administering interview to particular
individuals or in particular circumstances
INTERVIEWS: ADVANTAGES
NSF. (2002). Data collection methods: some tips and comparisons. In NSF. (2002). The 2002 User-Friendly Handbook for Project Evaluation.
Retrieved April 12, 2016 from http://www.nsf.gov/pubs/2002/nsf02057/nsf02057_4.pdf

““
INTERVIEWS: DISADVANTAGES
NSF. (2002). Data collection methods: some tips and comparisons. In NSF. (2002). The 2002 User-Friendly Handbook for Project Evaluation.
Retrieved April 12, 2016 from http://www.nsf.gov/pubs/2002/nsf02057/nsf02057_4.pdf
• expensive and time-consuming
• need well-qualified, highly trained interviewers
• interviewee may distort information through recall error, selective
perceptions, desire to please interviewer
• flexibility can result in inconsistencies across interviews
• volume of information very large; may be difficult to transcribe and
reduce data

““
• go beneath the surface of the
described experiences
• stop to explore a statement or topic
• request more detail or explanation
• ask about the participant’s thoughts,
feelings, and actions
• keep the participant on the subject
• come back to an earlier point
• restate the participant’s point to
check for accuracy
• slow or quicken the pace
• shift the immediate topic
• validate the participant’s humanity,
perspective, or action
• use observational and social skills to
further the discussion
• respect the participant and express
appreciation for participating
INTENSIVE INTERVIEWS ALLOW THE
INTERVIEWER TO
Charmaz, K. (2006). Constructing Grounded Theory: a practical guide through qualitative analysis. London: Sage Publications. p. 26

““
INTENSIVE INTERVIEWS ALLOW
PARTICIPANTS TO
• break silences and express
their views
• tell their stories and to give
them a coherent frame
• reflect on earlier events
• be experts
• choose what to tell and how
to tell it
• share significant experiences
and teach the interviewer
how to interpret them
• express thoughts and
feelings disallowed in other
relationships and settings
• receive affirmation and
understanding
Charmaz, K. (2006). Constructing Grounded Theory: a practical guide through qualitative analysis. London: Sage Publications. p. 26

DEGREE OF STRUCTURE
• structured
• semi structured
• unstructured

MAIN QUESTIONS
• broad and open-ended rather than tailored for a specific type of
answer
• avoid yes or no questions

CLARIFYING AND PROMPT QUESTIONS
• follow-up questions, to facilitate more detailed responses, such as ‘Can you tell me
more about...’
• probing questions, to encourage participants to expand on points already made
• specifying questions, to prompt participants to talk about processes and practicalities,
such as ‘Can you tell me how...’
• direct questions, to elicit responses on particular topics of interest, such as ‘How do
you use that space...’
• structuring questions, to shape the course of the interview
• interpreting questions, to facilitate understanding of a participant’s responses, such as
‘How did you feel...’ or ‘What does it mean when you...’.
Kvale, S. and Brinkmann, S. (2007). InterViews: learning the craft of qualitative research interviewing. 2nd Ed. London, UK : Sage Publications.

INTERVIEW SKILLS
• active listening
• be in the moment
• build rapport
• control your body language
• be prepared to rephrase
questions
• keep notes rather than
interrupting
• take notes unobtrusively
• let the conversation flow while
remembering to come back to
key points
• summarise key ideas
• structure it like a conversation
• redirecting the participant when
they go off topic
• directing the conversation without
leading
Clifford, S. (n.d.) “Tipsheet – Qualitative interviewing”. DISM. Duke Initiative on Survey Methodology. Retrieved February 8, 2017 from
http://www.dism.ssri.duke.edu/pdfs/Tipsheet%20-%20Qualitative%20Interviews.pdf

STEP 1: WATCH THE VIDEO
• as you watch the video, take notes on anything you see or hear the
interviewer do that you think demonstrates poor interviewing skills
together
7 minutes

STEP 2: SHARE
• what poor practices did you see?
together
5 minutes

SOME OF THE MISTAKES WE SAW
• interviewing skills:
o not covering the purpose of the study
o not building rapport
o not making eye contact
o not showing good quality of attention
and listening
o talking too much and in a rushed manner
o not being in the moment
o not valuing the answers
o interrupting the interviewee
o not controlling her body language
• questions:
o not asking many open-ended
questions
o asking short answer questions
o biasing responses
o not probing

WHY HOLD A FOCUS GROUP
• access more participants in a shorter time frame
• some participants may be more comfortable
• snowball effect
• more participants in the group!

DOWNSIDES OF FOCUS GROUPS
• may not be as willing to share personal stories
• data may not be as rich
• less time to explore certain stories
• more participants in the group!
• facilitation can be tricky

KNOW WHAT YOU WANT TO KNOW
• your research question should guide focus group questions
• don't talk 'library' at them!
• don't ask leading questions
• pilot the questions - if you have time, this can save time in the end

RECRUIT THE RIGHT PEOPLE
• who do you want to speak to?
• why?
• do you need to have separate focus groups for some
participants?
• how will you advertise?

DONUTS (OR OTHER INCENTIVES!)
• how will you entice participants / reward participants / incentivise
participation?

CROWD CONTROL
• you are in charge
• stay on topic (even if it's really hard!)
• make sure everyone gets a chance to speak
• have multiple facilitators
• record or take detailed notes

COMBINING DATA
COLLECTION STRATEGIES

QUALITATIVE + QUANTITATIVE
• mixing data collection strategies can help build a richer picture
• qualitative data can inform collection of quantitative data and vice
versa
• qualitative data can illustrate quantitative insights

WHICH COMES FIRST?
• qualitative first:
oget a deep understanding of the phenomenon
ouse qualitative approaches to inform development of quantitative tools /
approaches
ouse quantitative approaches to see if your qualitative findings are
generalisable
• quantitative first:
oget a broad understanding of the object
ouse quantitative approaches to inform development of qualitative tools /
approaches
ouse qualitative approaches to deepen insights and elicit stories

STEP 1: BRAINSTORM
• work together at your table to brainstorm the different types of
data that libraries routinely collect or generate
• one type of data per post it
tables
5 minutes

STEP 2: SHARE
• report back to the whole room
ofirst group shares all their ideas
osecond and subsequent groups just share additional ideas
• one minute per table
together
10 minutes

STEP 3: ANALYSE
• what might these data sources be able to tell us? pick 3 and brainstorm
• two minutes per data type
• use a different coloured post it than you used for
your data types
• one insight per post it
• you’ll be asked to report back on one type of
data – choose one now
tables
6 minutes

STEP 4: SHARE
• each table to share about one type of data – what might it tell us?
together
10 minutes

CASE STUDY 3
asknow instant messaging

THE PROJECT
• we ran a national pilot of a collaborative instant messaging
reference service, which allowed customers to contact us via their
preferred IM client
• novel because at that stage, libraries were only using proprietary
products that used web forms. we wanted to take reference into
the user’s space
• we wanted to evaluate the project against its aims and objectives

PRIMARY AIMS
• demonstrate a demand for an IM service
• gather data about potential users
• establish if an IM service would be feasible and sustainable in
terms of resourcing
• establish whether available open source software could provide
the functionality necessary in offering such a service
• develop a service model and operational plan

SECONDARY AIMS
• tracking the impressions of librarians operating the IM service, in
order to gauge training and documentation needs
• testing the reporting and logging capabilities of the selected
software to ensure our administrative requirements could be met
• determining what further information might be necessary to
implement a live service. This could inform further investigation
such as surveys and benchmarking with other libraries

TYPES OF DATA
• usage statistics (quantitative)
• a user survey (pushed to the user via a scripted message at the end of
every session) (primarily quantitative, with some qualitative questions)
• in depth chat transcript analysis (primarily qualitative, but quantitative
topic analysis was also undertaken)
• staff surveys (primarily quantitative, with some qualitative questions)
• focus groups (qualitative)
• an operator impressions wiki space (qualitative)
• a costing exercise (quantitative)

WHAT’S SIGNIFICANT HERE?
• we could not have evaluated against the aims without using multiple
data types, participants, and a mix of both qualitative and quantitative
data
• if we didn’t have qualitative data, we would have missed insights like this:
o From the operator point of view I found it more effective than traditional virtual
reference because it was quicker, more responsive and informal – I felt like I was
really connecting with most of the clients I dealt with, without the barriers of
traditional VR mediums (timelag, pushing pages and trying to cobrowse, which I
generally find confusing and frustrating for both operator and client).
• if we didn’t have quantitative data, we wouldn’t know how much it cost
or how satisfied users were

PLANNING FOR ANALYSIS
• plan from the outset as analysis approaches may influence
othe data you collect
ohow you collect it
ohow you store it
• selection of analysis approaches is often informed by the over-
arching methodology e.g. grounded theory uses a particular type of
analysis

““
The analysis of qualitative data can have several aims. The first aim may be
to describe a phenomenon in some or greater detail. The phenomenon can be the
subjective experiences of a specific individual or group (e.g. the way people continue to
live after a fatal diagnosis). This can focus on the case (individual or group) and its
special features and the links between them. The analysis can also focus
on comparing several cases (individuals or groups) and on what they have in common
or on the differences between them. The second aim may be to identify the
conditions on which such differences are based. This means to look
for explanations for such differences (e.g. circumstances which make it more likely that
the coping with a specific illness situation is more successful than in other cases). The
third aim may be to develop a theory of the phenomenon under study from the
analysis of empirical material (e.g. a theory of illness trajectories).
ANALYSING QUALITATIVE DATA
Chapter 1 Mapping the field in Flick, U. (2014). The SAGE Handbook of Qualitative Data Analysis. Sage.

““
Content analysis is the intellectual process of categorizing qualitative
textual data into clusters of similar entities, or conceptual categories, to
identify consistent patterns and relationships between variables or
themes. Qualitative content analysis is sometimes referred to as latent
content analysis. This analytic method is a way of reducing data and
making sense of them—of deriving meaning. It is a commonly used
method of analyzing a wide range of textual data, including interview
transcripts, recorded observations, narratives, responses to open-ended
questionnaire items, speeches, postings to listservs, and media such as
drawings, photographs, and video.
CONTENT ANALYSIS
Julien, H. Content Analysis, in Given, L. M. (2008). The SAGE Encyclopedia of Qualitative Research Methods.
Sage.

““
Thematic analysis is a data reduction and analysis strategy by which qualitative
data are segmented, categorized, summarized, and reconstructed in a way that
captures the important concepts within the data set. Thematic analysis is
primarily a descriptive strategy that facilitates the search for patterns of
experience within a qualitative data set; the product of a thematic analysis is a
description of those patterns and the overarching design that unites them.
Thematic coding is the strategy by which data are segmented and categorized
for thematic analysis. Thematic coding is a strategy of data reduction, in contrast
to the axial and open coding strategies characteristic of grounded theory
research, which enrich and complicate data through the inclusion of analytic
insights and inquiries used.
THEMATIC ANALYSIS
Ayres, L. Thematic coding and Analysis, in Given, L. M. (2008). The SAGE Encyclopedia of Qualitative
Research Methods. Sage.

6 STEPS TO THEMATIC ANALYSIS
1. Familiarizing yourself with
your data
Transcribing data (if necessary), reading and re-reading the
data, noting down initial ideas
2. Generating initial codes Coding interesting features of the data in a systematic fashion
across the entire data set, collating data relevant to each
code
3. Searching for themes Collating codes into potential themes, gathering all data
relevant to each potential theme
4. Reviewing themes Checking if the themes work in relation to the coded extracts
(Level 1) and the entire data set (Level 2), generating a
thematic ‘map’ of the analysis
5. Defining and naming themes Ongoing analysis to refine the specifics of each theme, and
the overall story the analysis tells, generating clear definitions
and names for each theme
6. Producing the report The final opportunity for analysis. Selection of vivid,
compelling extract examples, final analysis of selected
extracts, relating back of the analysis to the research question
and literature, producing a scholarly report of the analysis
Braun, V. & Clarke, C. (2006) Using thematic analysis in psychology. Qualitative Research in Psychology, 3:2,
77-101

ARGH! THIS IS ALL TOO THEORETICAL!
• analysis is hard, and it’s hard to explain without a methodological
framework and/or dataset in mind
• there are many ways to do coding with thematic analysis
• we’ve listed some key readings on the blog post about this
workshop

QUICK AND DIRTY THEMATIC ANALYSIS
• option 1: collect all your data first
oget as familiar as possible with your whole dataset before you start analysis
by reading transcripts or listening to recordings
oas you read / listen, make notes about any topics that keep coming up –
create a bit of a code list
ocreate a series of documents – one for each broad ‘theme’ (evernote and
scrivener are great for this, or just create separate google docs)
oworking quickly through each transcript, ’quote dump’ key quotes into the
theme documents (remember to note who said what!)
oorganise your quotes into sub themes
owrite around the quotes

QUICK AND DIRTY THEMATIC ANALYSIS
• option 2: simultaneous data collection and analysis
owhen you analyse your data as you collect it, you can modify your
interview strategy to explore emerging themes in future interviews. this
often results in a smaller sample size
ouse the quote dumping approach

A COUPLE OF TIPS
• it’s usually faster if the interviewer conducts the analysis because
they know the data intimately
• get a second person to sense check your themes
• one piece of content (a sentence, a phrase, a paragraph) might fit
into many different themes

HERE’S A SAMPLE WE CODED
the general idea: fracture the data then put the story back
together
{And then, like, even then, have connected with other
people who have gone through similar things}, and
so that’s been good.
Connecting with others
Sharing similar experiences
Connecting with people around shared experience

STEP 1: READ THROUGH THE DATA
• as a group, read through the data in front of you
tables
8 minutes

STEP 2: THEME THE DATA
• working together, looking at each sentence / paragraph / concept
and apply a code or theme to it
tables
10 minutes

STEP 3: SHARE
• each table to share the themes they applied to one page of the
data
• two minutes max per table
tables
10 minutes

WHETHER YOU NEED ETHICS APPROVAL OR
NOT…
• understanding what is and isn’t ethical in terms of data collection,
analysis, privacy, disclosure etc is important
• take a look at the National Statement on Ethical Conduct in Human
Research, particularly:
oSection 1: Values and principles of ethical conduct
oSection 3: Ethical considerations in the design, development, review and
conduct of research

BASIC PRINCIPLES
• you’re dealing with people – respect them and take good care of
them and their personal information
ocreate a comfortable environment – small things like offering water are
important
omaintain privacy and anonymity throughout the research process and in
reporting
ominimise risks
otake good care of data
oexplain the project – consider providing an information sheet
oget informed consent and keep records

CONSENT
• always get informed consent
• design a consent form based on your institution’s ethics application
requirements, or ask your peers for examples
• we’ve included an example in your handouts
• getting consent now will help you later and just keeps everything
nice and tidy

RECAP: DESIGNING A
RESEARCH STRATEGY

KEY QUESTIONS TO ANSWER
1. what is your problem?
2. what is your research question?
3. what type of data do you need to answer the question?
4. does any of the data already exist? where? how can you get access?
5. what method/s will you use to collect the data?
6. what will your data collection instrument look like? (what will your
interview questions be? what will your survey questions be?)
7. who will you collect data from? who are your participants? what is your
sampling strategy?
8. how will you recruit participants?
9. what ethical considerations are there? how will you mitigate them? do
you need to apply for ethical approval?
10. how will you analyse the data?
11. how will you ensure quality?
12. how will you report on findings / outcomes? and why?

ACTIVITY: DESIGN A
RESEARCH STRATEGY

STEP 1: CREATE A RESEARCH QUESTION
• working together at your table, take one of the most significant
problems you highlighted this morning and write a research
question that will help you address it
• since it’s really important to get this part right, we’ll come back and
debrief this before moving on
tables
8 minutes

STEP 2: SHARE
• share the research question your table came up with
together
10 minutes

STEP 3: DESIGN YOUR STRATEGY
• work through the prompt questions in front of you to design a
research strategy
tables
15 minutes

STEP 4: SHARE
• share the strategy you designed, specifically:
oremind us of your research question
owhat kind of data will you collect?
owho will your participants be?
ohow will you find participants?
• two minutes per table
together
15 minutes

STOP! COFFEE TIME!
see you back here at 3.30pm

TAKE AWAYS
here’s what we hope you’re taking away

WE HOPE THAT YOU…
• are excited about using research in your engagement activities
• feel like you have a broad understanding of how you can go about
designing a research activity
• have some insight into how you can apply qualitative data
collection strategies like interviewing and focus groups
• have some insight into analysis techniques (and some
understanding of how to do ‘real world analysis’)

WHAT ARE YOU ACTUALLY
TAKING AWAY?

BUILDING AN
ENGAGEMENT TOOLKIT
how you can understand your customers, evaluate your
programs and services and tell stories of your impact
Dr Kate Davis
Digital Life Lab | USQ
@katiedavis
Kathleen Smeaton
Library Services | UQ
@kathleensme

LET’S EAT!
see you back here at 2pm

Building an engagement toolkit (Information Online 2019)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Building an engagement toolkit (Information Online 2019)

Ähnlich wie Building an engagement toolkit (Information Online 2019) (20)

Mehr von Kate Davis

Mehr von Kate Davis (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Building an engagement toolkit (Information Online 2019)