Dr Beck Taylor of Theme 1, Maternity and Child Health, presented her latest project, comparing a rapid approach to synthesising evidence from qualitative research to traditional research methods, presented at CLAHRC WM Programme Steering Committee meeting, 22nd October 2015

1. Rapid qualitative analysis v the
‘traditional approach’:
early findings and reflections
Beck Taylor, CRF Theme 1
29/10/2015

2. The team
Home Birth evaluation:
• Beck Taylor
• Sara Kenyon
• Cathy Shneerson
Comparative analysis:
• Ian Litchfield
• Sheila Greenfield
• Louise Bentham

3. Acknowledgements
• Birmingham Women’s Hospital, particularly the
Home Birth Team
• Gill Coombes
• CLAHRC peer reviewers

4. Background: the original project
• Dedicated Home Birth Service – an innovation
• CLAHRC asked “to evaluate”
• Identified model and its implementation evolving, not well-
defined
• Designed a project based on Evaluability Assessment
methodology
– Identify key documents and stakeholders
– Determine the programme theory, and whether it has been
implemented as expected, and why
– Define the data sources that are available
– Areas for evaluation
– Recommendations for changes to programme design,
monitoring and evaluation systems, and for potential future
evaluation opportunities

5. Home Birth Exploratory Review
• Rationale: model not clear and evolving, a need to capture design
and implementation process
• Aim: to describe the intended and actual service and its
implementation over the course of the pilot
• Objectives:
– Establish the original ‘programme theory’
– Once each year of the pilot
• Identify how the Service as it stands differs from the original
design, and why
• Identify any facilitators or barriers to implementation
• Establish what data are available with respect to the HBS, and
how it is being / could be gathered
• Identify the areas that staff would like to focus on in order to
develop/improve the HBS

6. Home Birth Exploratory Review
Method:
• Data collection:
– Semi-structured interviews with all key stakeholders willing/able
to participate (not service users in this initial review) (n=21)
– Focus group with sample of community midwives (n=13)
– Key service documents identified by participants (n=9)
• Analysis:
– Documents and interview/FG transcripts
– Descriptive findings plus recommendations
• Verification:
– Member checking of findings (presented in report and discussed
face to face) with strategic team and HBT midwives
• Dissemination:
– Report shared with trust, commissioners and participants
– Further dissemination planned following longitudinal work

7. At the design stage…
We determined that timely feedback of value
• Sharing of model/lack of model quickly
• Inform ongoing development and decision-making
• Information not out of date, useful
Was there a way to analyse date and deliver findings
more quickly?

8. Rapid analysis
Little in the literature (that we could find)
James Beebe: Rapid Assessment Process: An Introduction
(2001)
Alison Hamilton. Qualitative Methods in Rapid Turn-
Around Health Services Research. Presented online for
the US Department of Veterans’ Affairs (2013)

9. Rapid Assessment Process
(Beebe, 2001)
Key ideas:
• Speed, rapid turnaround and feedback, fewer resources
• Teams of researchers
• Concurrent data collection and analysis, iterative
• Researchers with less qualitative experience
• Analysis not about coding data – a different approach
• Rapid reduction of data using ‘summary templates’
• Future in-depth, more inductive work may follow RAP

10. Our Rapid Assessment Process
Key ideas:
• Speed, rapid turnaround and feedback, fewer resources
• Teams of researchers
• Concurrent data collection and analysis, iterative
• Researchers with less qualitative experience
• Analysis not about coding data – a different approach
• Rapid reduction of data using ‘summary templates’
• Future in-depth, more inductive work may follow RAP

11. Basic approach

12. Summary template

13. Questions arising…
• How does this approach work in practice?
• Does it deliver findings more quickly than ‘traditional’
qualitative research?
• Does it elicit similar findings to traditional approaches?
If not, how do they differ?
• What impact might any differences in findings have?
• Is this approach useful?
• What might the applications of this approach be?

14. “Can’t we do it the ‘normal’ way and see
how the two compare?”
(Thank you Dr Kenyon!)

15. Comparative analysis project
• Independent, blinded researcher (Ian Litchfield)
• Thematic analysis, Framework method
• Input from second researcher, Louise Bentham

16. Comparative analysis project
Two research questions
Is there a difference in researcher time requirements?
Researcher time log
Total hours
Commentary on how this time was used
Are findings different?
Independent scrutiny before cross-team data sharing (SG)
Comparison of findings generated under key research question
headings (numerical and qualitative)

17. Impressions so far: time
• RA seems to deliver outputs more rapidly (25-
50% less time)
• However, to compare accurately we need both
projects to reach the report/recommendations
stage, as processes not in parallel

18. Impressions so far: findings
• So far only scoping comparison of RA key
headings and TA themes by SG
• As with time, need full TA ‘report’ to compare
accurately, including interpretation and
recommendations
• RA seems to identify same key issues as TA
• TA seems to offer more granularity

19. Comparing the two teams
Rapid Analysis
(Beck and Cathy)
Thematic Analysis
(Ian and Louise)
Embedded in the field No connection to team or context
BT collected the data Did not collect data
Using RA for first time – learning time,
need to avoid usual practice
Experienced in TA – no method to
‘learn’, doing what comes naturally
Informal reflection day to day No space for informal reflection
Main focus of work over short period Project ‘squeezed in’ among other
commitments
Experienced qualitative researchers More experienced qualitative
researchers
Important we acknowledge the differences

20. Reflections
The process
• Requires time discipline for researchers used to a
traditional approach.
• Uncomfortable at times spending so little time with data,
tension with what we feel to be academically rigorous.
• Did not use summary templates for all data – switched to
direct entry into matrix. Any advantage to using separate
template?
• How would this work for novice researchers? Existing
skills definitely useful. Would they get the same results?
• How would this work in larger teams? Would it take longer
to synthesise data?

21. Reflections - applications
• Not a replacement for more in-depth approaches
• Advantages and disadvantages – it’s about the
context
• Useful for
– Rapid findings where time of the essence
– Identifying key priorities for action (breadth not depth)
– Identifying areas for more in-depth scrutiny

22. Intended output
• Methodology paper
• Comparing process, time taken and
findings between the two approaches
• Key limitations in our approach but ?
interesting to editors due to novelty and
potential impact/relevance

23. Questions and comments
• Impressions from the group
• Would this be of interest to others?
• What are the key aspects to focus on?
• Is there anything we can do to strengthen the work?
• Where should we aim our outputs?