AAATE conference 2011 Anttila Heidi presentation

1. What do we know from systematic reviews
about the effectiveness of assistive products
for communication and information used by
people with disability?
Heidi Anttila, Kersti Samuelsson, Anna-Liisa Salminen, and Åse Brandt
Sept 2011 Heidi Anttila 1

2. Outline
• To describe a systematic approach to identify and
evaluate research evidence
• To summarize the available evidence
• To inspire clinicians to use the available evidence
• To point out potential pitfalls and inspire researchers
in doing better research in this field
Sept 2011 Heidi Anttila 2

3. Definition of the review question
• Populations: persons with disabilities
• Interventions: AT for communication and
information (ISO 9999:2007 classes: 22 09, 22 12, 22 21, 22 33,
22 36 and 22 39)
• Comparisons: no AT or other AT, or no
comparison
• Outcomes: all studied and reported outcomes
• Study types: systematic reviews that aimed to
evaluate effectiveness or harms of AT
Sept 2011 Heidi Anttila 3

4. Methods
A subsample of 5 reviews from a larger overview of
systematic reviews:
• Search from 12 databases 01/2000-04/2010, 2210
citations, selected 175 full text articles, included 44
systematic reviews (Anttila et al. Technol Disabil, 2011. In press)
• Process: Two authors independently
• Data extraction: Study types, sample sizes, types of
disabilities, settings, AT, outcome measures, outcomes.
• Methodological quality of the reviews: internal validity
by AMSTAR checklist (Shea BJ, BMC Medical Research Methodology 2007, 7:10)
and 2 questions on external validity
• Overall quality of evidence: principles from the GRADE
system. (GRADE Working group. BMJ 328, 2004:1490-4)
Sept 2011 Heidi Anttila 4

5. Quality of evidence
”Grading of Recommendations Assessment, Development, and Evaluation”
(BMJ 2004, 328: 1490-4)
High-quality One or more updated, high-quality systematic reviews that
evidence are based on at least 2 high-quality primary studies with
consistent results
Moderate- One or more updated systematic reviews of high or
quality moderate quality
evidence • Based on at least 1 high-quality primary study
• Based on at least 2 primary studies with methodological
limitations with consistent results
Low-quality One or more systematic reviews of variable quality
evidence • Based on primary studies of moderate quality
• Based on observational studies with control groups
• Based on inconsistent results in the reviews
• Based on inconsistent results in primary studies
Unclear Not possible to assess based on the information from the
review
• Quality of the included studies was not assessed
• Reporting was inadequate
Sept 2011 Heidi Anttila 5

6. Results
• The 5 reviews included:
– 1 review on AT for reading and writing
– 4 reviews on augmented and alternative
communication (AAC)
• 3 reviews had only minor methodolocical
limitations, 1 moderate and 1 major limitations.
• 3 reviews described the participants’ functional
limitations and 1 study contexts
• Overall quality of evidence remained unclear or
low (as derived from the primary studies, their
design, consistency and directness)
Sept 2011 Heidi Anttila 6

7. Effectiveness of AT for reading and
writing
• 1 review with moderate methodological limitations:
– effectiveness of AT for reading and writing for students with
reading, learning or mild cognitive disability (MacArthur et al.
2001)
• Computer-assisted speech-feedback when
reading: improved phonological
awareness, decoding skills and word
identification Quality of evidence: unclear
• Electronic text when reading: inconsistent effects
on text comprehension Quality of evidence: Unclear
• Word processing when writing: little effect on
quality of writing, slower text composition rate
Quality of evidence: Unclear
Sept 2011 Heidi Anttila 7

8. Effectiveness of AT for augmented and
alternative communication
• 4 reviews for children with autism or
developmental disorders evaluated:
– picture exchange communication systems (PECS)
– speech generating devices (SGD)
• PECS or SGD: gains in speech production, but
small and vary across individuals. Quality of evidence: LOW
(2 high-quality reviews by Schlosser & Wendt 2008, Millar et al. 2006)
• PECS increases initiations, but less eye contact
and challenging behaviour Quality of evidence: LOW (1 Low-quality
review by Schlosser & Sigafoos 2006)
• SGD increases spelling skills and spontaneous
speech, not effective in increasing vocalisations
Quality of evidence: LOW (1 High-quality review by Wendt 2007)
Sept 2011 Heidi Anttila 8

9. Limitations
• Possible publication bias, e.g. unpublished reviews?
• Rapidly accumulating reserach literature!
• Question of what evidence is enough?
Sept 2011 Heidi Anttila 9

10. Clinical implications
• A few high-quality systematic reviews are available!
• Use this synthesis as a core evidence resource for
clinical decision-making
• Do not use as a practice recommendation as such,
• Consider also: clinical experiences, costs, and your
local AT provision system and constraints.
Sept 2011 Heidi Anttila 10

11. Research implications
• Resources are needed for regular updating
• Methodological development needed for grading
small primary studies
• Only narrative conclusions on effectiveness, so the
effect sizes and their confidence intervals remain
unknown
• Need for participation or quality-of-life outcomes
• Methodological limitations and small sizes of
primary studies: an urgent need for further and
better quality research.
Sept 2011 Heidi Anttila 11

12. Conclusion
• This overview identified and summarized results of
5 reviews
• Three reviews were methodologically robust
• Despite low or unclear quality of evidence, potential
benefits of the reviewed AT were found and
summarized
Sept 2011 Heidi Anttila 12

13. Acknowledgements
• Special thanks to information specialists Jukka Lindeman, THL,
Finland and Silva Rintanen, Social Insurance Institution, Finland
• The authors declare no conflicts of interests.
• This study was fully financed by the author's institutions:
– National institute for Health and Welfare (THL), Finland;
– Linköping University Hospital, Clinical Department Rehabilitation
Medicine, Sweden
– Social Insurance Institution (Kela), Helsinki, Finland
– Danish Centre for Assistive Technology, Denmark
Sept 2011 13

14. Contact information
Anttila, H., Samuelsson K., Salminen A.-L. and
Brandt Å. Effectiveness of assistive technology
interventions for people with disability: an
overview of systematic reviews. Technol
Disabil, 2011. In press.
heidi.anttila@thl.fi
Sept 2011 Heidi Anttila 14