Slides were given as part of a presentation hosted by the National Library of Medicine's Boost Box program. The description of the session:
Miraida Morales will discuss the challenges of using easy-to-read health materials, such as their high reading level, lack of control or standardization of readability, and problems with readability formulas. In this session she will offer practical solutions for what librarians and other professionals can do to minimize these issues for our communities. Miraida will also share her research findings on how adult beginning and developing readers evaluate health information materials.
3. Agenda
Readability formulas
What are they?
How do they work?
Limitations
Should we be using
these formulas on
health materials?
Case Studies
How do different people
evaluate health
information?
4. Readability
a quality that determines how easy are
texts to read for a particular individual,
but...
identifying just what makes texts easy to
read remains a problem...
especially for health materials
6. Flesch Kincaid Grade Level = (0.39 x ASL) + (11.8 x ASW) - 15.59
Dale Chall = (Raw Score = 0.1579 * (PDW) + 0.0496 * ASL)
SMOG Formula =
SO MANY FORMULAS
! The formulas do not take into account meaning.
7. How do these formulas work?
They count the
average length of
words (syllables)
They count the
average number
of words in a
sentence
They match these
counts to grade
levels
8. Assumptions
longer words are more rare than shorter
words in English
&
longer sentences have more complex
structure
15. Let’s look at an example
Although gestational diabetes is a serious condition,
you can learn how to take care of it and prevent
problems for you and your baby. Because the placenta
leaves your body when the baby is delivered,
gestational diabetes usually goes away when the baby
is born. (American Diabetes Association)
22. Going beyond Easy-to-
Read materials
Don’t need to only
rely on docs in the
Easy-to-Read
collection
Readability of these
documents doesn’t vary
that much from
documents not included in
this collection. This means
we have a bigger selection
of topics available.
Readability formulas
are not necessarily
applicable to health
materials.
Vocabulary and sentence
length assumptions built
into readability formulas
don’t hold for health
materials. Health materials
tend to over-index on long
words. Health materials
don’t always include full
sentences (e.g. bulleted
lists, phases, etc.).
23. “Besides word and sentence length,
what features of text and reader
might better determine the
readability of health materials for
adult new readers?
25. [ Topic Knowledge ] [ Reading Expertise ] = X
What’s the relationship between these
factors?
26. CASE STUDIES WITH ADULT
READERS
2.
Adult women,
some high
school ed
3.
Evaluate
Diabetes
Brochures
1.
Diaspora
Community
Services
27. DESIGN FEATURES
Document Length
Preferred shorter
document to longer
one. One woman had
trouble recalling
information in the
longer document. This
is consistent with
perceived ease or
difficulty.
Charts and Lists
Though preferred,
these proved difficult
for participants to
interpret accurately.
Information presented
in these formats was
not necessarily easier
to recall.
Images
Provide visual markers
to anchor the text.
Allow for scanning.
Serve similar purpose
to section headings.
28. “That’s true.” AND “I know…”
Prior Knowledge
All participants viewed
favorably portions of
brochures that
resonated with their
prior knowledge.
Future Use
Usefulness of
information in the
brochures was an
important evaluation
criteria for
participants.
29. Sources of difficulty when reading
Statistics
Percentages and other
numerical figures were
difficulty for
participants to
interpret.
Word Use
Lack of definitions for
complicated health
terms, such as
gestational diabetes or
prediabetes created
uncertainty or
confusion.
Bulleted Lists
Because they often
lack function words or
phrases, information
presented in lists can
be difficult to interpret.
Participants often
expected list to follow a
narrative structure,
with each item linking
to the previous one.
30. MANY READERS, MANY TEXTS
A.
Types of
Readers
B.
Types of
Knowledge
C.
Types of
Texts
D.
Types of
Uses
34. References & More Reading
○ DuBay, W. H. (2004). The Principles of Readability. Costa Mesa: Impact Information, 76.
Retrieved from http://files.eric.ed.gov/fulltext/ED490073.pdf
○ Feng, L., Elhadad, N., & Huenerfauth, M. (2009, March). Cognitively motivated features for
readability assessment. In Proceedings of the 12th Conference of the European Chapter
of the Association for Computational Linguistics (pp. 229-237). Association for
Computational Linguistics.
○ McNamara, D. S., Graesser, A. C., McCarthy, P. M., & Cai, Z. (2014). Automated evaluation
of text and discourse with Coh-Metrix. Cambridge University Press.
○ Schriver, K. (2000). Readability Formulas in the New Millennium: What’s the Use? ACM
Journal of Computer Documentation, 24(3), 138–140.
○ http://www.readabilityformulas.com/
Hinweis der Redaktion
This is what we’re doing today. We’re going to take a deep dive into readability formulas. Then we’re going to consider what these formulas can and can’t do. Next, we’ll look at some general problems that arise when we apply these formulas to health information. Finally, I’ll present some initial findings of a series of case studies with adults as they evaluate health materials.
It’s important to be clear about what readability means, and why it’s problematic for health information. Simply put, is something easy to read? If yes, then we say it’s readable. However, language is multi-dimensional. When we talk about language, we are talking about vocabulary, meaning, and structure. It isn’t clear where the ease or difficulty lies: is it the words, the structure, or the meaning that makes something hard to read? This is really problematic for health information and we’re now going to take a look at why this might be so.
Let’s consider how we currently determine the readability of documents. Readability has been operationalized into mathematical formulas that make strong assumptions about the relationship of words and sentences to educational attainment.
Here are some well known and frequently used formulas. Formulas are very widely available. NOTE: The formulas do not take into account meaning.
Health terms, names of conditions, treatments, body parts, etc.
Rarity in relation to general language is not necessarily problematic. Think about a person who is dealing with a health condition--they already have some awareness or knowledge of an otherwise rare word.
Some readers need explanations, definitions, and context clues that will make text longer.
Not necessarily applicable to adults or to non-school texts such as newspapers, magazines, health information.
We now have the ability to use natural language processing techniques, the use of computers, we have lots and lots of machine readable texts to analyze. All of this can be done in a much grander scale, using more texts, faster, and cheaper.
Consider that health materials often include charts, graphs, lists, phrases, diagrams
Let’s take a step back and consider some of these assumptions and what all this looks like in our professional practice.
Consider this example. I’ve highlighted possible sources of difficulty in this segment.
1. Notice that both sentences start with a dependent clause. This might be stylistically pleasing to read, but it might be hard to read for a new/beginning reader.
2. Notice that placenta and its relationship to diabetes, is not explained.
Guidelines recommend the use readability formulas when creating health content.
NLM provides us with a collection of Easy-to-Read health information resources from various health agencies.
From the pov of authority and reliability, this service is indispensable.
Content creators
1. There is no verification system, so standardization is difficult to achieve.
2. Resources have not been evaluated by actual readers.
Terminology, jargon, structure. Think back to the gestational diabetes example presented earlier. Also, shorter doesn’t always mean easier to read. Formulas don’t tell us what in our document might cause difficulty in reading: is it word usage? Is it jargon? Is it long sentences? Is it complex grammar? Without knowing this, it’s difficult for us to amend or simplify text.
What do we do now?! Don’t despair.
Don’t limit ourselves to materials in the ETR collection--so in this way, our resources have grown. In fact, the readability (as determined by formulas) of docs in and out of the ETR collection is pretty similar/consistent across the board. Furthermore, readability formulas simply might not be well suited for health information. So what now?
What do I mean? Different people know different things about different health topics. Also, people have different levels of reading expertise. There are people who read a lot at work, for instance. There are people who read a lot in another language. There are people who read mostly novels. There are people who mostly read the news. All of these different reading practices contribute to different levels of expertise. Expertise includes the development of certain strategies: skimming, browsing, headline reading, speed reading, highlighting, note-taking, bookmarking, sharing, etc.
To learn more about how different adult readers evaluate health information, I carried out an exploratory case study. This early study has been expanded into a much bigger project, which I can discuss at the end if we have time and if you’re still interested. In any case, the earlier study was a collaboration with Diaspora (talk about Diaspora)