The ethical use of
Supervision to facilitate
the Integration of Spirituality
in Social Work Practice
Jerry Jo M. Gilham
Although the use of spirituality and religiosity in social work
intervention has
been growing over the past few decades, little information is
available regard-
ing the supervisor’s contribution to this process. This article
outlines some of
the difficulties inherent in the process and recommends twelve
tasks required
of supervisors in facilitating the effective integration of
spirituality in social
work practice. It also explores how each of these tasks relates to
social work
values, ethics, and principles. Finally, it identifies policy
implications related
to this process.
S
ince the 1980s, the social work profession has experienced a
renewed interest in spirituality and religion (Canda & Furman,
1999).
The National Association of Social Workers (NASW) Code of
Ethics
mandates that social workers obtain education about and seek to

understand
the nature of diversity and oppression with respect to religion
(NASW,
2008). Current Council on Social Work Education (CSWE)
standards re-
quire schools of social work to demonstrate their commitment to
diversity
throughout their curriculum. Furthermore, graduates must
demonstrate
competence in engaging diversity and difference in practice
(CSWE, 2008).
While numerous definitions are offered for spirituality, religion,
and
faith, no universally accepted definitions exist, and the terms
are often
used interchangeably. Holloway and Moss (2010), as well as
Spencer
(1961), one of the earliest social workers to offer a definition of
spiritual-
ity, explain that spirituality is a broad concept that can include
religion,
but also has a secular appeal. Canda (1997) offers the following
definition,
Social Work & Christianity, Vol. 39, No. 3 (2012), 255–272
Journal of the North American Association of Christians in
Social Work
ARTICLeS
SOCIAL WORK & CHRISTIANITY256
which embraces these ideas. He defines spirituality as a search

for purpose,
meaning, and connection between oneself, other people, the
universe and
the ultimate reality, which can be experienced within either a
religious or
a nonreligious framework. A religious person, according to
Hugen (2001,
p. 13), is one who belongs to or identifies with a religious
group; accepts
and is committed to the beliefs, values, and doctrines of the
group; and
participates in the required practices, ceremonies, and rituals of
the chosen
group. Various social work authors, including Derezotes (2006),
Canda
and Furman (2010), and Holloway and Moss (2010) have
discussed the
ritualistic as well as the social aspects of religion. Faith,
according to Fowler
(1981), must be understood in order to comprehend a person’s
relationship
with the transcendent. He identifies three components of faith,
including
centers of value, images of power, and master stories.
Spirituality serves
as a more encompassing term (Rose, Westefeld, & Ansley,
2001) and will
be utilized in this article.
Spirituality and Lifestyle
Walsh and Pryce (2003) point out that we are experiencing
numerous
changes in our world, including the hectic pace of life, changing
family
forms, the inundation of images of violence and sex, and a sense

of isola-
tion. They contend these changes have left many people
yearning for inner
peace, a connection to others, and a sense of purpose and
coherence in life.
Derezotes (2006) concurs as he notes our world has become
increasingly
frightening, and our culture has experienced a spiritual hunger.
Hugen
(2001) and Zastrow (2007) assert that there are numerous issues
that
individuals confront today, many that have spiritual or religious
dimen-
sions. A few examples include reproductive technology,
physician-assisted
suicide, divorce, abortion, and prayer in public schools.
Spirituality and
faith are part of the daily lives of many people and Hage (2006)
observes,
“a significant component of one’s identity is spiritual and
religious heritage”
(p. 306). Many decisions people make on a daily basis, such as
what to eat
or choices about their health, frequently involve faith and
spiritual beliefs.
Numerous ceremonies and celebrations are centered on religious
or spiritual
events and various developmental or life stage transitions, such
as the birth
of a child, marriage, and death, frequently involve religion or
spirituality.
Evidence reveals that we live in a culture that is largely
Christian. Ac-
cording to the U.S. Census Bureau, of the two hundred and
twenty eight

million adults in the United States, in 2008, nearly one hundred
seventy
five million identified as Christian (Statistical Abstract of the
U.S., 2011).
Findings from Pew research in 2008 with over 35,000 U. S.
adults revealed
that eighty percent of those surveyed identified as Christian. In
addition,
92% believe in the existence of God, 61% over 18 hold
membership in a
religious group, and 80% say they pray, with 60% of
respondents praying
257
daily. Miller and Thoresen (2003) found similar results whereby
in their
study, 90% of U. S. adults expressed belief in God or a higher
power.
Spirituality and Intervention
According to Parker (2009), greater numbers of clients are
seeking
to address spiritual and religious issues in treatment. A national
survey of
professional social workers (Canda & Furman, 1999) revealed
that many
social workers find it acceptable to use spiritually related
interventions with
clients, and a majority of workers admit to using spirituality as
a resource
for clients. The data revealed that 59% of respondents used
spiritual books,

58% prayed for a client, 54% encouraged spiritual journal
keeping, and
71% have helped clients consider the spiritual meaning of life.
A study by
Hathaway, Scott, and Garver (2004) revealed, however, that
most therapists
do not regularly assess client’s spiritual issues, nor are these
issues part
of their treatment planning for clients. This tendency might
neglect to
identify resources and issues that might otherwise be important.
Griffith
and Griffith (2002) assert that opening a conversation about
spirituality
or religion depends more on careful listening to spontaneous
sharing than
knowing what questions to ask. They conclude that clients are
freer to share
about religion or spirituality when they feel their personhood is
respected.
According to Richards and Bergin (2005), there definitely are
situations in
which religious or spiritual interventions are contraindicated.
Peteet (1981)
offers this important juxtaposition regarding the use of a
client’s spiritual
and religious issues. He says, “Therapists who avoid discussion
of religious
issues miss opportunities to help patients integrate their
religious and emo-
tional selves; those who focus their attention on religious issues
risk losing
sight of their therapeutic task (p. 563).” Kochems postulates
that when a
practitioner does not ask a client about religion, that he or she
is actually

isolating religion rather than being neutral or objective.
Furthermore,
there may also be countertransference issues at work in
situations like this.
Although several authors have suggested that social work
educators
should accept responsibility for helping students develop
competency in
assessing and understanding the religious and spiritual beliefs
of their
clients and their implications for practice (Canda, 1989; Furman
1994;
Zastrow, 1999), many practitioners feel they did not receive
adequate
training through their academic program to integrate spirituality
and faith
in clinical practice. Efforts to incorporate formal training have
been slow
and limited, and research has revealed that schools of social
work have not
met the call to prepare social workers to integrate spirituality in
practice
(Barker, 2007; Dudley & Helfgott, 1990; Miller, 2001;
Sheridan, Wilmer,
& Atcheson, 1994). Hage (2006) pointed out that psychology
training
programs face similar limitations, as do counseling programs,
according
to Bishop, Avila-Juarbe, and Thumme (2003).
THE ETHICAL USE OF SUPERVISION
SOCIAL WORK & CHRISTIANITY258

Social workers typically become involved with individuals and
families when problems arise (Ferraro & Kelley-Moore, 2000),
and it is
during these trials that people frequently look to spirituality as
a resource
or coping mechanism. Research by Bart (1998) revealed that
60% of his
respondents believed that their faith could be utilized to address
most of
their problems. In addition, he found that four out of five
people wanted to
have their values and beliefs integrated into the counseling
process, and two
out of three preferred to see a counselor with spiritual values
and beliefs.
In a study of 56 social workers in Utah, 89% expressed that
spirituality is
an important part of social work practice and 91% said that
clients bring
up the subject (Derezotes & Evans, 1995). In a study of 142
social work-
ers in North Dakota, 33% indicated they frequently encountered
issues of
religion and spirituality in practice (Furman & Chandy, 1994).
A study by
Rose, Westefeld, and Ansley (2001) also found that clients want
to include
spiritual or religious issues in treatment. They advise that some
clients
have concerns regarding a clinician’s attempt to undermine their
beliefs or
to convert them, which obviously requires caution on the
clinician’s part.
Sermsbeikian (1994) suggests that a worker who respects the
client’s values

and beliefs may find that therapeutic benefits can be achieved
through them.
The positive mental and physical health benefits of spirituality
and
faith are well documented. According to Koenig, McCullough,
& Larson,
(2001) and Levin (1994), the vast majority of cohort studies
have shown
that religious beliefs and practices are consistently associated
with better
health outcomes, both physical and mental. Richards & Bergin
(1997) ex-
plain that people who pray believe that prayer has helped them
overcome
physical and psychological suffering. Other research has
revealed that
patients have successfully managed the recovery process by
effectively
addressing their spiritual needs (Burns & Smith, 1991; Isaia,
Parker &
Morrow, 1999; Seeman, Dubin, & Seeman, 2003). In a meta-
analysis of
studies involving 126,000 participants, religious participation
was associ-
ated with reduced mortality, especially for women
(McCullough, Hoyt,
Larson, & Koenig, 2000).
Integration of Spirituality in Professional Practice
Several recommendations have been offered regarding the needs
of
practitioners in effectively managing appropriate ethical
integration of
spirituality in professional practice. They include being aware

of one’s
personal biases regarding spirituality (Derezotes & Evans,
1995; Young &
Cashwell, 2010), the influence of countertransference feelings
(Kochems,
1993), understanding the major religions of the world (Dudley
& Helf-
gott, 1990; Hodge, 2002), understanding various models of
spiritual and
religious development (Barker, 2007; Young & Cashwell,
2010), being
informed regarding the religious beliefs and value systems of
one’s clients
259
(Canda, 1989; Furman 1994; Sermabeikian, 1994; Zastrow,
1999), having
knowledge and skill regarding assessment approaches (Hodge,
2002), being
willing to include spirituality in practice (Derezotes, 2006),
being comfort-
able addressing the spiritual and religious needs of one’s clients
(Koenig,
McCullough, & Larson, 2001; Matthews, McCullough, &
Larson, 1998;
Larimore, 2001), seeking specialized consultation when needed
(Young
& Cashwell, 2010), and providing an atmosphere of
understanding and
receptivity (Sheridan and Bullis, 1991). Details as to how to
successfully
prepare practitioners are scarce and not well-tested. As a result,
the ap-

proaches being used vary widely.
Recommendations for Supervisors
A search in EBSCO host databases including Medline, Social
Work
Abstracts, Sociological Index, ERIC, Academic Search
Complete, Academic
Search Premier and Psychological and Behavioral Sciences,
reveals a dearth
of information regarding issues of supervision and supervisor
responsibili-
ties related to integration of spirituality in practice. Of one
hundred and
twenty articles identified using keywords supervision and
spirituality, only
about twenty referenced supervision. The absence of
information on this
topic was particularly evident in social work journals.
Today’s supervisors must be prepared to guide supervisees
regarding
the integration of spirituality in practice and oversee their
efforts in doing
so. Several authors, including Coffey, Frame, and Haug (cited
in Miller,
Korinek, & Ivey, 2004), recommend that spirituality should be
discussed
in supervision. Miller, Korinek and Ivey (2006) point out there
is little
certainty regarding the role and significance of spirituality in
supervision.
They explain that no evidence is available regarding the
frequency and
form of spiritual issues in supervision.

The role of the supervisor is to further enhance the supervisee’s
knowl-
edge base, values, and skills while monitoring and evaluating
their work.
Munson (2002) and Shulman (2005) explain that supervision is
an inter-
actional process, whereby supervisor and supervisee work
cooperatively in
a mutual effort to explore material and make decisions for client
benefit.
Both authors emphasize the necessity of a trusting relationship
between
the parties for effective supervision. Furthermore, according to
Shulman,
the supervisee learns more from the interaction itself than from
what is
said in the interaction. That is, supervisor interaction with the
supervisee
should model the type of relationship that reflects a helping
professional
relationship. As Williams (1997) explains, the supervisory
relationship
and the therapeutic relationship both focus on learning, personal
growth,
and empathy.
Munson outlines several important tasks of supervisors. These
in-
clude reading professional literature to keep apprised of timely
informa-
THE ETHICAL USE OF SUPERVISION
SOCIAL WORK & CHRISTIANITY260

tion that must be shared with supervisees, keenly observing the
practices
and performance of supervisees, and discussing issues and
concerns with
supervisees. These tasks are needed to direct and guide the
supervisee in
their work. Munson’s principles provide some direction for the
following
recommended list of tasks, which is also based on the literature
and my
own clinical experiences. These tasks provide direction for the
effective and
ethical integration of spirituality in practice through clinical
and reflective
supervision. How each of these tasks relates to social work
values, ethics,
and principles is also included.
1. Just as the supervisee must examine his or her personal
beliefs and biases,
a supervisor must examine his or her own beliefs and biases.
This will typically include a review of one’s spiritual journey,
includ-
ing family influence and expectations. Frame (2003) and Ripley,
Jackson,
Tatum, and Davis (2007) point out that having practitioners
assess spiritual
and religious issues in their own families may influence their
effectiveness
with clients. Bufford (2007) explains that being cognizant of
our worldviews
may lead to greater insight into the beliefs underpinning our
supervisory
roles and may lead to more effective supervisory relationships.

This self-awareness is addressed in the NASW Code of Ethics,
which
outlines that social workers should be aware of their personal
values and
cultural and religious beliefs and practices, and how they
impact profes-
sional decision-making. Brody (2005) explains that the effective
manager
engages in continuous self-appraisal. Kirst-Ashman and Hull
(2009) write
that insufficient self-awareness is a barrier to culturally
competent social
work practice. They add that assuming clients think as you do is
a bar-
rier for workers. Furthermore, supervisors must be aware of any
conflicts
between their personal and professional values, especially if
they are to
advise supervisees in this regard.
2. The supervisor must be comfortable with the use of a client’s
beliefs and
values in the intervention process in order to successfully aid
the supervisee
in this integration.
Many authors have pointed out the importance of a
practitioner’s
comfort level regarding personal belief systems in addressing
the spiritual
and religious needs of their patients and clients (Koenig,
McCullough, &
Larson, 2001; Matthews, McCullough, & Larson, 1998;
Larimore, 2001).
Bishop, Avila-Juarbe, and Thumme (2003) observe that

counselors do
not always find it easy to approach clients’ concerns about
spirituality or
religion, and that there has been a negative bias in the past
regarding the
influence of spirituality on client well-being. Young and
Cashwell (2010)
maintain that all individuals have a religious history that affects
how they
view religion and the religious views of their clients. Goldstein
(1987) as-
serts that spiritual bias can be as dangerous as other biases such
as racism
261
or sexism. Of course, even a pro-spirituality bias can be
potentially prob-
lematic. Raines (2003) found that social workers who share
similar beliefs
with clients tend to underestimate the level of dysfunction in
their clients.
Interestingly, there is some research that indicates that
psychologists have
a tendency to perceive religious clients with greater pessimism
than those
who are nonreligious (cited in Aten & Hernandez, 2004).
According to the Code of Ethics, a social worker must be
mindful of indi-
vidual differences and diversity while treating each individual
with care and
respect. Conversely, shared religious beliefs can serve as a
means of resistance

in the therapeutic process (Kehoe & Gutheil, 1984.) Spero
(1981) cautions
that similarity between therapist and client may limit the
former’s sensitivity
to the client and his or her ability to fully accept the client. A
study by Probst,
Ostrom, Watkins, Dean, and Mashburn (1992) revealed that
shared values
may not necessarily enhance therapeutic outcomes. In addition,
as explained
by Carroll (1997), workers must be alert to the client’s point of
view as well
as their level of awareness of their spirituality. Canda (2005)
explains that
addressing spirituality is consistent with the mission of
promoting dignity,
respect, and well-being as well as the person-environment
perspective. It is
important for supervisors and other social workers to maintain
an informed
balance between assuming too much when there is a perceived
similarity in
values or religious beliefs and making intrusive interventions,
on the one
hand, or (conversely) being blinded to clients’ possible
“spiritualized” ma-
nipulations in the helping process, on the other.
3. The supervisor must monitor and oversee the supervisee’s
willingness
to accept the client’s perspective, perhaps particularly when
their views
conflict, and limit any effort to influence the views or beliefs,
including
religious orientation, of their clients.
It is entirely possible that a supervisee and client may have

conflicting
beliefs. According to Richards and Bergin (2005), clients must
understand
that they have the right to different religious or spiritual views
from those
of their therapist and that their therapist will not try to proselyte
or convert
them. Zastrow (2003) states that a social worker should never
behave in a
way that might be viewed as seeking to convert a client. A
supervisee is not
justified in overriding a client’s belief system (Linzer, 2006;
Martin, 2008),
and the supervisor is responsible for guarding against correcting
a client’s
belief system. Martin adds that everyone loses when spiritual
understand-
ing is missing. Aten and Hernandez (2004) warn that
supervisors and
supervisees must not make assumptions regarding a person’s
beliefs based
solely on the person’s religious affiliation. As many have noted,
religion
and spiritual practices have both healthy and unhealthy aspects
(Koenig,
McCullogh, & Larson, (2001; Richards & Bergin, 1997.
This task relates to client self-determination, an ethical
responsibility
outlined by the NASW Code of Ethics. According to the Code,
social workers
THE ETHICAL USE OF SUPERVISION

SOCIAL WORK & CHRISTIANITY262
must both respect and promote the right of clients to socially
responsible
self-determination. This further relates to the value of dignity of
the human
person. If we are to honor an individual’s dignity, we must
respect their
beliefs and their right to autonomous decision making.
4. The supervisor can help the supervisee to determine how the
client’s
worldview and beliefs are impacting their situation.
In some cases this influence will be positive and can serve as a
source
of strength in dealing with their circumstances. In others, a
client’s belief
system may actually have a negative influence. Sermabeikian
(1994) ex-
plains that we can allow preconceived notions about what may
be helpful
to enter into our thinking. A worker must determine whether the
beliefs
provide trust and hope in the midst of difficulties or foster fear
and guilt
(Frame, 2003; Sermabeikian, 1994). Bishop (1995) asserted that
it is im-
portant for workers to help clients see that religious values are
an accepted
part of the therapeutic process as well as part of the solution. A
plethora
of studies support the use of one’s spirituality as a helpful
resource, and
the benefits for dealing with health and mental health issues are

clearly
evidenced (Burns & Smith, 1991; Isaia, Parker & Morrow,
1999; Koenig,
McCullough, & Larson, 2001; Levin, 1994; Richards & Bergin,
1997; See-
man, Dubin & Seeman, 2003).
This supervisory task is associated with the notion of client
empower-
ment. Gutierrez (1990, p 149) defined empowerment as “the
process of
increasing personal, interpersonal or political power so that
individuals can
take action to improve their life situations.” According to Kirst-
Ashman
and Hull (2009), empowerment allows clients to nurture their
strengths,
in this case, spirituality, to assert control over their life. Where
beliefs are
perceived to be negative, a worker may seek to alter or
neutralize the influ-
ence of the belief system. The supervisor can assist the
supervisee in this
process of helping clients explore and evaluate their beliefs
without the
imposition of the supervisee’s beliefs or evaluations. Richards
and Bergin
(2005) state that therapists should avoid condemnation of
clients for value
and lifestyle choices. Rather, they should help clients examine
consequences
of their choices. Furthermore, the authors suggest that it is
inappropriate
for a therapist to tell a client that they are bad or deficient
because of their
behavior or choices or attempt to shame them.

5. The supervisee and supervisor must be willing to
acknowledge any ideo-
logical or philosophical issues that might interfere with their
relationship
and their ability to address spiritual issues in intervention.
The potential for differences is particularly likely as it relates
to per-
sonal belief systems. On the other hand, similar views or beliefs
may serve
as an obstacle or barrier in the relationship, just as it may in the
relationship
between therapist and client (Kehoe & Gutheil, 1984). Peteet
(1981) and
263
Spero (1981) agree that workers can over-identify with clients
and inflate
their level of functioning, which may also impact the supervisor
and su-
pervisee relationship. From the work of Young and Cashwell
(2010), we
understand that individuals, such as supervisor and supervisee,
may be at
differing points in their spiritual development, which will lead
to differing
perspectives and expectations.
Bufford (2007) explains that a Christian worldview may
influence one’s
choices about the means, motives and goals of supervision,
consequently

affecting the supervisor and supervisee. Rosen-Galvin (2005)
found that
supervisors are more willing than therapists to discuss issues
related to val-
ues and spirituality. Bishop, Avila-Juarbe and Thumme (2003)
recommend
that in the future, researchers further investigate the impact of
similarities
and differences between supervisor and supervisees’ spiritual
values on
the supervision process.
The professional relationship between worker and client is vital
to
the helping process, and a healthy relationship between
supervisor and
supervisee is equally essential. According to Brody (2005),
effective leaders
and supervisors must create a climate in which staff members
feel positive
about how they are being treated in order to perform at their
highest level.
He asserts that the trust that develops between them is a result
of a com-
mitment to one another that is based on a willingness to
challenge issues
while maintaining respect for one another.
6. Supervisors must encourage supervisees to seek knowledge
and under-
standing of various faith and religious traditions, however the
supervisee
must establish boundaries between his or her role and that of
spiritual
advisor or teacher.
The supervisor is responsible for ensuring the supervisee does

not
attempt to act in the role of spiritual teacher or advisor. Miller,
Korinek,
and Ivey (2006) and Zastrow (2007) state that practitioners must
clearly
delineate their roles and avoid tasks that should be reserved for
clergy mem-
bers. Richards and Bergin (2005) assert that workers must not
undermine
the authority and credibility of a client’s religious leader by
attempting to
displace them or neglecting to consult them. Hage (2006)
explains that
while both therapists and pastors provide counseling, therapists
must not
carry out the functions of the clergy such as performing
religious rituals
or giving blessings. A supervisor’s job is to monitor the
supervisee’s ability
to perform competently and within his or her area of expertise.
Staff members and agencies must have community or clergy
members
whom they can call on for consultation and advice in cases
where they need
such guidance (Young & Cashwell, 2010). The Code of Ethics is
clear in
stating that social workers must seek advice and counsel
whenever such
consultation is in the clients’ best interest. Clearly, social
workers are not
trained for the role of spiritual advisor or leader and must not
assume this
THE ETHICAL USE OF SUPERVISION

SOCIAL WORK & CHRISTIANITY264
role, regardless of their personal values or convictions. Further,
the worker
must be aware of an individual’s area of expertise when seeking
information
and guidance from that individual.
7. The supervisor must protect clients by ensuring the
supervisee does not
impose his or her values on the client.
Although one study found no evidence that those therapists who
use
a client’s spirituality in practice are more likely to impose their
values on
clients (Smith & Richards, 2005), there is always the potential
for it to oc-
cur. Richards and Bergin (2005) offer many suggestions to help
clinicians
respect client values and avoid imposition of their own values.
Lannert
(1991) tells us of the importance of self-monitoring regarding
personal
resistances, countertransference issues and value systems
regarding spiri-
tual and religious issues so that we are both ethical and
efficacious in our
work with clients.
The Codes of Ethics of all the helping professions mandates that
professionals must honor and respect their clients and their
personal

values (Codes of Ethics, 2007). Indeed, empowerment,
autonomy, and
self-determination are highly valued principles of social work
practice and
must be safeguarded.
8. Supervisors must know their staff well, including staff
members’ tendencies
regarding the use of spirituality in their work.
Some workers may be more inclined to address spiritual and
religious
issues while others may be more hesitant. Bishop (1995)
suggested that
counselors must be aware of how resistance or caution about
religious
issues might be perceived by clients. Kochems (1993) revealed
that even
clinicians who feel positively about a client’s faith may be
under-involved
with a religious client. West emphasizes that all interventions
should be
utilized only with client consent and with client benefit in mind.
West
(2011) and Richards and Bergin (2005) discuss the possible
imposition
of beliefs on clients based on personal symbols and material
found in the
practitioner’s office. West adds that therapists must be aware of
how easily
their beliefs can be imposed on clients, especially more
vulnerable ones.
Campbell (2007) warns that many supervisees may feel pressure
to hast-
ily implement techniques that promote spiritualization of
problems or to

perceive problems as related to sin rather than psychological
dysfunction.
Supervisors must be aware of staff intentions and the purity of
those
intentions. As the core value of service expresses, workers must
elevate
client needs above their own. Professional integrity demands
that workers
conduct themselves honestly and responsibly. Brody (2005)
advises that
a supervisor must help staff reflect on their work and provide
feedback
regarding their efforts. In addition, he writes that supervisors
must clearly
and continually convey expectations to staff since there is a
positive as-
265
sociation between understanding and performance. Different
supervision
tasks are needed for different therapists according to their
experience and
skill level (Gingrich & Worthington, 2007).
9. The supervisor and supervisee must continuously process
changing beliefs
or values as a result of their experiences with clients.
Self-knowledge and understanding are essential to effective
practice
(Boyle, Hull, Mather, Smith, & Farley, 2006). West (2011)
shares Wyatt’s

conclusion that one’s faith and beliefs change over time, but
understanding
one’s current beliefs and being comfortable with them allows
the therapist
to be present to the client. West emphasizes the positive aspects
of the
changing nature of one’s beliefs and values. Further support of
this notion
is offered by Young and Cashwell (2010) who express that
spiritual and
religious life is a developmental construct that can evolve as
one’s thinking
and experience evolve.
The Council on Social Work Education (CSWE) identifies
ongoing
growth and development as facilitators of professional
enhancement. Some
states require that licensed professionals obtain continuing
education units
(CEUs) on values and value issues. Kirst-Ashman and Hull
(2009) relate
that workers must conduct ongoing evaluation of personal
values and their
influence on their effectiveness in intervention.
10. The supervisor must be aware of and acknowledge the
research that dem-
onstrates the benefits of spirituality for dealing with many life
situations.
This empirical evidence must be shared with supervisees so they
can
utilize the information in practice. A sampling of this evidence
was shared
earlier in the literature review. Zastrow (2007) tells us that

social workers
should only use interventions that have evidence of proven
therapeutic value.
The current demand for this type of evidenced-based practice in
social
work cannot be more pronounced than in the expectations of the
CSWE
and NASW. CSWE standards require that schools of social work
prepare
students for research-informed practice and the Code of Ethics
mandates
that workers are current in their knowledge and utilize research
in practice.
Hage (2006) explains that a lack of knowledge of spiritual
benefits can
diminish the supervisee’s repertoire of approaches with clients.
11. Supervisors must recognize when a client should be referred
to a clergy
member or spiritual advisor or even a clinician who is better
prepared to
address the client’s needs.
The supervisors’ role in this case is to ensure that clients are
directed
to the appropriate resources (Aten and Hernandez, 2004).
According to
Young and Cashwell (2010), it is not uncommon for clinicians
to see clients
who have needs for spiritual development that are beyond the
clinician’s
or supervisor’s scope of competence.
THE ETHICAL USE OF SUPERVISION

SOCIAL WORK & CHRISTIANITY266
According to the Code of Ethics, a worker must be dedicated to
prac-
ticing within his or her area of knowledge, training, and skill
level. This
includes being sensitive to a client’s culture and providing
services that are
sensitive to a client’s spiritual culture.
12. The supervisor must be keenly aware that some settings are
more likely
to involve situations regarding religious beliefs or spirituality.
Some examples of settings more likely to involve religious
beliefs or
spirituality in one way or another include practice in settings
such as mental
health, addictions, medical social work, hospice, care of the
elderly, church
social work, and work with the GLBT population. According to
Derezotes
(1995), there is not a single bio-psycho-social problem that does
not have
a spiritual component, but he also acknowledges that a person’s
spiritual
dimension or awareness might be underdeveloped (2006).
Green, Fullilove,
and Fullilove (1998) said they were struck by how often
spiritual issues
were part of the conversation with drug abusers. Anticipation of
spiritual
issues allows workers and supervisors to better plan for
potential situations,

including countertransference issues.
Agency and Policy Implications
The absence of direction regarding spiritual integration is even
more
pronounced when the matter of agency policy is introduced.
Administrators
must consider several questions dedicated to the development of
policy in
the process of spiritual integration.
(1) The principal or primary question relates to whether an
agency
has a policy that addresses integration of spirituality in practice.
If they do,
further questions include whether staff members are aware of
the policy and
how the policy is to be implemented within the organization.
The policy
must outline any restrictions regarding the use of spirituality as
well as
guidelines for assuring its ethical and effective use. Staff must
be aware of
the policy and expectations for their behavior. Staff must not
only be aware
of the formal policy, but how the policy is implemented within
the agency.
That is, some supervisors or staff members may be more open to
the process
while others are less flexible. This confusion or contradiction
may lead to
uneven implementation and prohibit the positive aspects of the
practice.
(2) The next policy issue relates to clients’ rights. One

important
factor is whether clients have the choice regarding whether they
choose
a worker with similar or dissimilar beliefs. As has been
explained above
there are both positive and negative consequences when worker
and client
share belief systems. Raines (2003) warns that spiritual
similarity between
therapist and client often leads to countertransference and
minimization
of perception of client psychopathology. On the other hand, he
also points
out that there is some evidence that reveals its benefit.
267
As indicated in the literature, many clients are looking to use
their
spirituality as a resource and want a worker who is willing to
incorporate
spirituality in the process. Agencies must consider the practical
implications
of this practice, including identifying those staff members who
are comfort-
able being identified as willing to use spirituality in practice,
those who
have been trained to do so, and how these staff members will be
supervised.
(3) Each agency must have a policy regarding corrective action
when
a worker engages in a boundary violation related to spirituality
such as

proselytizing or imposing one’s values on a client. Agencies
must have
clear definitions of these violations and specific guidelines for
reporting
and dealing with these situations. Administrators must not be
overzealous
in pursuing disciplinary action.
(4) Another policy issue revolves around dealing with workers
who
prefer not to deal with religious or spiritual issues of clients or
who have
insufficiently developed skills in this area. Even workers who
associate
with a faith tradition may not be comfortable using spirituality
in inter-
vention. Agencies must consider this possibility and develop a
plan for
this potentiality.
(5) Agencies must consider the implications for a worker who
resists
offering intervention services to a client who holds differing or
opposing
values or beliefs. Organizations must develop policies and
procedures for
staff regarding this type of situation. By doing so, agencies will
likely reduce
or prevent uncomfortable situations and areas of conflict.
Furthermore,
agencies must address whether workers are to be expected to
work with
all clients regardless of conflicting value systems.
(6) And finally, an agency will have to make decisions
regarding how

they will publicize or market their use of spirituality in
practice, or if they
will do so at all. Some clients may be attracted to an agency on
this basis
while others may be deterred from seeking services due to
reluctance to
discuss what may be deemed a highly personal matter. Clearly,
agencies
will increasingly be faced with these types of questions and
dilemmas.
Unfortunately, the research will not provide enough direction
and answers
at this point, leaving this an area ripe for study.
Conclusion
The use of spirituality and religiosity in clinical practice is
certain
to expand. Sherwood (1999) explains that competent integration
is
intentional, not accidental, and must be undertaken with
integrity and
responsibility. Many challenges remain regarding successful
integration,
especially surrounding choices of both clients and clinicians as
partners in
the intervention process. While some research outlines the
advantages of
shared belief systems between client and clinician, other
research indicates
that there may be a neutral, if not negative, impact of shared
beliefs and
THE ETHICAL USE OF SUPERVISION

SOCIAL WORK & CHRISTIANITY268
values. These contradictory findings pose difficult questions for
agencies
and indicate the need for clear and specific policies. West
(2011) offers
that all practice regarding spirituality needs to be done from a
clear ethical
perspective, which protects both client and clinician.
It seems clear that more attention must be directed to the
supervisor’s
role in the integration process. Martin (2008) poses an
important ques-
tion when she asks how we can possibly consider excluding
spirituality in
supervision if we emphasize the importance of spirituality in
assessment
and practice. Although some literature is available to guide
supervisors in
the integration process, and additional guidelines are offered
here, these
approaches and tasks have not been empirically tested. Research
is required
to determine the most effective and ethical approach to
supervising this
process. In any event, further training regarding this integration
process
must be a priority in social work education programs and social
service
organizations for staff at all levels, but especially for
supervisory staff. v
reFerenceS

Aten, J. D. & Hernandez, B. C. (2004). Addressing religion in
clinical supervision:
A model. Psychotherapy Theory, Research, Practice, &
Training, 41(2), 152-160.
Barker, S. L. (2007). The integration of spirituality and religion
content in social
work education: Where we’ve been, where we’re going. Social
Work & Chris-
tianity, 34(2), 146-166.
Bart, M. (1998). Spirituality in counseling: Finding believers.
Counseling Today,
41(6), 1,6.
Bishop, D. R. (1995). Religious values as cross-cultural issues
in counseling. In M.T.
Burke & J. G. Miranti (Eds). Counseling: The spiritual
dimension (pp. 59-71).
Alexandria, VA: American Counseling Association.
Bishop, D. R., Avila-Juarbe, E., & Thumme, B. (2003).
Recognizing spirituality as
an important factor in counselor supervision. Counseling and
Values, 48, 34-46.
Boyle, S.W., Hull, G. H., Mather, J. H., Smith, L. L., & Farley,
O. W. (2006). Direct
practice in social work. Boston, MA: Allyn & Bacon.
Brody, R. (2005). Effectively managing human service
organizations, 3rd Ed. Thousand
Oaks, CA: Sage Publications.
Bufford, R. K. (2007). Philosophical foundations for clinical
supervision within a

Christian worldview. Journal of Psychology and Christianity,
26(4), 293-297.
Burns, C. M. & Smith, L.L. (1991). Evaluating spiritual well-
being among drug and
alcohol-dependent patients: A pilot study examining the effects
of supportive/
educative nursing interventions. Addictions Nursing Networks,
3, (3), 89-94.
Campbell, C.D. (2007). Integrating Christianity across the
supervisory process.
Journal of Psychology and Christianity, 26(4), 321-327.
Canda, E. R. (1989). Religious content in social work
education: A comparative
approach. Journal of Social Work Education, 25(1), 36-45.
Canda, E. R. (1997). Spirituality. Encyclopedia of social work:
1997 supplement (19th
ed.). Washington, DC: NASW Press.
269
Canda, E. R. (2005). The future of spirituality in social work:
The farther reaches
of human nature. Advances in Social Work, 6(1), 97-108.
Canda, E. R. & Furman, L. D. (1999). Spiritual diversity in
social work practice: The
heart of helping. New York, NY: The Free Press.
Carroll, M. M. (1997). Spirituality and clinical social work:
Implications of past

and current perspectives. Arete, 22(1), 25-34.
Codes of Ethics for the Helping Professions, 3rd Ed. (2007).
Pacific Grove, CA:
Brooks/Cole.
Council on Social Work Education. (2008). Educational policy
and accreditation
standards. Alexandria, VA: Author.
Derezotes, D.S. (1995). Spirituality and religiosity: Neglected
factors in social work
practice. Arete, 20(1), 1-15.
Derezotes, D. S. (2006). Spiritually-oriented social work
practice. Boston, MA: Al-
lyn & Bacon.
Derezotes, D. S. & Evans, K. E. (1995). Spirituality and
religiosity in practice: In-
depth interviews of social work practitioners. Social Thought,
18(1), 38-56.
Dudley, J. R. & Helfgott, C. (1990). Exploring a place for
spirituality in the social
work curriculum. Journal of Social Work Education, 26, 287-
294.
Ferraro, K. F. & Kelley-Moore, J. A. (2000). Religious
consolation among men and
women: Do health problems spur seeking? Journal of the
Scientific Study of
Religion, 39(2), 220-234.
Fowler, J. (1981). Stages of faith. San Francisco, CA: Harper &
Row.

Frame, M. W. (2003). Integrating religion and spirituality into
counseling: A compre-
hensive approach. Thousand Oaks, CA: Brooks/Cole.
Furman, L. E. (1994). Religion and spirituality in social work
education: Preparing
the culturally sensitive practitioner for the future. Social Work
& Christianity,
21, 103-115.
Furman, M. W. & Chandy, J. M. (1994). Religion and
spirituality: A long-neglected
cultural component of rural social work practice. Human
Services in the Rural
Environment, 17(3/4), 21-26.
Gingrich, F. & Worthington, E. L. (2007). Supervision and the
integration of faith
into clinical practice: Research considerations. Journal of
Psychology and
Christianity, 26(4), 342-355.
Goldstein, H. (1987). The neglected moral link in social work
practice. Social
Work, 32, 181-186.
Green, L. L., Fullilove, M. T. & Fullilove, R. E. (1998). Stories
of spiritual awaken-
ing: The nature of spirituality in recovery. Journal of Substance
Abuse Treatment,
15(4), 325-331.
Griffith, J. L. & Griffith, M. E. (2002). Encountering the sacred
in psychotherapy: How
to talk with people about their spiritual lives. New York, NY:

The Guilford Press.
Gutierrez, L. (1990). Working with women of color: An
empowerment perspective.
Social Work, 35(2), 149-153.
Hage, S. M. (2006). A closer look at the role of spirituality in
psychology training
programs. Professional Psychology: Research and Practice.
37(3), 303-310.
Hathaway, W. L., Scott, S. Y., & Garver, S. A. (2004).
Assessing religious/spiritual
functioning: A neglected domain in clinical practice?
Professional Psychology:
Research and Practice, 35, 97-104.
THE ETHICAL USE OF SUPERVISION
SOCIAL WORK & CHRISTIANITY270
Hodge, D. R. (2002). Equipping social workers to address
spirituality in practice
settings: A model curriculum. Advances in Social Work, 3(2),
85-103.
Holloway, M. & Moss, B. (2010) Spirituality and social work.
New York: Palgrave
Macmillan.
Hugen, B. (2001). Spirituality and religion in social work
practice: A conceptual
model. In Van Hook, M., Hugen, B., & Aguilar, M. (2001).
Spirituality within

religious traditions in social work practice (pp. 9-17). Thousand
Oaks, CA:
Brooks/Cole.
Isaia, D., Parker, V., & Murrow, E. (1999). Spiritual well-being
among older adults.
Journal of Gerontological Nursing, 25(8), 15-21.
Kehoe, N. & Gutheil, T. G. (1984). Shared religious belief as
resistance in psycho-
therapy. Journal of Psychotherapy, 38(4), 579-585.
Kirst-Ashman, K. K. & Hull, G. H. (2009). Understanding
generalist practice (5th
Ed.). Thousand Oaks, CA: Brooks/Cole.
Kochems, T. (1993). Countertransference and transference
aspects of religious mate-
rial in psychotherapy: The isolation or integration of religious
material. In. M.
L. Randour, (Ed.), Exploring sacred landscapes: Religious and
spiritual experi-
ences in psychotherapy (pp. 34-54). New York, NY: Columbia
University Press.
Koenig, H. G., McCullogh, M. E. & Larson, D. B. (2001).
Handbook of religion and
health. New York, NY: Oxford University Press.
Lannert, J. L. (1991). Resistance and countertransference issues
with spiritual and
religious clients. Journal of Humanistic Psychology, 31(4), 68-
76.
Larimore, W.L. (2001). Providing basic spiritual care for
patients: Should it be the

exclusive domain of pastoral professionals? American Family
Physician, 63,
36-40.
Levin, J. S. (1994). Religion and health: Is there an association
is it valid and is it
causal? Social Science and Medicine, 38, 1475-1482.
Linzer, N. (2006). Spirituality and ethics in long-term care.
Journal of Religion and
Spirituality in Social Work, 25(1), 87-105.
Martin, L. (2008) The impact of religion/spirituality in social
work supervision.
Paper presented at the North American Association of
Christians in Social
Work (NACSW) Convention, Orlando, FL.
Matthews, D. A., McCullough, M. E., & Larson, D. B. (1998).
Religious commit-
ment and health status: A review of the research and
implications for family
medicine. Archives of Family Medicine, 7, 118-124.
McCullough, M. E., Hoyt, W. T., Larson, D. V. & Koenig, H. G.
(2000). Religious
involvement and mortality: A meta-analytic review. Health
Psychology, 19,
211-222.
Miller, D. W. (2001). Programs in social work embrace the
teaching of spirituality.
Chronicle of Higher Education, 47(36). Retrieved from EBSCO
Host on May
18, 2007.

Miller, M. M., Korinek, A., & Ivey, D. C. (2004). Spirituality in
MFT training: De-
velopment of the spiritual issues in supervision scale.
Contemporary Family
Therapy, 26(1), 71-81.
Miller, M. M. & Korinek, A., & Ivey, D. C. (2006). Integrating
spirituality into
training: The spiritual issues in supervision scale. The American
Journal of
Family Therapy, 34, 355-372.
Miller, W. R. & Thoresen, C. E. (2003). Spirituality, religion,
and health: An emerg-
ing research field. American Psychologist, 58, 24-35.
271
Munson, C. E. (2002). Handbook of Clinical Social Work
Supervision, 3rd Ed. New
York, NY: The Haworth Social Work Practice Press.
National Association of Social Workers (2008). NASW Code of
Ethics. Washington,
D.C.: Author.
Parker, S. (2009). Faith development theory as a context for
supervision of spiritual
and religious issues. Counselor Education and Supervision, 49,
39-53.
Peteet, J.R. (1981). Issues in the treatment of religious patients.
American Journal
of Psychotherapy, 35(4), 559-564.

Probst, L. R., Ostrom, R., Watkins, P., Dean, T. & Mashburn,
D. (1992). Compara-
tive efficacy of religious and nonreligious cognitive-behavioral
therapy for the
treatment of clinical depression in religious individuals. Journal
of Consulting
and Clinical Psychology, 60(1), 94-103.
Raines, J. C. (2003). Social work countertransference issues
with spiritually similar
clients. Social Work & Christianity, 30 (3), 256-277.
Richards, P.S. & Bergin, A. E. (1997). A spiritual strategy for
counseling and psycho-
therapy. Washington, D.C.: American Psychological
Association.
Richards, P.S. & Bergin, A. E. (2005). A spiritual strategy for
counseling and psycho-
therapy, 2ndEd. Washington, DC: American Psychological
Association.
Ripley, J. S, Jackson, L. D., Tatum, R. L., & Davis, E. B.
(2007). A developmental
model of supervisee religious and spiritual development.
Journal of Psychology
and Christianity, 26(4), 298-306.
Rose, E. M., Westefeld, J. S. & Ansley, T. N. (2001). Spiritual
issues in counseling:
Clients’ beliefs and preferences. Journal of Counseling
Psychology, 48(1), 61-71.
Rosen-Galvin, C.M. (2005). Values, spirituality and religious
topics discussed in

counseling supervision. Dissertation Abstracts International
Section A: Humani-
ties and Social Sciences, 65 (9-A), 3323.
Shulman, L. (2005). The clinical supervisor-practitioner
working alliance: A parallel
process. The Clinical Supervisor, 24(1/2), 23-47.
Seeman, T. E., Dubin, L. F., & Seeman, M. (2003).
Religiosity/spirituality and
health: A critical review of the evidence for biological
pathways. American
Psychologist, 58, 53-63.
Sermabaikian, P. (1994). Our clients, ourselves: The spiritual
perspective and social
work practice. Social Work, 39(2), 178-183.
Sheridan, M.J. & Bullis, R.K. (1991). Practitioners’ views on
religion and spirituality.
Spirituality and Social Work Journal, 2(2), 2-10.
Sheridan, M. J., Wilmer, C. M., & Atcheson, L. (1994).
Inclusion of content on
religion and spirituality in the social work curriculum: A study
of faculty
views. Journal of Social Work Education, 30(3), 363-376.
Sherwood, D.A. (1999). Integrating Christian faith and social
work: Reflections of
a social work educator. Social Work & Christianity, 26(1), 1-8.
Smith, T. & Richards, P.S. (2005). The integration of spiritual
and religious issues
in racial-cultural psychology and counseling. In R. T. Carter
(Ed.), Handbook

of racial-cultural psychology and counseling: Vol. 4. Theory
and research, pp.
132-159. New York, NY: Wiley.
Spencer, S. (1961). What place has religion in social work
education? Social Work,
3, 161-170.
Spero, M. H. (1981). Countertransference in religious therapists
of religious patients.
American Journal of Psychotherapy, 35(4), 565-575.
THE ETHICAL USE OF SUPERVISION
SOCIAL WORK & CHRISTIANITY272
Statistical Abstract of the United States (2011). Self-Described
Religious identifica-
tion of adult population 1990 to 2008. U. S. Census Bureau, p.
61.
Van Hook, M., Hugen, B., & Aguilar, M. (2001). Spirituality
within religious tradi-
tions in social work practice. Thousand Oaks, CA: Brooks/Cole.
Walsh, F. & Pryce, J. (2003). The spiritual dimension of family
life in F. Walsh, (Ed.),
Normal family processes: Growing diversity and complexity,
3rd ed., 337-372. New
York, NY: The Guilford Press.
Waring, N. (2000). Can prayer heal? Hippocrates, 14, 22-24.
West, W. (2011). Exploring therapy, spirituality and healing.
New York, NY: Palgrave

Macmillan.
Williams, A. B. (1997). On parallel process in social work
supervision. Clinical
Social Work Journal, 25(4), 425-435.
Young, J. S. & Cashwell, C. S. (2010). Religion, spirituality
and clinical supervision
in J. R. Culbreth & L. L. Brown (Eds.), State of the art in
clinical supervision,
pp. 151-171. New York, NY: Rutledge.
Zastrow, C. (1999). The practice of social work, 6th ed. United
States: Brooks Cole
Publishing Company.
Zastrow, C. (2003). The practice of social work: Application of
generalist and advanced
content, 7th ed. United States: Brooks Cole Publishing
Company.
Zastrow, C. (2007). The practice of social work: A
comprehensive worktext, 8th ed.
United States: Brooks/Cole Publishing Company.
Jerry Jo M. Gilham, Ph.D., LISW-S, is Professor/Field
Coordinator, Franciscan
University of Steubenville, 1235 University Blvd., Steubenville,
OH 43952.
Phone: (740) 284-5366. Email: [email protected]
Key Words: spiritual integration, supervision and spiritual
integration,
supervision, ethics and spiritual integration

Reproduced with permission of the copyright owner. Further
reproduction prohibited without permission.
O R I G I N A L P A P E R
Using Methodological Search Filters to Facilitate Evidence-
Based
Social Work Practice
Aron Shlonsky • Tobi Michelle Baker •
Esme Fuller-Thomson
Published online: 11 January 2011
� Springer Science+Business Media, LLC 2011
Abstract The process of Evidence-Based Practice (EBP)
requires clinical social workers to conduct systematic
searches of academic databases in order to ascertain current
best evidence and integrate this with client preferences/
values and clinical state/circumstances. Yet social workers
are often pressed for time, and searches for evidence dis-
regarded as too time-consuming to conduct. There is hope.
Searches of the literature can be more easily and quickly

facilitated through the use of methodological search filters.
This study introduces a new methodological search filter
created especially for social care and evaluates the extent
to which this and four other filters accurately and effi-
ciently identify known social care effectiveness studies in
two major scholarly databases (Psycinfo and Medline).
Sensitivity, specificity, and a new metric for establishing
efficiency (the AVALANCHE INDEX) are reported.
Keywords Evidence-based practice � Methodological
search filters � Sensitivity � Specificity
Introduction
The use of evidence in practice is an issue of social justice.
That is, when we as social workers offer up interventions
for our clients, particularly those clients who are
involuntary, it is our ethical duty to bring current best
evidence into the decision-making context. But what evi-
dence? How do we find what is best? Evidence-Based
Practice (EBP) is a process involving several distinct steps.

As defined by Gibbs (2003), EBP is a process of ‘‘lifelong
learning that involves continually posing specific questions
of direct practical importance to clients, searching objec-
tively and efficiently for the current best evidence related
to the question’’ (p. 6), evaluating the identified literature in
terms of its methodological rigor and applicability of
findings to the client, and ‘‘taking appropriate action guided
by the evidence’’ (p. 6). In this context, posing a question
involves phrasing a question in a way that the answer can
be located using various research databases (i.e., for chil-
dren diagnosed with ADHD whose parents do not wish to
use medication, which form of psychosocial counseling
works best to improve academic functioning?).
Two of the main criticisms of EBP involve the lack of
empirical literature and the limited time practitioners have
to search for evidence (Gibbs and Gambrill 2002; Gray
et al. 2009; Rosen and Proctor 2003). This paper is focused
on evaluating strategies to improve practitioners’ skills in

the quick and efficient location of research articles evalu-
ating the effectiveness of a given intervention, in this case,
random controlled trials or RCT’s. Although there are
many valid forms of evidence, the search for studies of
effectiveness (i.e., one type of intervention compared to
another type, usual treatment, or nothing) is an appropriate
starting point. As any practitioner or student who has tried
to search for evidence is aware, a poorly specified question
and search can lead to thousands of irrelevant ‘hits’ or
results, and wading through these can be cumbersome,
inefficient, and discouraging. For instance, simply stating
the term ‘depression’ in a search engine such as Google
Scholar will return tens of millions of hits, most of which
A. Shlonsky (&) � T. M. Baker � E. Fuller-Thomson
Factor-Inwentash Faculty of Social Work, University of
Toronto,
Toronto, ON, Canada
e-mail: [email protected]
E. Fuller-Thomson
The Department of Family and Community Medicine

and the Faculty of Nursing, University of Toronto,
Toronto, ON, Canada
123
Clin Soc Work J (2011) 39:390–399
DOI 10.1007/s10615-010-0312-3
are not empirical studies. Even in PsycINFO and Medline,
such a search would result in hundreds of thousands of hits,
a virtual avalanche of irrelevant peer-reviewed articles.
One of the main tools used to combat this inefficiency is
the use of methodological search filters. These are com-
binations of search terms that retrieve certain types of
studies (e.g., experimental, quasi-experimental, diagnostic
and prognostic accuracy, qualitative) from scholarly dat-
abases, and they hold the promise of allowing practitioners
to search more quickly and accurately for relevant studies
that can be used to help guide their clinical decisions. For
example, Leonard Gibbs (2003) proposes using the fol-

lowing terms, in combination with subject specific search
terms, to efficiently find highly controlled studies best
suited for questions of intervention effectiveness: (Ran-
dom* OR Controlled Clinical trial* OR Control group* OR
Evaluation stud* OR Study design OR Statistical* Signif-
ican* OR Double-blind OR Double blind OR Placebo).
The use of search filters, however, is not a panacea. Search
filters are similar to diagnostic and prognostic tests in the
sense that they can be of variable quality and their ability to
correctly identify relevant studies can be measured with
similar techniques.
Specifically, filters can be measured in terms of their
sensitivity (degree to which filters can accurately identify
relevant studies—range is from 0 to 1 with 1 reflecting the
ability to capture all relevant studies) and specificity
(degree to which filters can accurately exclude non-rele-
vant studies—range if from 0 to 1 with 1 reflecting the
ability to exclude all irrelevant studies). Haynes et al.

(1994) tested medline-specific methodological search fil-
ters intended to retrieve random controlled trials (RCT’s)
related to adult general medicine. They found that the fil-
ters could be used to produce both specific and sensitive
results, with sensitivity ranging from 0.72 to 0.99
(depending on search year and combination of terms used)
and specificity ranging from 0.70 to 0.79 (in general, sen-
sitivity and specificity levels are best if they are over 0.9
for prognostic and diagnostic tests. However, such general
rules may not apply in this situation since these constructs
are being used simply to compare approaches). Leeflang
et al. (2005) examined eight articles describing 28 sets of
validated methodological search filters for studies of
diagnostic accuracy through testing the capacity of the
filters to retrieve articles identified in systematic reviews,
finding that the filters ranged widely in their ability to
correctly identify relevant articles.
Yet there are indications that methodological search

filters specific to medicine are not accurate across disci-
plines. Murphy (2002) examined effectiveness search fil-
ters for the veterinary sciences and found wide variability
in accuracy when broken down by journal and purpose of
study. In their analysis, sensitivity ranged from 0.05–0.88
and specificity from 0.10–1.00, indicating that, overall, the
search filters they used were not effective in identifying
desired articles. Similarly, Dickersin et al. (1994) reviewed
articles which identified RCT filters in ophthalmology and,
while many studies were correctly identified, the authors
felt that the achieved sensitivity levels of 51–77%, were
not satisfactory.
The accuracy and efficiency of methodological search
filters for social care has not been established. In social
work and the non-medical helping professions, Gibbs
(2003) appears to be the only author to have proposed a
unique set of methodological filters and a process for car-
rying out searches in the context of EBP. In the spirit of

inquiry that is very much in line with the EBP approach
and the legacy of Leonard Gibbs, we decided to test Gibbs’
methodological search filters in the hopes of building and
improving upon his seminal work. This study uses 12
Campbell Collaboration systematic reviews of effective-
ness to: (1) ascertain which scholarly database is most
likely to yield the largest number of relevant articles for
effectiveness questions in social care; (2) develop a new set
of social care methodological search filters; and (3) com-
pare the sensitivity and specificity of these methodological
search filters with two other commonly used filters. In the
process, we also introduce a new metric, the Avalanche
Index, for measuring the efficiency of searches. The crea-
tion of this tool and other filters like it is crucial if clinical
social workers are expected to efficiently locate evidence
within the large and growing body of academic studies.
Methods
Using the final included set of studies identified in 12

Campbell Collaboration systematic reviews (5 pilot, 7 final
study sample) as a reference standard, a new methodo-
logical search filter was developed and validated and
compared with the filters used by Gibbs (2003) and the
Medline filters referenced in the Cochrane Collaboration
Handbook for Systematic Reviews of Interventions (2009).
The Cochrane Search filter was originally designed by
Carol Lefebvre as published in Dickersin et al. (1994). The
terms have been modified, as necessary, over time.
Search Strategy
Each Campbell Collaboration systematic review is based
on an exhaustive and time-consuming search of the liter-
ature, across multiple databases, and is the most compre-
hensive process for identifying relevant and rigorous
experimental and quasi-experimental studies used to
answer a particular social welfare question. Similar to
Cochrane Collaboration systematic reviews of the medical
Clin Soc Work J (2011) 39:390–399 391

123
literature, Campbell Collaboration reviews are built upon
pre-specified and inclusive searches of the peer-reviewed
and gray (unpublished) literature. Since sensitivity (i.e.,
finding all studies for a given question) for these reviews is
of utmost importance, methodological search filters are not
used. Initial ‘Hits’ or results from Campbell Collaboration
content searches are subjected to manual screening
involving at least two raters, with studies being screened
for both content relevance and methodological rigor.
While this process is certainly meticulous and inclusive,
and the results of systematic reviews are a rich source of
evidence for practitioners, the process is time-consuming
and painstaking. It is unrealistic to expect practitioners to
ascertain answers to pressing clinical questions in this
manner. Nonetheless, due to the quality of their search
processes, Campbell Collaboration reviews can be seen as

the benchmark, or reference standard, for identifying the
highest quality effectiveness studies for a given effective-
ness question.
Final included studies from existing Campbell Collab-
oration (C2) systematic reviews in social welfare were used
as a subject-specific reference standard upon which our
stable of methodological search filters was tested, a strat-
egy that has been employed using Cochrane Collaboration
systematic reviews (see, for example, Leeflang et al. 2005).
For this study, Campbell Collaboration search strategies
from existing C2 reviews were combined with five differ-
ent sets of methodological search filters in an effort to
identify as many of the final included studies in each
review (a measure of sensitivity) while cutting down the
number of irrelevant studies (a measure of specificity). In
order to narrow down the number of reviews used to test
the methodological filters, only published social welfare
reviews available in August 2009 were selected (n = 26).

This number was further reduced (n = 14) by selecting
only reviews that exclusively contained randomized con-
trolled trials (RCTs). Although constricting the C2 sample
in this way is somewhat limiting and does not reflect
standard practice in C2 reviews (i.e., C2 reviews often
include different types of study designs), methodological
search filters tend to be method-specific. Although there is
some debate about the hierarchy of evidence (Upshur and
Tracy 2004; Rubin 2008), locating well-conducted RCTs
where they exist is imperative for any reasonable search of
effectiveness studies.
Finally, two of the reviews used the same search strat-
egies and therefore one of the systematic reviews was
removed. Thus, there were 12 systematic reviews which
met our inclusion criteria. Five were used as pilot studies
(Barlow et al. 2003; Kristjansson et al. 2007; MacDonald
and Turner 2007; Smedslund et al. 2007; Zwi et al. 2007)
and the remaining seven were used to test the filters

(Barlow and Parsons 2005; Coren and Barlow 2004;
Ekeland et al. 2005; Littell et al. 2005; MacDonald and
Turner 2007; Mayo-Wilson et al. 2008; Scher et al. 2006).
Databases
The first step in the process was to ascertain which dat-
abases contained the largest number of articles found in C2
reviews. The final list of included studies from two C2
reviews (Barlow et al. 2003; Zwi et al. 2007) was searched
using 8 databases: Social Science Abstracts, PsychINFO,
Cochrane CRCT, ERIC, Medline, EMBASE, CINAHL,
and Social Service Abstracts (Table 1). Please note that
Barlow et al. (2003) is an earlier version of the Barlow
et al. 2005) review, ‘Parent-training programmes for
improving maternal psychosocial health.’
PsycINFO and Cochrane CRCT were the top performing
databases, containing all of the articles for Barlow et al.
(2003) and a substantial proportion of studies contained in
Zwi et al. (2007). Medline also performed fairly well for

the Zwi et al. (2007) study. However, it should be noted
that Cochrane CRCT is updated with articles included in
Cochrane systematic reviews, and the two selected reviews
were co-registered with Cochrane. The remaining data-
bases did not hold many articles for Barlow et al. (2003),
ranging from one study found to nine studies found out of
22 possible, while ERIC, Social Science Abstracts, and
EMBASE held a moderate amount of articles for Zwi et al.
(2007), between 4 and 11; CINAHL held none (Table 1).
Results from this process indicate that PsycINFO is, far and
away, the most likely database to contain relevant studies
Table 1 Methodological filters
Filter set Filters
Avalanche RCT; randomi*; control* trial*; control* clinical;
clinical trial*; random* assign*; random* allocat*;
wait* list*; wait*-list*; control* group*; control*
condition*; quasi-ex*; quasi ex*; control* near
intervention; control* near treat*

Avalanche-
RCT
RCT; randomi*; control* trial*; control* clinical;
clinical trial*; random* assign*; random* allocat*
Gibbs Random*; controlled Clinical trial*; Control group*;
evaluation stud*; study design; statistical*
significanc*; double-blind; placebo
Gibbs-RCT Random*; controlled clinical trial*; control group*
Cochrane Randomized controlled trial; controlled clinical trial;
randomized; placebo; clinical trials as topic;
randomly; tria; not (animals not (humans and
animals)
* Indicates a ‘wild card’ truncation using a ‘wild card’ (a
marker at
the end of a string of words prompting the database to search
for all
terms containing the letters to the left of the *)
392 Clin Soc Work J (2011) 39:390–399
123

for the purpose of retrieving social care articles. In order to
simplify the validation process, we decided to use Psy-
cINFO and MEDLINE as our validation databases.
Methodological Filters
Development of EFFECTS
Efficient Methodological Filter for Experiments Comparing
Treatments in Social Care. The social care methodological
search filter terms for this study were developed in a pilot
study that involved a trial and error process using five C2
social welfare reviews as test cases. These five reviews
(Barlow et al. 2003; Kristjansson et al. 2007; MacDonald and
Turner 2007; Smedslund et al. 2007; Zwi et al. 2007) were
not used as part of the later validation phase. Individual
methodological search filter terms were selected based on
their ability to find the studies listed in each C2 review using
PsycINFO. Specifically, we began by applying the Gibbs
filter on a single review (Barlow et al. 2003) and systemat-

ically tried to improve its performance (i.e., increase its
capacity to correctly identify studies included in the C2
review while minimizing the number of false positives) by
modifying, adding, or deleting individual search terms
within the filter. For instance, the Gibbs term ‘random’ might
find studies using random sampling rather than random
assignment, while the EFFECTS term randomi* would only
find studies using the word ‘randomized’ or ‘randomization’.
Similar to validation studies of predictive instruments, there
is a danger that a tool (in this case, the filter) will be ‘overfit’
to the data at hand (in this case, the review upon which it is
being developed). That is, the filter would predict well for the
systematic review upon which it was constructed, but would
perform less well when applied to other reviews. Similar to
validation studies using a construction and validation sam-
ple, we then applied the EFFECTS filter to four other sys-
tematic reviews (Kristjansson et al. 2007; MacDonald and
Turner 2007; Smedslund et al. 2007; Zwi et al. 2007) and

adjustments were made as we proceeded.
Comparison Filters
EFFECTS terms were compared to two other sets of
methodological filters: Gibbs (2003) effectiveness meth-
odological search filters and those detailed in the Cochrane
Collaboration Handbook for Systematic Reviews of Inter-
ventions (2009). While the Cochrane filter terms were
designed to identify randomized controlled trials (RCTs),
the EFFECTS and Gibbs filter terms were designed to
retrieve both RCTs and quasi-experimental studies. Thus,
to be fair, RCT-only versions of the EFFECTS and Gibbs’
filter terms were created and tested as well. For a list of the
five sets of methodological filters, please see Table 1.
Analysis
Searches were individually run in both PsycINFO and
MEDLINE for each included review. The following search
results were recorded: the number of hits found using each
systematic review’s original search terms and the number

of hits using the review search terms in combination with
each of the methodological filters. Using the number of
reviewed articles found by the searches, the results of the
searches can be grouped into four categories:
1. True positive (TP): article found AND included in
original systematic review
2. False positive (FP): article found but NOT included in
original review
3. False negative (FN): article NOT found but was
included in original systematic review
4. True negative (TN): article NOT found and is NOT
included in original review.
The example in Table 2 is the search history in Psy-
cINFO of our analysis of Barlow and Parsons (2005)
review and the EFFECTS terms.
Evaluations of methodological search filters in the
medical sciences have used various measures including:
sensitivity (Dickersin et al. 1994; Haynes et al. 1994;

Leeflang et al. 2005; Murphy 2002); specificity (Haynes
et al. 1994; Leeflang et al. 2005; Murphy 2002); precision
(Dickersin et al. 1994; Haynes et al. 1994; Taylor et al.
2003, 2007; Watson and Richardson 1999). These are all
strategies to measure the number of desired articles
retrieved as a proportion of all articles retrieved; positive
likelihood ratio (Ingui and Rogers 2001), indicating the
proportion of desired articles retrieved as compared to the
proportion of undesired articles retrieved; and diagnostic
odds ratio (Deville et al. 2000), which is the positive
likelihood ratio divided by the negative likelihood ratio
(the proportion of desired articles that were not found as
compared to the proportion of undesired articles that were
not found). The present study uses sensitivity and speci-
ficity to evaluate the effectiveness of the methodological
filters. Sensitivity is calculated by TP/(TP ? FN) and
specificity is calculated by TN/(FP ? TN). A measure was
also created, the Avalanche Index (AI), that identifies the

number of hits one would need to read through in order to
find one of the desired studies (i.e., studies included in a
review). A similar measure was used by Bachmann et al.
(2002) ‘‘number needed to read’’, which, like the AI, is
calculated as 1/precision. In the present study, the AI is
calculated by taking the total number of hits (TP ? FP)
yielded by the full search (i.e., subject terms combined
with methodological search filter terms) and dividing by
the number of actual studies included in the review that
were found using this search (TP) or ((TP ? FP)/(TP)).
Clin Soc Work J (2011) 39:390–399 393
123
Similar to Number Needed to Treat (NNT) and Number
Needed to Harm (NNH), the AI can also be defined as the
Number Needed to Find (NNF). The AI is probably a better
tool for gauging the success of EBP searches since it
provides an overall measure of efficiency for a search

strategy whereas specificity and sensitivity can only
describe predictive capacity of a strategy as it relates to
previous searches.
Results
Visual comparisons were made for each of the seven
reviews used to validate the search filters in terms of their
sensitivity, specificity, and Avalanche Index (Table 3) for
each of the five sets of methodological search filters
(EFFECTS, EFFECTS-RCT, Gibbs, Gibbs-RCT, and
Cochrane). The number of hits retrieved in Medline using
the content search terms for each review (without the
benefit of a methodological filter) ranged from 26 (Littell
et al. 2005) to 143,084 (Scher et al. 2006) with an average
of 29,525, indicating substantial variability in terms of the
precision of the C2 content searches. There was also sub-
stantial variability in PsycINFO, with total hits ranging
from 559 (Littell et al. 2005) to 66 017 (Scher et al. 2006)
and an average of 17,542 hits. In combination with the

methodological filters, results were also varied. In Medline,
hits ranged from 14 (Littell et al. 2005) to 27,178 (Scher
et al. 2006); in PsycINFO they ranged from 50 (Coren and
Barlow 2004) to 6,239 (Scher et al. 2006).
The first step in the process of measuring each of the
filters’ capacity to retrieve the desired articles was to iso-
late only those studies from reviews that were available in
each of the databases through a title and author search. The
total number of articles that could possibly be found in
PsycINFO ranged from two (Coren and Barlow 2004) to 20
(Ekeland et al. 2005; Littell et al. 2005) with an average of
11. In Medline, possible hits ranged from one (Coren and
Barlow 2004) to 18 (Scher et al. 2006) with an average of
seven. Once the number of possible studies was found,
content terms were combined with each of the filters. Hits
from the combination of filters and review search terms in
PsycINFO ranged from zero (Coren and Barlow 2004) to
16 (Littell et al. 2005) and, in Medline, from one (Coren

and Barlow 2004) to 16 (Scher et al. 2006).
Next, we measured the sensitivity, specificity, and
Avalanche Index for each of the studies, and then calcu-
lated the average for each of these constructs across all
reviews (Table 3). Sensitivity ranged from 0 (Coren and
Barlow 2004) to 0.88 (Scher et al. 2006) in PsycINFO and
from 0.67 (Barlow and Parsons 2005; Ekeland et al. 2005)
to 1.00 (Barlow and Parsons 2005; Coren and Barlow
2004; Littell et al. 2005) in Medline. The EFFECTS filter
had the best sensitivity scores in PsycINFO, achieving the
highest in 5 out of 7 reviews, with an average of 0.67.
However, the EFFECTS-RCT filters performed the poor-
est, achieving the lowest sensitivity in 4 of the 7 reviews,
with an average of 0.48. The Cochrane, Gibbs, and Gibbs-
RCT filters did not demonstrate any clear trends for sen-
sitivity in PsycINFO, with averages of 0.51, 0.60, and 0.60
respectively. In Medline, the filters performed similarly,
achieving identical sensitivity scores in five of the seven

reviews. The EFFECTS filters performed the best in the
remaining two reviews, with an average of 0.87, however
the sensitivity scores between filters did not vary
appreciably.
Table 2 Example search strategy
Step Search terms Hits Review
articles
found
Step—1 Barlow and Parsons (2005)
search
(parent* training or parent* program* or parent* education) and
(toddler or
infant or preschool or pre-school or pre school or baby or
babies)
2,137 4
Step 2—EFFECTS Search (RCT or randomi* or control* trial*
or control* clinical or clinical trial* or
random* assign* or random* allocat* or wait* list* or wait*-
list* or control*
group* or control* condition* or quasi-ex* or quasi ex* or

control* near
intervention or control* near treat*)
68,804 3
Step 3—Barlow and Parsons (2005)
terms combined with EFFECTS terms
(parent* training or parent* program* or parent* education) and
(toddler or
infant or preschool or pre-school or pre school or baby or
babies) and (RCT
or randomi* or control* trial* or control* clinical or clinical
trial* or
random* assign* or random* allocat* or wait* list* or wait*-
list* or control*
group* or control* condition* or quasi-ex* or quasi ex* or
control* near
intervention or control* near treat*)
341 3
* Indicates a ‘wild card’ truncation using a ‘wild card’ (a
marker at the end of a string of words prompting the database to
search for all terms
containing the letters to the left of the *)
394 Clin Soc Work J (2011) 39:390–399

123
Table 3 Review summary
Filters Database
PsycInfo Medline
Sensitivity Specificity Avalanche index Sensitivity Specificity
Avalanche index
Barlow and Parsons (2005). Group-based parent-training
programmes for improving emotional and behavioural
adjustment in 0–3 year old
children.
EFFECTS 0.75 0.84 114 1.00 0.82 49
EFFECTS-RCT 0.25 0.92 182 1.00 0.84 43
Gibbs 0.50 0.85 160 1.00 0.78 60
Gibbs-RCT 0.50 0.86 150 0.67 0.81 76
Cochrane 0.25 0.90 205 0.67 0.86 57
Coren and Barlow (2004) Individual and group based parenting
for improving psychosocial outcomes for teenage parents and
their children
EFFECTS 0.50 0.94 116 1.00 0.94 116

EFFECTS-RCT 0.00 0.97 N/A 1.00 0.96 79
Gibbs 0.50 0.94 115 1.00 0.89 212
Gibbs-RCT 0.50 0.94 105 1.00 0.93 132
Cochrane 0.00 0.96 N/A 1.00 0.96 80
Ekeland et al. (2005) Exercise to improve self- esteem in
children and young people
EFFECTS 0.35 0.89 496 0.67 0.90 121
EFFECTS-RCT 0.25 0.94 405 0.67 0.92 92
Gibbs 0.30 0.86 720 0.67 0.88 143
Gibbs-RCT 0.30 0.88 630 0.67 0.90 121
Cochrane 0.30 0.90 516 0.67 0.93 83
Littell et al. (2005) Multisystemic therapy for social, emotional,
and behavioral problems in youth aged 10–17
EFFECTS 0.75 0.72 11 1.00 0.79 1
EFFECTS-RCT 0.70 0.81 9 1.00 0.64 1
Gibbs 0.65 0.76 11 1.00 0.71 1
Gibbs-RCT 0.65 0.80 10 1.00 0.71 1
Cochrane 0.80 0.78 9 1.00 0.86 1
MacDonald and Turner (2007) Treatment foster care for
improving outcomes in children and young people

EFFECTS 0.82 0.94 32 0.75 0.96 14
EFFECTS-RCT 0.82 0.96 18 0.75 0.97 12
Gibbs 0.73 0.93 39 0.75 0.93 24
Gibbs-RCT 0.73 0.94 36 0.75 0.95 16
Cochrane 0.82 0.95 27 0.75 0.97 11
Mayo-Wilson et al. (2008) Personal assistance for adults (19–
64) with physical impairments
EFFECTS 0.67 0.92 671 0.80 0.88 1,529
EFFECTS-RCT 0.67 0.94 462 0.80 0.90 1,318
Gibbs 0.67 0.90 772 0.80 0.85 1,980
Gibbs-RCT 0.67 0.91 690 0.80 0.89 1,419
Cochrane 0.67 0.93 535 0.80 0.92 1,102
Scher et al. (2006) Interventions intended to reduce pregnancy-
related outcomes among adolescents
EFFECTS 0.82 0.91 439 0.89 0.89 1,016
EFFECTS-RCT 0.71 0.94 323 0.72 0.91 1,003
Gibbs 0.88 0.89 505 0.89 0.81 1,699
Gibbs-RCT 0.82 0.91 446 0.89 0.90 924
Cochrane 0.71 0.92 425 0.72 0.93 822

Clin Soc Work J (2011) 39:390–399 395
123
Specificity ranged from 0.72 (Littell et al. 2005) to 0.97
(Coren and Barlow 2004) in PsycINFO and from 0.64
(Littell et al. 2005) to 0.97 (MacDonald and Turner 2007)
in Medline. The EFFECTS-RCT terms performed best in
PsycINFO, achieving the highest specificity scores in all of
the reviews, with an average of 0.93. In Medline, the
Cochrane filters achieved the highest specificity scores in
all of the reviews, with an average of 0.92. However,
similar to specificity in PsycINFO, specificity scores in
Medline did not vary greatly and, while the EFFECTS-
RCT and Cochrane filters achieved the highest specificity,
they did not substantially outperform the other filters.
In PsycINFO the Avalanche Index (an indicator of how
many studies a practitioner would have to sift through in
order to find a highly relevant article) ranged from nine

(Littell et al. 2005) to 772 (Mayo-Wilson et al. 2008); in
Medline it ranged from one (Littell et al. 2005) to 1980
(Mayo-Wilson et al. 2008). The EFFECTS-RCT filters
achieved the best avalanche scores in PsycINFO ranking
first in 4 of the 8 studies and an average AI of 204. The
EFFECTS filters ranked second, with top indexes in two
reviews and an average AI of 239. The Gibbs filters had the
highest (most inefficient) overall Avalanche Index scores,
ranking last in 5 of the 7 reviews with an average of 233. In
Medline the Cochrane filters had the best AI in 5 of the 7
reviews, with an average of 308. Ranking second, the
EFFECTS-RCT filters had the best Avalanche Indexes in
two of the reviews, with an average of 364.
Discussion and Applications to Clinical Social Work
Practice
Clients deserve no less than our level best to provide them
with services that have a high likelihood of being suc-
cessful. But in order for EBP to become a practice reality,

caseworkers need to be able to quickly locate and evaluate
the research evidence. The results of this study indicate
that, when conducting searches for effectiveness studies in
social care, using the proposed EFFECTS-RCT and full
EFFECT filter terms would be a beneficial addition to a
practitioner’s search methodology.
1
While all of the
methodological search filter sets were able to substantially
reduce the number of hits produced by the searches found
in the reviews, the EFFECTS-RCT and full EFFECTS filter
terms were often superior at retrieving the desired articles
while reducing the total amount of hits in PsycINFO.
When comparing the EFFECTS-RCT to the full
EFFECTS filter terms, and the Gibbs-RCT to the full Gibbs
filter terms, we found that methodological search filters
specifying only RCTs yielded a much smaller number of
irrelevant hits. As such, we recommend that, when

searching for articles, the full EFFECTS filter terms should
be used if one is searching for quasi-experimental studies
or when searching for articles on topics that are not highly
researched. Similar to Gibbs’ (2003) suggested methods for
searching, the full EFFECTS terms can also be used first
and, if a large number of hits are returned, the EFFECTS-
RCT filter can be used to better limit the search.
In interpreting the results, an important caveat regarding
the Avalanche Index should be made. The Avalanche Index
is calculated by taking the number of hits retrieved using
the substantive area search terms combined with the
methodological search filter terms, and dividing this by the
number of correct articles found using these combined
terms. The number of correct or total articles found is
dependent upon the number of correct articles carried in the
database searched. All else being equal, the larger the
number of correct articles found, the smaller the Avalanche
Index becomes. Therefore, unless the total number of

Table 3 continued
Filters Database
PsycInfo Medline
Sensitivity Specificity Avalanche index Sensitivity Specificity
Avalanche index
Average for all reviews
EFFECTS 0.67 0.88 268.43 0.87 0.88 406.57
EFFECTS-RCT 0.49 0.93 233.17 0.85 0.88 364.00
Gibbs 0.60 0.88 331.71 0.87 0.84 588.43
Gibbs-RCT 0.60 0.89 295.29 0.83 0.87 384.14
Cochrane 0.51 0.91 286.17 0.80 0.92 308.00
Average 0.57 0.90 282.95 0.84 0.88 410.23
Sensitivity TP/(TP ? FN), Specificity TN/(FP ? TN), Avalanche
index ((TP ? FP)/(TP))
1
It should be noted that, while we use the term ‘effectiveness’
here,
most of these studies would be more accurately described as
efficacy
studies—the difference being the efficacy studies test whether
an

intervention works in tightly controlled settings while
effectiveness
studies would test whether interventions work in more typical
practice
settings.
396 Clin Soc Work J (2011) 39:390–399
123
articles retrieved and the total possible number of correct
articles within a database happen to be the same across
studies (which is unlikely), direct Avalanche Index com-
parisons can only be made within, not between, individual
reviews. That is, content searches also have greater and
lesser degrees of precision, and this will probably influence
the AI to a much greater extent than the selection of
methodological filter. Some narrowly specified searches
will yield a small number of total hits while less carefully
specified searches will yield a large number of total hits.
There is little that even a well-constructed methodological

search filter can do to overcome the avalanche of irrelevant
hits triggered by a poorly specified search.
The process of developing and testing methodological
filters prompted a number of important observations. First,
as indicated by the AI, the total number of hits yielded by
substantive area search terms strongly influences the total
number hits found when using methodological search fil-
ters. For example, Barlow et al. (2005) search yielded
5,126 hits and, when combined with the full EFFECTS
filter, yielded 612 hits (a reduction of 88%). On the other
hand, Littell et al. (2005) original content search terms
yielded 559 hits and, when combined with the EFFECTS
terms, yielded 169 hits (a reduction of 70%). While the
Barlow, Coren, and Stewart-Brown search reflects a larger
reduction in hits, the number of remaining hits is still very
large due to the review’s use of fairly broad content search
terms. This finding highlights the importance of using
efficient substantive area search terms in combination with

methodological search filter terms in order to reduce the
number of hits that one must examine. An important dis-
tinction must be made, though, between systematic review
searches and practical evidence searches. Specifically,
systematic reviews require broadly specified content sear-
ches in order to ensure that all studies in a particular area
are found, and such an approach requires a great deal of
time, effort, and person-hours. Alternatively, practitioners
in the helping professions must be able to search quickly
and efficiently in order to find methodologically rigorous
studies needed to guide their practice decisions. Both
approaches are necessary components of evidence
informed practice. Users would do well to first carefully
consider the question they are asking and then follow
guidelines for content searches established by such authors
as Gibbs (2003) and McGibbon et al. (1991).
Another important finding was the role played by the
descriptive terms used in keywords, titles, and abstracts

within journal publications. When methodological filters
failed to find an article present in a database, it was often
due to the fact that the article’s keywords, title, or abstract
did not contain any methodology descriptors. This finding
highlights the need for authors and journal editors to apply
strict guidelines with respect to the inclusion of key words
in their subject headings and abstracts that indicate that
type of methodology employed by the study. While this
appears to occur with more frequency in more recent
articles and in articles published in the health sciences,
there is surprisingly little consistency in the published
literature.
Another important consideration in using search filters is
the capacity of any single database to retrieve relevant
articles. We found that, at least for studies contained in
Campbell systematic reviews, PsycInfo consistently con-
tained the greatest number of relevant articles. Our results
are perhaps related to the findings of others who have

investigated the capacity of Social Work Abstracts (SWA)
and found it limited. Both Shek (2008) and Holden et al.
(2009) assessed the database Social Work Abstracts (SWA)
and concluded that SWA does not contain a satisfactory
number of social work articles and, even when articles are
found, they can be difficult to retrieve. Based on our
findings, we contend that it is important to ensure that
methodological filters are being employed within databases
that have the capacity to yield relevant articles. At this
point, SWA may not be up to the task.
Recommendations and Conclusion
Based on the findings of this study, we recommend that
journal editors across disciplinary boundaries find a way to
standardize their use of methodological descriptors and
require that these are added to every new journal article
submitted. Over time, searches will become far more effi-
cient and accurate. Similarly, Taylor et al. (2007) contend
that standardization of search terms is required in order to

improve the efficiency of searches. Taylor et al. (2003)
further recommend that these methodological descriptors
be clearly included in abstracts in order support ease of
searching. We could not agree more. If EBP is to become a
reality, publishers must make the search process as easy
and efficient as possible.
The following is a recommended search strategy inten-
ded to help practitioners locate high quality effectiveness
studies on their topic of interest. Step 1: Refine substantive
area search terms. This first step involves identifying
specific terms related to the area of interest. For example, if
a practitioner were interested in learning about the effects
of cognitive behavioral therapy on anxiety in school-aged
children, the following subject related search terms would
be selected for use in PsycINFO: ‘‘((cognitive behavio*
therapy or cognitive therapy or CBT) and (anxiety)).’’ Step
2: Methodological Filters. The search terms identified in
Step 1 should be combined with the appropriate set of

methodological search filter terms or MOLES. Using the
example from Step 1, the EFFECTS-RCT methodological
Clin Soc Work J (2011) 39:390–399 397
123
search filters might be appropriate since CBT and anxiety
are common research topics and the EFFECTS-RCT filter
terms are more likely to return a higher number of more
rigorous studies. Again, the set of methodological search
filters selected should be based on the amount of literature
available in the topic area of interest. If one is searching for
a more obscure set of literature, the full-set of EFFECTS
terms would be more appropriate. Step 3: Outcomes. If,
after combining the search terms from Step 1 and 2, the
number of hits remains large, search terms that specify the
outcome of interest should be included. Carrying on with
the example, outcome search terms may include: academic
achievement or academic achievement motivation. Step 4:

Demographic variables. If the number of hits is too large,
search terms specifying information regarding the popula-
tion of interest should be included. In this example, search
terms might be: elementary school student *. By following
these guidelines, practitioners should be able to quickly
locate a set of articles on their topic of interest while
avoiding what can be called an avalanche of irrelevant hits.
There are several limitations to this study. First, C2
reviews tend to have rigid inclusion criteria and may only
be representative of high quality studies that ask a partic-
ular question in a particular way. They may exclude studies
for various reasons that have nothing to do with the prac-
titioner question and, as a result, may not be representative
of all relevant RCT’s. The full EFFECTS search terms are
better suited for avoiding this type of problem than either
EFFECTS-RCT or the Medline filter. Second, we only
tested effectiveness filters. There are other types of ques-
tions that require different filters (e.g., risk/prognosis,

assessment/diagnosis, qualitative). Finally, we only tested
these effectiveness filters in two databases: PsycINFO and
Medline. This approach was taken after finding that these
two databases contained a substantial portion of the studies
included in systematic reviews. There may be excellent
studies in other databases and our filters may be less
effective than the original Gibbs (2003) filters at identify-
ing appropriate articles in those databases. Nonetheless,
our finding that PsycINFO is the best database for finding
rigorous studies in social care should serve as a wake-up
call to social services database developers. Finally, it
should be noted that the search strategies identified in the
reviews were, oftentimes, very broad and did not identify
concise subject terms. While practitioners should use more
specific search criteria, for the purpose of the present study
the search terms were taken at face value. As such, the
results produced by our searches probably contain many
more false positives or irrelevant articles than would be the

case with better specified subject searches.
In conclusion, practitioners are being asked to use evi-
dence to guide their decisions when working with con-
sumers of social services. Yet the impossibly large number
of articles in certain subject areas and the enormous effort
required to ascertain the quality of studies is daunting even
for the most seasoned and committed helping profession-
als. The information age is upon us and requires an
evolving set of tools to help us locate and use evidence
efficiently. Efficient and effective methodological search
filters, such as EFFECTS and EFFECTS-RCT can help us
meet this challenge.
References
Bachmann, L. M., Coray, R., Estermann, P., & Ter Riet, G.
(2002).
Identifying diagnostic studies in MEDLINE: Reducing the
number needed to read. Journal of the American Medical
Informatics Association, 9, 653–658.
Barlow, J., Coren, E., & Stewart-Brown, S. (2003). Parent-
training

programmes for improving maternal psychosocial health.
Campbell Systematic Reviews, 1–101 [no longer available].
Barlow, J., & Parsons, J. (2005). Group-based parent-training
programmes for improving emotional and behavioural adjust-
ment in 0–3 year old children. Campbell Collaboration System-
atic Reviews, 2, 1–56.
Barlow, J., Coren, E., & Stewart-Brown, S. (2005). Parent-
training
programmes for improving maternal psychosocial health. Camp-
bell Systematic Reviews, 3, 101.
Coren, E., & Barlow, J. (2004). Individual and group based
parenting
for improving psychosocial outcomes for teenage parents and
their children. Campbell Collaboration Systematic Reviews, 1,
1–38.
Deville, W., Bezemer, P., & Bouter, L. (2000). Publications on
diagnostic test evaluation in family medicine journals: An
optimal search strategy. Journal of Clinical Epidemiology, 53,
65–69.
Dickersin, K., Scherer, R., & Lefebvre, C. (1994). Identifying
relevant studies for systematic reviews. British Medical Journal,
309, 1286–1291.

Ekeland, E., Heian, F., Hagen, K. B., Abbott, J., & Nordheim,
L.
(2005). Exercise to improve self-esteem in children and
young people. Campbell Collaboration Systematic Reviews, 4,
1–49.
Gibbs, L. (2003). Evidence based practice for the helping
profes-
sions: A practical guide with integrated multimedia. Pacific
Grove: Brooks Cole-Thompson.
Gibbs, L., & Gambrill, E. (2002). Evidence-based practice:
Count-
erarguments to objections. Research on Social Work Practice,
12, 452–476.
Gray, M., Plath, D., et al. (2009). Evidence-based social work:
A
critical stance. New York: Routledge.
Haynes, R., Wilczynski, N., McKibbon, A., Walker, C., &
Sinclair, J.
(1994). Developing optimal search strategies for detecting
clinically sound studies in MEDLINE. Journal of the American
Medical Informatics Association, 1, 447–458.
Higgins, J., & Green, S. (Eds.). (2009). Cochrane handbook for
systematic
reviews of interventions version 5.0.2. The Cochrane

Collaboration,
2009. Available from www.cochrane-handbook.org.
Holden, G., Barker, K., Covert-Vail, L., Rosenberg, G., &
Cohem, S.
A. (2009). Social Work Abstracts fails again: A replication and
extension. Research on social work practice, 16(6).
Ingui, B. J., & Rogers, M. A. M. (2001). Searching for clinical
prediction rules in MEDLINE. Journal of the American Medical
Informatics, 8, 391–397.
398 Clin Soc Work J (2011) 39:390–399
123
http://www.cochrane-handbook.org
Kristjansson, B., et al. (2006). School feeding for improving the
physical and psychosocial health of disadvantage students.
Campbell Systematic Reviews, 14, 1–132.
Leeflang, M., Scholten, R., Rutjes, A., Reitsma, J., & Bossuyt,
P.
(2005). Use of methodological search filters to identify diagnos-
tic accuracy studies can lead to the omission of relevant studies.
Journal of Clinical Epidemiology, 59, 234–240.
Littell, J. H., Popa, M., & Forsythe, B. (2005). Multisystemic
therapy

for social, emotional, and behavioral problems in youth aged
10–17. Campbell Collaboration Systematic Reviews, 1, 1–60.
MacDonald, G. M., & Turner, W. (2007). Treatment foster care
for
improving outcomes in children and young people. Campbell
Collaboration Systematic Reviews, 9, 1–95.
Mayo-Wilson, E., Montgomery, P., & Dennis, J. (2008).
Personal
assistance for adults (19–64) with physical impairments.
Campbell
Collaboration Systematic Reviews, 3, 1–36.
McGibbon, A., Eady, A., & Marks, S. (1991). PDQ evidence-
based
principles and practice. Ontario, Canada: BC Decker.
Murphy, S. A. (2002). Applying methodological search filters to
CAB
abstracts to identify research for evidence-based veterinary
medicine. Journal of the Medical Library Association, 90,
406–410.
Rosen, A., & Proctor, E. K. (Eds.). (2003). Developing practice
guidelines for social work intervention: Issues, methods, and a
research agenda. New York: Columbia University Press.
Rubin, A. (2008). Practitioner’s guide to using research for
evidence-
based practice. New Jersey: Wiley.