أ. رودني هوبسون الرئيس لعام 2012 في رابطة التقييم الامريكية

1. Rodney K. Hopson, Professor &
Past-President, American Evaluation Association
Department of Educational Foundations & Leadership
School of Education
Duquesne University (Pittsburgh, PA USA)
hopson@duq.edu
International Exhibition and Forum for Education Riyadh,
Saudi Arabia
18 – 22 February, 2013

2. } Sahar Abugharbieh, Project Manager/ADD Projects
} Susan Kistler, Executive Director, American Evaluation
Association
} Prior collaborative and funded work from lectures,
presentations, research grants, published papers
◦ National Science Foundation, California Endowment, Robert Wood
Johnson Foundation, WK Kellogg Foundation, American Evaluation
Association/Center for Disease Control, American Educational
Research Association, Arizona State University (USA), Claremont
Graduate U (USA), Lincoln U (USA), Manchester Metropolitan U
Education and Social Research Institute (UK)

3. } Recognize the relevance and value-
addedness of culture in evaluation
theory and practice
} Identify strategies for assisting
evaluators and agencies in becoming
culturally responsive

5. the way of life of a group of people, the complex of
shared concepts and patterns of learned behavior that
are handed down from one generation to the next
through the means of language and imitation.
(Barnouw, 1985)
the ever-changing values, traditions, social and political
relationships, and worldview created, shared and
transformed by a group of people bound together by a
combination of factors that include a common history,
geographic location, language, social class, and
religion…
(Nieto, 1999)

6. Student
Servant
Babadi
Multi
Olwayinda
Pan
African
Pillars of
life
How do you think about
yourself?
Think about all of the cultural
roles you play and how
playing these roles define how
you perceive yourself.
Identify influences in your
world view.
How do you think your world
view influences what you do
personally? Professionally?

7. } How you define yourself is the template by
which you organize the world
◦ Culture and one’s cultural background influences
our template(s), decisions, and practices

9. } Social location of evaluator matters: lived
experiences shape assumptions and
frames of reference in evaluation process
} Evaluators play roles in furthering social
change and justice: we are “more than
technicians” but have duty to recognize
power relations, challenge systems of
inequity

10. } Avoiding ethnocentrism means embracing
multiple cultural perspectives
} Culture is central to the evaluation process
} Culturally and ethnically diverse
communities have contributions to make
in redefining field of evaluation
(Hopson, 2003)

11. } Evaluation in rudimentary form in developing
countries, even among international organizations
} Evaluation as auditing versus examination of how
useful and/or appropriate a certain program is for
the local community
} Western ideas of what "should" happen become
the default priority for organizations
} Difficulties understanding the population's ways of
thinking, living, and understanding

12. } Researchers/evaluators struggle to address
cultural conflicts they have encountered
} Culture is either not taken into consideration or it
is unknown how to be successfully taken into
account
} Culture seems to be something to learn so that
one can simply get the evaluation done; it’s not
really considered
} Often include cultural communities as passive
participants

14. Understand and recognize the larger context
for programs or projects
Design evaluation with participants in mind
Allow for self-determination by stakeholders
and program participants
Casillas, Hopson, & Gomez (in preparation)

15. Build trust and facilitate communication
Understand the evaluation audience and help
the audience to understand the evaluation
purpose and process
Make the evaluation accessible
Understand evaluator attributes that may
affect professional practice

16. ◦ Demographic, sociopolitical, contextual
dimensions
◦ Characteristics as dynamic, multifaceted,
learned, created, influenced
◦ From personal to global position
◦ Majority vs. minority
◦ More than about race and ethnicity but
inclusive of other identifications (SenGupta,
Hopson & Thompson-Robinson, 2004; Nieto,

17. } Approach/model used to guide evaluation
} System and culmination of evaluation
strategies
} Theoretical and political positioned (as are
all evaluations)

18. Cultural competence is a stance taken
toward culture, not a discrete status or
simple mastery of particular knowledge
and skills. A culturally competent
evaluator is prepared to engage with
diverse segments of communities to
include cultural and contextual
dimensions important to the evaluation.
Public Statement on Cultural Competence in Evaluation
(American Evaluation Association, 2011)
Accessible at www.eval.org

20. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
20

21. } Be informed by the sociocultural
context of the evaluand, including
◦ History
◦ Formal and informal power relationships
◦ Communication and relational styles
} Assemble an evaluation team whose
collective lived experience fits the
context of the evaluand.
◦ Evaluator awareness of own cultural values,
assumptions, prejudices, stereotypes
◦ Not merely about matching demographics

22. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
22

23. } Develop a stakeholder group representative
of the population served by program.
} Seek to include persons impacted by the
program directly and indirectly.
} Pay attention to issues of power, status and
social class.
} Include multiple voices in meaningful
preparation process and activities.

24. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
24

25. } Document, examine program
implementation
◦ How well is the program connecting with its
intended consumers?
◦ Is the program operating in ways that are
respectful of cultural context?
◦ Are program resources equitably distributed?
} Document, examine progress toward
goals
◦ Who is benefiting from the program, and are
these benefits equitably distributed? Who is
burdened by the program?
} Evaluate overall effectiveness

26. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
26

27. } Reflections on embedding culturally
competent evaluation in Indian Country
◦ Build understanding of values that underlie
programs and projects and create value-added
evaluative contribution
◦ Engage stakeholders in participatory manner
– Build ethic of participation and capacity building that
values community, relationships, respect
◦ Frame purpose by building conceptual picture/
model
◦ Careful of “too sequential and narrative driven”
logic models

28. SANKOFA MODEL
©2010 P. Frazier-Anderson, S. Hood. & R. Hopson.
All Rights Reserved.

29. } Questions of relevance to significant
stakeholders
} Determine what will be accepted as evidence
} Notice whose voices are heard in the choice of
questions and evidence.
} Reflect on how questions limit what can be
learned and how they might be posed
differently.
} Notice how different questions may expand
understanding. Revise and refine questions.
} Can questions be answered with available
resources?

30. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
30

31. } Build design appropriate to both evaluation
questions and cultural context.
} Seek culturally appropriate mixed methods,
combining qualitative and quantitative
approaches.
} Try to collect data at multiple points in time,
extending the time frame of the evaluation as
needed.
} Construct control or comparison groups in ways
that respect cultural context and values.

32. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
32

33. } Identify, develop or adapt instruments for the
local context.
} Establish evidence of reliability and validity.
} Language and content of instruments should
be culturally sensitive.
} Use best translation practices, validating both
semantic and content equivalence.
} Norms must be appropriate to the group(s)
involved in the program.

34. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
34

35. } Evaluation of undergraduate STEM research
program designed for students of color
◦ Deliberate design of evaluation team intimately
connected with program of study and background
of program, including similar lived experiences of
participants
◦ Questions were sensitive to lived experiences of
participants and focused on substance of
participant experiences

36. } Procedures used to collect both
qualitative and quantitative data must be
responsive to cultural context.
} Nonverbal as well as verbal
communications provide keys to
understanding.
} Train data collectors in culture as well as
technical procedures.
} Recognize how cultural identifications of
the evaluation team affect what they can
hear, observe.

37. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
37

38. } Understanding cultural context is necessary
for accurate interpretation.
} A cultural interpreter may be needed to
capture nuances of meaning.
} Stakeholder review panels can more accurately
capture the complexity of cultural context,
supporting accurate interpretation.
} Disaggregate data and cross-tabulate to
examine diversity within groups.
} Remember that data are given voice by those
who interpret them.

39. Cultural
Responsive
7
Collect
the data
8
Analyze
the data
3
Identify purpose
of the evaluation
4
Frame the
right questions
1
Prepare for
the evaluation
5
Design the
evaluation
6
Select and adapt
instrumentation
2
Engage
stakeholders
9
Disseminate and
use the results
39

40. } Maximize community relevance of findings;
invite review by community members prior to
dissemination.
} Communication mechanisms must be
culturally responsive.
} Inform a wide range of stakeholders.
} Make use consistent with the purpose of the
evaluation.
} Consider community benefit and creating
positive change.

41. } Discussion of urban school-to-career
intervention program using culturally
responsive evaluation approach
◦ Findings disaggregated by gender and age to get
breakdown of career attitudes and beliefs for
participants
◦ Findings provided to numerous stakeholders in
particular ways (e.g. student findings presented
in student-friendly manner)

42. } Culture matters and affects every evaluation
} CRE encourages the consideration of a
variety of factors in conducting evaluations
in agencies and communities
◦ Consideration of cultural context and factors can
provide a more responsive and accurate evaluation

43. Rodney K. Hopson, Professor &
Past-President, American Evaluation Association
Department of Educational Foundations & Leadership
School of Education
Duquesne University (Pittsburgh, PA USA)
hopson@duq.edu
International Exhibition and Forum for Education Riyadh,
Saudi Arabia
18 – 22 February, 2013