1) The document discusses evaluation methods for programs that target very young adolescents (VYAs). It explores participatory and qualitative methods that allow VYAs to share their perspectives rather than relying solely on adult assessments. 2) Various participatory techniques are presented, such as card games, storytelling exercises, photo elicitation, and pile sorting, that can be used to evaluate changes in attitudes, knowledge, and behaviors related to topics like gender norms, fertility awareness, and HIV prevention. 3) Examples are given of evaluation designs for VYA programs that have used mixed methods with quantitative and qualitative data collection, including from structured interviews, focus groups, and photovoice exercises, to measure outcomes over time.

1. Leave those surveys behind!
Key issues to consider in evaluating
VYA programs
Rebecka Lundgren and Susan Igras
USAID Global Health Mini University, October 2010

8. Recent IRH Youth Initiatives
• Evaluation of the New “My Changing Body”
(CRS, Rwanda ), (APROFAM, Guatemala)
• Evaluation of Save the Children Curriculum
“Choices” (Nepal)
• New! Gender Forming and Transforming
Interventions for Youth in Uganda
(Pathfinder, Save the Children)

9. Technical Consultation for Advancing Program and
Research/Evaluation Practices for Curriculum-based
Programs Reaching Very Young Adolescents……
• June 2010
• 30 participants
• Literature and Program Review document
prepared for meeting
• Report of meeting (forthcoming)

10. Social construction of childhood
“Children live in and negotiate
worlds that they create for
themselves,
worlds others create for them,
and
worlds in concert with others.”
Myrna Bluegood

11. Childhood studies
“Listening to the voices of children themselves
(rather than what adults say about them) reveals
what is important to them, such as the
patterning of gender in children’s social
relationships.” (Alison James, 2007)

12. Methodological Issues
• Research methods advantage
adults in terms of social or
communication skills or
knowledge.
• Need for methods that shift
balance of power
• Move from verbal to visual

13. Visual, participatory methods
• Communicate meanings
• Adaptation of methods used
in play therapy (clay, cultural
artifacts, drawing, role
playing)
• Children show what they want
to communicate, as well as
expressing their thoughts
verbally.

14. • Protection
• Provision
• Participation
• Voice,
representation and
authenticity
Implications for Research with VYA:
ethics and methods

15. Key VYA Program Evaluation Questions
- What effect can we realistically
expect from programs?
- How can these be measured in a
project time frame?
- How can we operationalize concepts
such as gender attitudes and norms?
- How reliable and valid are our
results?

16. Fertility Awareness/Body Literacy:
•Understanding fertility
•Accepting sexuality
•Understanding changes in puberty
•Self-care
•Intergenerational communication skills
•Self-advocacy of these topics with peers,
parents and other adults
Selected Outcomes for VYA Programs
Gender:
•Gender consciousness
•Attitudes toward gender roles & equity
•Gender equitable behavior
Agency:
•Self efficacy
•Self esteem
•Self confidence
•Assertiveness

17. HIV/STI Prevention:
•Understanding
risky/protective behaviors,
symptoms, routes of
transmission
•Communication (e.g.
condom negotiation, seeking
advice from providers
Sexual Behaviors:
•Delay of early marriage
•Age at first intercourse/child
•Contraceptive use
•# adolescent/unintended
pregnancies
•STI prevention behavior: #
partners, abstinence, condom use
Selected Outcomes for VYA Programs
Gender-based Violence:
•Acceptability of GBV
•Experience of GBV

18. Environmental:
•Availability of peer networks
•Social support
•Safe spaces
•Parental supervision
•Asset building opportunities
•Youth friendly services
availability
•Availability of gender-equitable
recreational opportunities, such as
sports & other activities
Selected Outcomes for VYA Programs

19. Our Challenge: Develop participatory
methods appropriate for VYAs to evaluate
changes in cultural/social meanings
Participatory Methods
Youth play active
role in info-
gathering. Useful
to encourage
change based on
results.
Qualitative Methods
Explore
meanings,
processes,
explanations

20. BUT… how to measure change using
these methods?
• Compare with control group
• Triangulate with quantitative
data
• Use structured matrices to
“process” and compare data
• Take photographs of visual
data
• Capture open-ended answers in
quantifiable ways

21. Techniques for collecting
information from VYAs on gender
and fertility awareness
•Card games
•Gender role game/pile sort
•Stories/Q&A
•Advice letters
•Photo elicitation
•PhotoVoice
•Free listing
•Collage/artwork

22. Evaluation Methods
• Stories and advice
(explore gender
attitudes)
• My week (pie chart
activity to find out how
boys and girls spend their
time)
• Pile sorts with photos (to
explore frequency of
gender-equitable
behavior)

23. This is the story of Juan who is
14 years old. Last month when he
woke up his underwear was a
little wet and he realised he had
his first ejaculation. Sometimes
his penis gets hard when he sees
someone he likes. He is not sure
if that is normal, but he is too
embarassed to talk about this
with anyone.
Juan’s Story

24. Do you think that Juan needs
to ejaculate each time he has
an erection?
Is it bad for Juan to touch his
genitals (masturbate) often?
Starting now, will Juan be
fertile every day or only some
days?
What do you tell Juan?

25. These techniques can be used
with groups…

26. Or individually….

27. They can be used to collect
qualitative….

28. ...or quantitative data

29. LET’S TRY A FEW……

30. Agree or
Disagree? OPINION
CARDS
FACT
CARDS
ADVICE
CARDS
CARD GAME

31. FREE LISTING
AND PILE SORT
Boys! Girls!
Gender!

32. Hopes and Dreams
PHOTO
ELICITATION

33. Life in my
community
Photo Voice

34. HOW HAVE
WE USED
THESE
TECHNIQUES?

35. Experimental Child Clubs
(12)
Choices
Control Child Clubs
Data collection
Structured interviews
w/youth (600)
O1 O2
In-depth interviews w/youth O2
Focus groups with parents O2
Photovoice with youth O2
Choices Evaluation Design

36. Evaluation Indicators:
Behaviors
Boys and girls…
• talk about their feelings and dreams with each
other
• promote gender equity in their lives
• take action to improve the lives of their sisters
• don’t tease their peers for behaving in non-
traditional gender norms

37. Evaluation Indicators:
Attitudes
Boys and girls…
• imagine a life in which men and women have
equal opportunities
• accept non-traditional gender roles
• value relationships based on equality, respect
and intimacy
• value the role of men nurturing their family as
well as providing financial support
• expect to make decisions jointly with their
spouses

38. Study Design: My Changing Body
Phase 1:
Formative Research
• 400 boys and girls and
their parents and teachers
(2 sessions)
Phase 2: Evaluation
 Pre/post measurement
of control and
experimental group
(Individual interviews)
 Post-test only, control
and experimental group
(Group activities)

39. RESULTS

40. Knowledge of male fertility among
youth and parents before and after
MCB
Nocturnal
ejaculation
normal
Fertile every
day
p<.01

41. Youth shift towards less stereotyped
gender attitudes after MCB-Guatemala
(n=57)

42. Hopes and Dreams of “Choices” Participants
Girls say…
“I would ask my brothers to
talk to my parents to continue
my education and I would
study higher and become a
successful woman some day.”
“To make my dream come
true, I should ask for help
form my brothers and work
hard.”
Boys say..
“I want to be a banker
and also want my sisters
to have a respected job in
the future.”

43. “People laugh at a man who cooks food in their home. But from the
day we have taken “Choices” classes, our brothers have started
helping us and we help them too. We will teach the same to our
friends in our village as well.”
- Girl after participating in “Choices”

44. “Life for boys and girls is not equal in our community. Most of
the boys go to school while girls have to look after household chores.
Men have more freedom, they don’t have to work at home Girls are
married and sent to her husband’s home. However, we have learnt
from the child club that life for boys and girls is equal. We have to
work together and help each other”.
Boy after participating in “Choices”

45. “Life for boys and girls is not equal in
our community. Boys have freedom,
they can go any where they want ,but
our parents do not allow our sisters to
go outside home. Boys can play
games, while girls have to look after
household chores .Boys should do
boyish work not girlish work. If people
in our community see boys doing
girl’s work they laugh at them, they
are believed to be inferior.“
Boy in Control Group

46. Final thoughts on
methods for evaluating
programs reaching very
young adolescents