ICF recent scientific poster “Collaboration Development, Promotion, and Evaluation of a Web-Based Intervention for African American Breast” concludes that for a successful delivery of a web-based intervention for African American young breast cancer survivors, in-person communication with trusted sources, and culturally-tailored digital and social media are critical.

1. Collaborative Development, Promotion, and Evaluation of a Web-Based Intervention for African American Young Breast Cancer Survivors
Ten evaluation questions guided the design and conduct of the evaluation:
1. Which of the YSI’s core elements were implemented?
2. Which of the YSI’s core elements were implemented as intended?
3. Were activities related to the partnership element implemented as intended?
4. How many (and what type of) SNI partners were engaged and involved in efforts to recruit YBCS to
participate in the YSI by accessing the YSI site?
5. Were activities related to the Web presence element implemented as intended?
6. What were the barriers and facilitators to YSI implementation?
7. How many people visited the YSI to obtain psychosocial and reproductive health support information?
8. What proportion of people who accessed the YSI match the demographic profile of the intended
target population?
9. How many African American YBCS visited the YSI to obtain psychosocial and reproductive health
support information?
10. Did African American BCS and YBCS value and use the information available on the YSI site?
Overall Users BCS
300
YSI visitors were very pleased with the YSI’s provision of psychosocial and reproductive health
information. Overall, respondents were satisfied with the YSI. Most in the four groups represented in
Exhibit 6 reported that they were very satisfied. Seventy percent of all respondents and BCS, 74%
of YBCS, and 81% of YBCS under age 45 were either somewhat or very satisfied with the YSI.
The use of a mixed-method evaluation design helped evaluators identify the strengths and best
practices of the YSI program. Findings reveal that there is great value in developing and implementing
culturally appropriate interventions providing African American YBCS with psychosocial and
reproductive health support. Partnerships with organizations that can reach vulnerable audiences and
use of proven Web-based intervention strategies to implement culturally appropriate interventions
may effectively contribute to the elimination of cancer health disparities.
Centers for Disease Control and Prevention. (1999). Frame work for program evaluation in public health.
Morbidity and Mortality Weekly Report, 48(RR11), 1–40.
Background and Purpose
African American women under age 45 experience the highest breast cancer incidence of any ethnic/
racial group and, across all ages, have a disproportionately higher breast cancer morbidity rate.
In March 2010, as part of the Patient Protection and Affordable Care Act, Congress passed the
Education and Awareness Requires Learning Young (EARLY) Act. This directed the Centers for
Disease Control and Prevention (CDC) to develop and implement national campaigns to educate
young women (particularly African American and Ashkenazi Jewish women, who are at increased
risk) and health care providers about breast cancer risk and early diagnosis. In September 2010,
CDC launched the Breast Cancer in Young Women (BCYW) project, a 3-year study to identify,
strengthen, promote, and evaluate real-world, evidence-based interventions that support YBCS.
Through the BCYW project, CDC supported Sisters Network® Inc. (SNI) for the development and
evaluation of the Young Sisters Initiative: A Guide to a Better You! program (the YSI). The YSI is a
Web-based program that provides psychosocial and reproductive health support to African American
breast cancer survivors (BCS).
Collaborative Development of the YSI
The YSI was developed and evaluated through a collaborative partnership between CDC, SNI,
and subject matter experts in the areas of breast cancer and health communications. The YSI
was modeled after the Sisters Peer Counseling in Reproductive Issues After Treatment (SPIRIT)
program, an evidence-based behavioral intervention designed for African American BCS of all ages.
CDC, SNI, and subject matter experts conducted formative evaluation to inform the adaptation of
SPIRIT for African American YBCS. The resulting YSI consists of culturally appropriate materials
and messages intended to address the specific needs of African American YBCS. The YSI is broadly
accessible and easy to use, and its images, design, and tone are tailored to appeal to this specific
audience.
Authors
* Interviews were conducted before the YSI launch and postimplementation.
Temeika Fairley, PhD • Dara Schlueter, MPH • Ashani Johnson-Turbes, PhD • Angela R. Moore, MPH • Natasha Buchanan, PhD • Mary Ann K. Hall, MPH
• Marnie House, PhD • Linda Baffo, MPH • Kelly Hodges, MS • Leslie Schover, PhD
During the evaluation, YSI visitors commonly reported SNI as their primary source for learning about
the YSI. Among BCS, the next most commonly reported referral source is the Internet (likely via the use
of search engines). Over the course of the evaluation period, there were 1,442 visitors to the YSI; 524
of the visitors returned one or more times to the site, resulting in a total of 1,966 unique page views. Of
the 1,442 visitors, 476 (33%) completed the demographic screener to access the YSI (Exhibit 2).
Exhibit 2. YSI Visitors and Multiple Site Visits
Count of Visits Visitors Page Views
1 1,442 6,166
2 242 1,068
3 72 255
4 34 122
5 23 76
6 16 48
7 11 24
8 11 20
9–14 33 121
15–25 24 183
26–50 45 214
51–100 13 30
Visitors commonly accessed the YSI by typing the URL directly into their browsers (Exhibit 3). The
second most common way to access the YSI was via Facebook, followed by referral from SNI’s program
Web site, and then by e-mail links.
Exhibit 3. Web Traffic Sources
Count of Visits Page Views
Direct entry 1,034
Facebook 338
SNI Web site 226
E-mail links 200
Search engine 72
SNI social network 35
Twitter 24
News Web sites 21
Nonprofit organizations 11
Visitors who accessed the site did so with a combination of devices. Exhibit 4 depicts the proportions
of visitors (1,319) who used a desktop, along with the 647 others who used mobile devices (404) and
tablets (243).
Acknowledgements
Exhibit 4. Devices Used to Access the YSI Site
The project was funded by CDC contract number 200-2008-27957. Findings and conclusions are those of the authors and do not necessarily represent
the official position of CDC.
Key Results From Qualitative Data
Key Results From Quantitative Data
Key Results From Quantitative Data (cont.)
Evaluation Questions
Conclusion
References
Evaluation Design
Following the steps outlined by the CDC Framework for Evaluation in Public Health (CDC, 1999)
and principles of culturally responsive research design, a mixed-method process evaluation
design was developed to address the 10 evaluation questions. The evaluation design included
mixed-method data collection before, during, and after intervention implementation to assess
reach, Web site usage, and user satisfaction.
The qualitative and quantitative data sources used for this evaluation are in Exhibit 1
• As reported across all interviews, both of the YSI core elements—partnerships and Web presence—
were implemented during YSI implementation from March to August 2013. The YSI core elements
were implemented as intended.
Collaborative partnerships were essential to implementing and sustaining the YSI, as partners were
involved to support YSI development (e.g., logic model review, review and finalization of the YSI site
structure and Guide) and implementation (e.g., promotion, Web-based and “on the ground” recruitment).
• Key strategies for marketing the YSI included conducting personal outreach to the target audience
to promote the YSI’s features and encourage use of the intervention; leveraging existing resources
to market the YSI, including local and national partnerships and digital networks; disseminating
consistent, culturally appropriate messaging in YSI Web-based materials; and using social media
and other Web-based strategies (e.g., e-mail) for YSI promotion.
• Facilitators to implementation included CDC support for the YSI; existing partnerships; consistent
YSI messaging, look, and feel; use of culturally appropriate materials, messages, and marketing
strategies; and the YSI’s involvement in existing SNI activities (e.g., the SNI’s annual breast
cancer walk).
• Barriers to implementation included the delay in launching the YSI; limited time for implementation;
limited resources for implementation; inability to offer incentives to YSI visitors; challenges reaching
African American YBCS; and the large amount of information in the YSI Guide.
The proportion of individuals who registered for and accessed the YSI Web site closely matched
that of the intended target audience, particularly among BCS, African American YBCS, and African
American YBCS younger than age 45 (Exhibit 5). When non-BCS participants were removed, the
proportion of visitors who identified as Black or African American rose from 58.6% to 93.0%.
Most respondents—81% of overall visitors, BCS, and YBCS, and 88% of YBCS under age 45—were
either somewhat or very likely to recommend the YSI to others.
Desktop, 1,319 Other, 647 Mobile devices, 404 Tablets, 243
Level of Satisfaction With
the YSI Program
Overall Visitors
(n = 180)
BCS
(n = 161)
African American YBCS
(n = 77)
African American YBCS < 45
(n = 37)*
Very unsatisfied 9 (6.67%) 8 (6.67%) 5 (7.25%) 2 (6.25%)
Somewhat unsatisfied 4 (2.96%) 3 (2.5%) 2 (2.9%) 0 (0%)
Neutral 27 (20%) 25 (20.83%) 10 (14.49%) 4 (12.5%)
Somewhat satisfied 34 (25.19%) 31 (25.83%) 19 (27.54%) 8 (25%)
Very satisfied 61 (45.19%) 53 (44.17%) 33 (47.83%) 18 (56.25%)
Missing 45 41 8 5
Data Source Data Collection
Primary or
Secondary Data
Source
Evaluation Question
Addressed
Qualitative Data Sources
SNI program documents
ICF reviewed SNI program documents to gather information
about the organization and provide context for program imple-mentation.
Secondary 1, 2, 3, 4, 5, 6
In-depth interviews of
SNI staff members
ICF conducted interviews with SNI staff to understand their
roles and experience in YSI development and implementation;
engagement of partners; barriers and facilitators to implemen-tation;
and best practices for implementation.
Primary 1, 2, 3, 4, 5, 6, 10
In-depth interviews of
SNI partners
ICF conducted interviews with SNI partners to learn about their
roles and experiences with the YSI, especially their roles in YSI
recruitment and perceptions of the program’s value.
Primary 1, 2, 3, 4, 5, 6, 10
Quantitative Data Sources
Google Analytics
Google Analytics was used to investigate how visitors used the
YSI site. Google Analytics data (i.e., detailed statistics about the
YSI Web site’s traffic and traffic sources) were collected from
launch and throughout the evaluation period.
Primary 7, 9
Demographic screener
The screener was administered through the YSI Web site to
collect demographic data about YSI users. The 12-question
screener gathered dataon how users heard about the YSI,
along with information regarding age, race, survivorship status,
family structure, treatment history, and willingness to participate
in the post use survey.
Primary
7, 8, 9
Postuse survey
The post use survey gathered data from YSI users, self-identi-fied
as BCS, to understand their perceptions of the YSI, particu-larly
their attitudes toward the YSI Guide, and whether the pro-gram
met their needs for reproductive and psychosocial health
information
Primary 8, 9, 10
Exhibit 6. Satisfaction With YSI Across Visitor Types
Exhibit 1. Mixed-Method Evaluation Data Sources*
Black or
African
American
White American
Indian or
Alaska Native
Asian Native Hawaiian
or Other
Pacific Islander
250
150
100
50
0
279 279
167
20 16
4 3 2 1 0
200
Exhibit 5. Racial/Ethnic Background of YSI Visitors