Course Acceleration & Holistic Advising

1. Copyright © 2015 by Educational Testing Service. All rights reserved. ETS, the ETS logo, LISTENING. LEARNING. LEADING. and
SUCCESSNAVIGATOR are registered trademarks of Educational Testing Service (ETS). All other trademarks are the property of their respective owners.
27563
1
Driving Student Success:
Course Acceleration and Holistic Advising

2. ATTAINED
35%
ENROLLED
19%
NO
DEGREE,
NOT
ENROLLED
46%
Students Starting at
Two‐Year Institutions
1National Center for Educational Statistics’ Beginning Postsecondary Students Survey (BPS: 04‐09)
2National Center for Education Statistics, Integrated Postsecondary Education Data System (IPEDS), Spring 2004.
Degree Attainment for Beginning College Students:
Starting in 2003-04 Academic Year, followed through 20091
ATTAINED
64%
ENROLLED
12%
NO
DEGREE,
NOT
ENROLLED
24%
Students Starting at
Four‐Year Institutions
Of the 1.4 million students2 who enrolled in 4‐
year schools in the Fall of 2003, an estimated
525,686 STUDENTS
had not received a degree as of 2009.
Of the 1.1 million students2 who enrolled in 2‐
year schools in the Fall of 2003, an estimated
743,909 STUDENTS
had not received a degree as of 2009.

3. More than 1.7 million students began
community college in Fall 2014...
But are they prepared?
• 50% of community college students need
remedial courses.1
• 20% of four‐year college students also start
in remediation. 1
• 30% of these students who are placed into
remedial courses ‐ frustrated by their low
placement ‐ don’t even bother to enroll in
classes.2
• Fewer than 1 in 4 students who enroll in
developmental courses complete a degree.2
1Complete College America (2012)
2Bailey, Jeong, & Woo‐Choo (2008)

4. Developmental Education:
Jumping through Hoops
Practical
Challenges: Students
face a long, difficult
sequence,
sometimes
containing 4‐5
additional semesters
of coursework.
Academic
Challenges:
Students enter with
deficiencies, and are
placed into the same
class settings that
created those
deficiencies.
Noncognitive
Challenges:
Time management
and study skills help
students manage
college‐level
expectations and
work load.
4

5. • Curricular alignment with K-12
Russell (2008)
• Course redesign (e.g., co-requisite models)
Adams, Gearhart, Miller, & Roberts (2009)
• Curricular redesign (e.g., emporium models)
Twigg (2009)
• Course acceleration
Complete College America (2012); Edgecombe (2011); Scott-Clayton
(2012)
• Improved/holistic assessment and placement
Boylan (2009); Burdman (2012); Conley (2007); Levine-Brown, Bonham
(2008); Saxon, & Boylan (2008)
5
How can we address
developmental education?

6. What are noncognitive skills?
Institutional
Commitmen
t
Conscientiousne
ss
SocialSupport
Teamwork
Self-efficacy
6
Motivation
Metacognition
Study Skills
Goal
Setting
Response to
Stress
Test Taking
Strategies

7. 7
Construct rGPA
Academic-related Skills .129
Academic Goals .155
Academic Self-efficacy .378
Institutional Commitment .108
Social Support .096
SES .155
ACT® or SAT® Scores .368
Robbins, et al. (2004) meta-analytic correlations with retention, GPA
Construct rGPA rretention
Academic-related Skills .129 .301
Academic Goals .155 .212
Academic Self-efficacy .378 .259
Institutional Commitment .108 .206
Social Support .096 .204
SES .155 .212
ACT® or SAT® Scores .368 .121

8. n=713
Success rates for students
with low readiness but high
effort more than three times
as those for low effort, low
readiness
7
Li., K., et al. (2013). Readiness, behavior, and foundational mathematics
course success. Journal of Developmental Education, 13 (1), 14‐22.
Achievement, Behavior,
and Success in Math
Courses Effort Level
(participation, attendance, complete assignments)
High Medium Low
Math
Readiness
High 92% 80% 59%
Medium 86% 67% 29%
Low 74% 50% 19%

9. Holistic Assessment and Developmental Education:
The SuccessNavigator® Assessment
• A 30‐minute online, nonproctored, noncognitive assessment for incoming
college students: Low stakes, diagnostic and developmental
• Can be used with or without academic markers (SAT® scores, HS GPA,
course placement score)
• Assessments are scored and available immediately to students and
advisors. Administrators have access to reports and data within the online
system.
• Uses in a developmental education, course acceleration context:
– Provides a composite score, independent of placement tests, that
recommends students who are likely to succeed when placed into
higher level courses.
– Provides noncognitive scores and feedback, as well as tailored action
plans, tools and tips, and recommendations for campus resources that
help students focus on skills and behaviors that can improve success
9

10. General Skill
10
Subskill Definition Example Items
Academic
Skills
Tools and strategies
for academic success
Organization Strategies for organizing work and time.
I make a schedule for getting my school work done.
I take due dates seriously.
Meeting Class
Expectations
Doing what’s expected to meet the
requirements of your course including
assignments and in‐class behaviors.
I attend almost all of my classes.
I complete the reading that is assigned to me.
Commitment
Active pursuit toward
an academic goal
Commitment to
College Goals
Perceived value and determination to succeed
in and complete college.
One of my life goals is to graduate college.
The benefit of a college education outweighs the
cost.
Institutional
Commitment
Attachment to and positive evaluations of the
school.
This is the right school for me.
I’m proud to say I attend this school.
Self‐
management
Reactions to
academic
and daily stress
Sensitivity to
Stress
Tendency to feel frustrated, discouraged or
upset when under pressure or burdened by
demands.
I get stressed out easily when things don't go my
way.
I am easily frustrated.
Academic Self‐
Efficacy
Belief in one’s ability to perform and achieve in I'm confident that I will succeed in my courses this
an academic setting.
semester.
I can do well in college if I apply myself.
Test Anxiety
General reactions to test‐taking experiences,
including negative thoughts and feelings (e.g.,
worry, dread).
When taking a test, I think about what happens if I
don't do well.
Before a test, my stomach gets upset.
Social
Support
Connecting with
people and students
resources for success
Connectedness A general sense of belonging and engagement.
I feel connected to my peers.
People understand me.
Institutional
Support
Attitudes about and tendency to seek help
from established resources.
If I don't understand something in class,
I ask the instructor for help.
I know how to find out what's expected of me in
classes.
Barriers to
Success
Financial pressures, family responsibilities,
conflicting work schedules and limited
institutional knowledge.
Family pressures make it hard for me to commit to
school.
People close to me support me going to college.

11. Feedback: Determine by
more specific “facet” scores
(see next page).
Action Plans: Suggested
interaction with programs
and services on campus.
Success Indices: Separate
indices for both
classroom and
enrollment success.
Based on background,
cognitive and psycho‐
social information and
supported by statistical
relationships with
success.
Background Information:
Communicate key student
information from both
SuccessNavigator® and SIS
to faculty/advisor.
Domain Scores: Four general
areas of student strengths
and weaknesses. Scores are
presented normatively.
11

12. Academic Success:
Classroom success
broken into ability to
meet expectations and
organizational skills.
Commitment: Measures
Self‐management: Ability
to handle stress, testing
anxiety and academic self‐
efficacy.
Social Support: Tendency
to seek help, attitude
toward barriers to success
and ability to relate to
peers in school.
.
both commitment to
college and commitment
to the specific institution.
12

13. 13
Holistic Assessment and Course
Acceleration

14. Level COMPASS Score IWCC Math Course(s)
1 0 – 39 MAT 075
2 40 – 44 MAT 102, MAT 110
3 45 – 52 MAT 157
4 53 or above MAT 121, MAT 129
Level COMPASS Score
IWCC Math Course Based on
SuccessNavigator® Course Acceleration Indicator
Yellow – Caution Green ‐ Accelerate
1 0‐35 MAT 075
Decision Zone 1/2 36 ‐ 39
MAT 075 or
MAT 102/110
MAT 102/110
2 40*** MAT 102/110
Decision Zone 2/3 41 – 44
MAT 102/110
or MAT 157
MAT 157
3 45 – 48 MAT 157
Decision Zone 3/4 49 – 52
MAT 157 or
MAT 121/129
MAT 121/129
4 53 or above MAT 121/129

15. • Studied 70,000 first-time degree-seekers from a large,
urban CC system
• Used regression discontinuity modeling to predict
performance of dev. ed. students in college level courses
using multiple measures
• Major findings:
1. One quarter of math students and one third of English students
are “severely misplaced”
2. Using multiple measures could lower remediation rates by 8% in
math and 12% in English
3. Roughly 20% of students in math and 30% of students in English
are severely under-placed, meaning they could were predicted to
receive a B or better in the college-level course
Scott‐Clayton, J. (2012). Do high‐stakes placement exams predict college success?
(CCRC Working Paper No. 4). New York: Community College Research Center.
15
Does course acceleration actually work?
Evidence from the Community College Research Center

16. ETS Course Acceleration Case Study
Study Description
• Spring 2014: 1,549
students from 4 CC’s in
a large urban system
• 40% African American,
20% Hispanic/Latino
• 25% Credit bearing
• 65% Remedial credit
• 15% Non‐credit
Placement
Test Scores
Decision
Zones
+ SuccessNavigator
= COURSE ACCELERATION
1. Students with green
acceleration indices (56%)
were more likely to pass
their math courses than
students with yellow
indices (46%; p < .01).
RESULTS
2. This effect was larger at
higher‐level math courses:
3. Students who were
accelerated into college‐
level math courses (SN +
placement test) passed at
rates similar to students
who placed “naturally”
(placement test only).
See Rikoon et al. (2014)
Green Yellow
College 66.8 37.7
Dev. Ed. 1 53.4 48.3
Dev. Ed. 2 47.3 47.4
Dev. Ed. 3 53.3 50.7

17. • The decision zone model is designed to simplify and
standardize the placement process
• There are several ways of creating these zones:
– A normative approach – e.g., the top 20% of students in a
placement level
– A distribution approach – e.g., within one standard
deviation of the cut score
– A measurement approach – e.g., within one standard
error of the cut score
• Factors to consider when choosing a decision zone
include the size of the band and the number of
students who would be eligible for acceleration
17
Creating Decision Zones

18. 18
Combining Holistic Placement
and Holistic Advising

19. Course Placement
• Students face a long,
complex sequence of
developmental
courses.
Early Academic
Success
• Students lack the
effective behaviors
(organization, study
skills) that are needed
to succeed in college‐
level courses.
Persistence Over
Time
• Without well‐
developed and
aligned goals, self‐
management skills
and social
connections,
students may fail to
persist to a degree.
19
Where We Lose Students ...

20. TESTS +
The Success
Navigator®
Assessment
PLACEMENT
BANDS (?)
ACCELERATE
ACCELERATE WITH
SUPPORTS?
ACCELERATION
DECISION
1st Level
Support
Intrusive
Advising
2nd Level
Support
Supplemental
Instruction
DO NOT
ACCELERATE
Instructional interventions at
the class/program level
Interventions at the
individual student level
26

21. Student J
• 15 ACT®, 2.5 HSGPA
• Strong Academic Skills
• Moderate Commitment
• Low Self-management
• Strong Social Support
• Why does Student J
have such a strong
likelihood of success?
• What interventions
might we recommend
for Student J?
21

22. Student J
• Academic Skills and
Commitment to College
Goals are very relevant to
both academic success
and persistence
• Low Self-management can
actually increase students’
likelihood for persistence
(a lack of “stress” can also
be a lack of
“engagement”)
• Moderate Academic
Success Index is likely
coming from tests
scores and HSGPA:
tutoring and other
academic interventions
are likely most important
22

23. Student J – Self-
management Interventions
23

24. Understanding your institution’s resources...
Academic Skills Commitment Self-management Social Support
 CAPS
 Accessibility Resource
Center
 College Enrichment &
Outreach Programs
(e.g., CEP, SSS)
 Advisement
(University, Fine Arts,
Engineering)
 Academic Coaching
 Academic Level
Student Support (e.g.,
Engineering Student
Services, Student
Services, etc.)
 Faculty Office Hours
(Noted on course
syllabi)
 Career Services
 Academic Departments
 Student Activities (i.e.,
Greeks, Student
Organizations, etc.)
 Advisement (University,
Fine Arts, Engineering)
 Alumni Support
Mentoring Program
 Student Employment
 Residential Life
 Student Health &
Counseling Center (SHAC)
CAPS
Center, Veteran’s
Resource Center)
Religious Organizations
(e.g., Newman Center, etc.)
College Enrichment &
Outreach Programs (e.g.,
CEP, SSS)


 Academic Coaching
 Recreational
Services_Johnson Center
 Targeted Resource
Centers (African-
American Student
Services, American-Indian
Student
 Targeted Resource
Centers (African-American
Student Services,
American-Indian Student
Services, El Centro,
LGBTQ Resource Center,
Men of Color Initiative,
Women’s Resource
Services, El Centro,
LGBTQ Resource Center,
Men of Color Initiative,
Women’s Resource
Center, Veteran’s
Resource Center)
 Student Activities (i.e.,
Greeks, Student
Organizations, etc.)
 College Enrichment &
Outreach Programs (e.g.,
CEP, SSS)
 Dean of Students
 Student Employment
 Recreational
Services_Johnson Centerhttp://success.unm.edu/success‐navigator/index.html
29

25. <<Institution>>
• Administration window: <<08/01/13–09/15/13>>
• Total sample size = <<1,138>>
SUCCESS INDICES
Predictions of student success are based on two criteria. First, ACADEMIC SUCCESS is a student’s likelihood of succeeding in the classroom, indicated by
GPA. Second, ENROLLMENT SUCCESS indicates a student’s likelihood of returning to your institution for a second year. Both of these scores are modeled
using a large, nationwide study across varying types of institutions and students. They have been shown to be highly predictive of student success.
The tables below show the proportion of your students who have fallen in each of three categories — high, medium and low likelihood of success.
Institution Report
25
Academic Success Index
Success Likelihood Definition Percent of Students Across
Comparable Institutions
Percent of Students in the
Cohort/Institution
High Projected 1st Semester GPA
> 2.97
26 33
Medium Projected 1st Semester GPA
between 2.23 and 2.97
46 46
Low Projected 1st Semester GPA
< 2.23
27 21
Retention Success Index
Success Likelihood Definition Percent of Students Across
Comparable Institutions
Percent of Students at Your
Institution
High Probability of retention
> 93%
21 42
Medium Probability of retention between
84.1% and 93%
48 29
Low Probability of retention < 84.1% 32 39

26. The pages that follow will provide subscores within each domain to explain them more thoroughly.
26
Institution Report
GENERAL SKILL SCORES

27. • Ross Markle
• Senior Research & Assessment Advisor, Global Education Division,
ETS
• rmarkle@ets.org or 1-615-835-3483
• Vikki Monaghan
• Director of Strategic Initiatives, Global Education Division, ETS
• vmonaghan@ets.org or 1-609-683-2965
Copyright © 2015 by Educational Testing Service. All rights reserved. ETS, the ETS logo, LISTENING. LEARNING. LEADING. and
SUCCESSNAVIGATOR are registered trademarks of Educational Testing Service (ETS). All other trademarks are the property of their respective owners.
27563
27
Contact Information

28. • Adams, P., Gearhart, S., Miller, R., & Roberts, A. (2009). The Accelerated Learning Program: Throwing Open the
Gates. Journal of Basic Writing (CUNY),28(2), 50-69.
Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in developmental education
sequences in community colleges. Economics of Education Review, 29(2), 255-270.
Boylan, H. R. (2009). Targeted intervention for developmental education students (TIDES). Journal of Developmental
Education, 32(3), 14-23.
Burdman, P. (2012). Where to begin? The evolving role of placement exams for students starting college [Report].
Boston, MA: Jobs for the Future.
Complete College America. (2012). Remediation: Higher education’s bridge to nowhere. Washington, DC: Author.
Conley, D. T. (2007). Toward a more comprehensive conception of college readiness. Eugene, OR: Educational Policy
Improvement Center.
Levine-Brown, P., Bonham, B. S., Saxon, D. P., & Boylan, H. R. (2008). Affective assessment for developmental students,
part 2. Research in Developmental Education, 22(2), 1-4.
Li, K., Zelenka, R., Buonaguidi, L., Beckman, R., Casillas, A., Crouse, J., Allen, J., Hanson, M.A., Acton, T., & Robbins, S.
(2013). Readiness, behavior, and foundational mathematics course success. Journal of Developmental Education, 37(1),
14.
Markle, R.E., Olivera-Aguilar, M., Jackson, T., Noeth, R., & Robbins, S. (2013). Examining evidence of reliability, validity,
and fairness for SuccessNavigator. (ETS Research Report 13-12). Princeton, NJ: Educational Testing Service.
Olivera-Aguilar, M., Markle, R., & Robbins, S. (April, 2014). Using latent profile analysis to identify profiles of college
droupouts. Paper presented to the National Council on Measurement in Education: Philadelphia, PA.
Rikoon, S., Liebtag, T., Olivera-Aguilar, M., Robbins, S., & Jackson, T. (2014). A pilot study of holistic assessment and
course placement in community college: Findings and recommendations. ETS Research Memorandum (No. RM-14-10).
Princeton, NJ: Educational Testing Service.
Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors
predict college outcomes? A meta-analysis. Psychological Bulletin, 130, 261-288.
Russell, A. (2008). Enhancing college student success through developmental education. (Higher Education Policy Brief ).
Washington, DC: American Association of State Colleges and Universities.
Saxon, D. P., Levine-Brown, P., & Boylan, H. R. (2008). Affective assessment for developmental students, part
1. Research in Developmental Education,22(1), 1-4.
Scott-Clayton, J. (2012). Do high-stakes placement exams predict college success? (CCRC Working Paper No. 41). New
York: Community College Research Center. Retrieved online from http://ccrc.tc.columbia.edu/Publication.asp?UID=1026.
Twigg, C. A. (2011). The math emporium: Higher education’s silver bullet. Change: The Magazine of Higher
Learning, 43(3), 25-34.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
28
References