This is a brief presentation on comparing standardized tests and scored e-portfolio artifacts.

1. Matching E-Portfolios to Standardized
Tests for Value-Added Assessment
Richard Robles
EDLD 812: Data Improvement

2.  About the University Honors Program
 UC’sinvolvement in the Voluntary System of
Accountability
 Comparing the Collegiate Learning Assessment to
e-portfolio artifacts
 What we have learned thus far

3. University Honors Program
 Revised the program during AY 2007-08 to focus honors activities on
experiential learning in the following thematic areas:
 Community engagement
 Global studies
 Leadership
 Research/Creative arts
 Comprised of1,600 students from all of the colleges within the
University (except Raymond Walters and Clermont Colleges)
 Last 2 entering cohorts represent about 25% of the population
 Average entering ACT is 31 (SAT 1346)

4. Voluntary System of Accountability
 Voluntary initiative for 4-year public colleges and
universities
 Developed through a partnership between the AASCU
and APLU to help institutions
 Demonstrate accountability and stewardship to public
 Measure educational outcomes to identify effective
educational practices
 Assemble information that is accessible, understandable,
and comparable

5. Voluntary System of Accountability
 Ohio is the only state to have all state institutions
participating in the VSA
 UC’s initial participation is the implementation of the
Collegiate Learning Assessment

6. Collegiate Learning Assessment (CLA)
 Developed by the Council for Aid to Education (CAE) in
2004 – www.cae.org/cla
 A standardized and nationally normed test that measures
the institutional contributions (value added) to the
learning gains made by students
 Direct assessment of student learning

7. Collegiate Learning Assessment (CLA)
 Areas of measurement
 critical thinking
 analytical reasoning / problem solving
 written communication
 Time required:
 90 minutes for a performance task (complex problem to be
solved)
 75 minutes for two writing prompts (make an argument and
break an argument)

8. The CLA’s Range
 First-year students and then seniors
 Longitudinal study is better
 Cross-sectional study is possible (but more expensive)
 The institution is the primary unit of analysis
 Institutions can be compared for value-added gains

9. The “Dual Pilot” study at UC
 To consider the applicability of learning portfolios (and
especially e-portfolios) alongside the CLA among a
group of first-year Honors students
 Best strategies towards
 The transition to become student scholars
 The development of General Education baccalaureate
competencies over the first year
 The need for intervention mechanisms to facilitate further
transitions

10. Methodology
 Administer the CLA as a course requirement to 117 first-
year, first-time baccalaureate seeking, Honors students
 Average ACT is 31
 Representing a range of UC colleges and programs
 Half took the CLA tests to “make an argument” and to
“critique an argument,” with the other half doing the
problem-solving “performance task”

11. Additional information for gathering
 Level of effort exercised in completing each instrument
 Sense of engagement with this process
 Sense of learned perceptions from completing the CLA
test
 Reactions to and self-assessment of their performance
 Role of any “ceiling effect”

12. E-Portfolio Artifacts
 Two essays posted in the e-portfolio with
 Focus on personal academic development
 Focus on designing a experiential learning project
 Use of AAC&U metarubrics
 Integrative Learning
 Critical Thinking
 Written Communication

13. PreliminaryResults:Overall Impression
I believe the test is a waste of time and that it 6 6.6%
would not be at all useful in testing students’
abilities
I believe the test would be only mildly useful in 40 44%
testing students’ abilities
I believe this test would be useful in testing 36 39.6%
students’ abilities
I believe the test would be very useful in testing 9 9.9%
students’ abilities
I believe this test is one of the most useful tests I 0 0%
have taken and that it will provide outstanding
insight into students’ abilities

14. Preliminary Results: Testing
Experience
Taking this test was a very unpleasant experience 9 9.9%
Taking this test was a somewhat unpleasant 25 27.5%
experience
Taking this test was neither a pleasant nor 43 47.3%
unpleasant experience
Taking this test was a somewhat fun, interesting or 12 13.2%
pleasant experience
Taking this test was a fun, interesting or pleasant 2 2.2%
experience.

15. Preliminary Results: Effort Expensed
Almost no effort at all 2 2.2%
A little effort 5 5.5%
Some effort 36 36.9%
Very much effort 41 45.1%
Serious and sustained effort 7 7.7%

16. Conclusions Thus Far
 Questionable reliability of results
 Lack of integration into the curriculum
 Disconnect between the CLA and UC’s mission
 Absence of ownership by the faculty
 Questionable motivation by seniors taking the test
 High costs to administer the test (over $100 per student)

17. Matching E-Portfolios to Standardized
Tests for Value-Added Assessment
Richard Robles
EDLD 812: Data Improvement