1. Using a Standards Alignment Model as
a Framework for Doctoral Candidate
Assessment
Sketching a Road Map
CPED October 2013 Convening
Hosted by Rutgers University
Dr. Valerie A. Storey

2. Fall Convening, 2007
Hosted by Peabody College, Vanderbilt University
The question is…
“How do we create a framework for assessment
and accountability that takes advantage of our
diversity and yet helps us account for our
efforts to reclaim education’s doctorates
within and across programs, strands, and
institutions?”


3. CPED Meeting at American Association of Colleges for
Teacher Education (AACTE), New Orleans,
Imig & Perry, 2008
Program Design Concepts:
Capstones
•Backward mapping
•Use of state standards
•Program design & leadership analysis

4. Spencer Foundation Grant, 2008
Preparation of Professional Practitioners


$75,000 funding to identify outcomes of phase 1
institution programs.
Consortium identified six characteristics of graduates
that should result from preparation in a CPEDinfluenced EdD program:
1. Equity
4. Inquiry stance
2. Commitment to continuous change
5. Community engagement/social
responsiveness
3. Leadership capabilities
6. Harnessing human capital
Ref: Imig, Perry, Syed, 2009

5. Palo Alto, CA Convening,
June 2009

Goldring & Yinger challenged consortium
members to



Reflect on the results of the outcome data derived from the
Spencer funded research;
Consider how outcomes would be tested both by program and
candidate.
Members responded by developing principles of
best practice for CPED EdD programs.
Ref: Perry & Imig, 2010

6. Working Principles for the Professional Practice
Doctorate in Education
1.Is framed around questions of equity, ethics, and social justice to bring about
solutions to complex problems of practice.
2.Prepares leaders who can construct and apply knowledge to make a positive
difference in the lives of individuals, families, organizations, and communities.
3.Provides opportunities for candidates to develop and demonstrate collaboration
and communication skills to work with diverse communities and to build
partnerships.
4.Provides field-based opportunities to analyze problems of practice and use
multiple frames to develop meaningful solutions.
5.Is grounded in and develops a professional knowledge base that integrates both
practical and research knowledge, that links theory with systemic and systematic
inquiry.
6.Emphasizes the generation, transformation, and use of professional knowledge
and practice.

7. CPED Meeting at American Association of Colleges
for Teacher Education (AACTE)
Atlanta Hilton - February 19, 2010
Program Design Concepts:
CPED



Start
members:
Need to operationalize conceptual ideas;
Need to stay faithful to CPED definitions as we move from the theoretical to the operational;
Need to develop an assessment model grounded on CPED principles, and test over time.

Identify what we have and what is the gap?
a student-generated data base that is evidence of how we work with CPED
principles
Signature assessments: Define 3 that all institutions use and study their efficacy
Strong formative aspect and summative; focus on capacities of students as they
develop and then leave the program; student-tracking database;
Content analysis of capstone process; guiding question of what impacts our grads are
making in the field

8. This presentation-3 broad topics
How we decided as an institution to move
forward



Process of program design
Development of “gatekeepers.”
Design of assessment activities to provide
ongoing data for continuous improvement

9. We relied on two inter-related
theories of action
1.
2.
Understanding institutional change as
individual and collective learning growth,
and capacity building grounded by CPED
critical friends ( Tharp, Estrada, Dalton,
Yamauchi, 2000);
Utilizing curriculum design as the
framework for planning for institutional
renewal.

10. Starting with the End in Mind:
What Are Our Intended Results?

Outcomes: What changes in student attitudes, behaviors,
skills, status, and/or level of functioning do we intend?

Outputs: What changes in our program design do we
hypothesize will lead to these outcomes?

Impact: How will our outcomes enhance educational practice,
research, and policy in the metropolitan area that the
university serves?

BUT FIRST-who are our candidates? Conduct needs assessment

11. Soul searching…

Putting the Needs Assessment data to work



Vision statement-What is the purpose of the program?
What is the rationale and educational purpose of each
element of the professional practice doctoral program?address themes, elements, and learning outcomes.
What evidence aids the answering of the above
questions?
Golde, Jones, Bueschel, Walker, 2006

12. Cognizant of …

More value in a department assessing the quality of its
program regularly and continuously when done in
reference to clearly agreed upon vision for the program
and its students;
Golde, Jones, Bueschel, Walker, 2006

Need to measure the growth and development of
students during the doctoral program.
Golde, Jones, Bueschel, Walker, 2006

13. Coincidentally…

University actively develops its outcomes assessment program, requires faculty in all
programs to develop statements of intended learning outcomes and gather assessment
data to evaluate whether or not the desired outcomes are being achieved;

Program Advisory Committee-Educational Leadership & Interdisciplinary faculty begin a
two-year process of identifying the knowledge, skills, competencies, and perspectives
that they wished all students to possess upon graduation;

Create a set of standards (i.e., rubrics) that describe the characteristics of excellent work
in each of the learning outcome areas.

Develop student and faculty guidelines for using LiveText to craft a Scholarly Portfolio
incorporating assignment rubrics, enabling students to monitor and shape their own
learning and, in the process, develop reflective and self-awareness skills.

14. 1. Program theory of action underpins program design and
identified assumptions on
candidate outcomes









What content is essential and why?
Which pedagogical practices are essential?
What learning experiences are created, and how are these developmental?
How are Laboratories of Practice designed and supported ?
What program supports (such as cohort structure, advisement, and alternative
delivery formats) facilitate program delivery?
What is the role of assessment in fostering candidate development?
What are the knowledge, skills, and program commitments of core faculty
How does the program recruit faculty, and what values guide faculty recruitment and
selection, as a means of pursuing the program’s theory of action?
What assumptions link these components (i.e., how do these components, taken
together, effectively prepare candidates who reflect the program’s core values)?

15. 2. Program Design & Program Standards





What are the key understandings (e.g., big ideas)?
What are the essential questions of the field that this course addresses?
What are students expected to know and be able to do as a result of this
course?
What performance tasks will students do to demonstrate their knowledge
and understanding?
Core readings:



To what extent do readings engage candidates in connecting core research with the skills
necessary to support recommended practices?
How are candidates asked to measurably demonstrate competency?
How do these relate to the program’s theory of action?

16. Setting Quality Program Standards

file://localhost/Volumes/KINGSTON/Academic A

17. Mapping to Program Standards
Exemplary (3 pts)
Leadership
FL-LYNNEDD.3.1
Leadership
FL-LYNNEDD.3.3
Proficient (2 pts)
Unacceptable (1 pt)
Analyzes an authentic problem of
practice in which the elements and
interactions of leadership, vision,
mission and goals can be maximized
and lead to an inclusive action plan or
agenda which is clear, easily translated
into work tasks, and evaluated when
completed.
Understands and, with minor difficulty,
analyzes an authentic problem of practice
exhibiting the inter-relatedness among
leadership, vision, mission, and goals and
plan development.
Is not able to analyze an authentic
problem of practice or exhibit an
understanding of the interactive
nature between leadership, vision,
mission, and goals or translate them
into a coherent action plan or agenda.
Is able to identify the situational,
contextual, and cultural aspects of an
organization which are necessary to
attain balance and that will lead to
enhanced educational practice and
outcomes.
Has some understanding of the
situational, contextual and cultural
aspects of an organization relative to
balance and enhanced educational
practice and outcomes.
Is not able to identify or relate any
specific organizational context,
culture, or situationally unique aspects
of organizational balance in any
setting to enhance educational
practice and outcomes.
Standard 3-Leadership for Learning,
EDU 701- Leadership, Policy & Context , Critical Assignment Performance Rubric

18. Authentic Measure
Scholarly Practitioner Portfolio (SPP)

Electronic Web-based format i.e. LiveText.

Organization of the SPP framed by the program’s standards & learning
outcomes.

Incorporates




Summary data, curriculum vita etc
Individual reflection papers and artifacts matched against the dimensions of each
learning outcomes. Emphasis on meta cognitive development.
Demonstrate via portfolio evidence the ability to address practitioner problem with
scholarly knowledge.
Admission to candidacy is predicated on successful presentation of SPP to
SPP committee.

19. Scholarly Practitioner Electronic Portfolio

Scholarly Practitioner Portfolio presentation:

Authentic assessment evaluating the desired learning
outcomes of the program, abilities and dispositions that
characterize a professional practice doctorate;
(Wiggins, 1998).

Authentic tests require the performance of exemplary
tasks, individual judgments, reflection, and dialogue.
(Wiggins, 1989).

20. Scholarly Practitioner Portfolio (SPP)
Presentation

Student:



Presents selected assignment that addresses a program
standard;
Reflects and demonstrates via artifacts that program knowledge
applied to practitioner competencies;
SPP Faculty Committee:




Individual rubrics combined for overall assessment.
Verbal and later written feedback to student.
Identification of program standards met –strengths and
weakness in meeting learning outcomes;
Program review to address identified program weakness.

21. Dissertation in Practice

Designed to engage students in a consultative
relationship with an educational agency e.g. school
district, college, university, or other educational
organization;

Demonstrate the knowledge & competencies of a
scholarly practioner;

22. 
Both the SPP & DiP designed to provide faculty
with data they could use to make judgments not
only about individual students’ learning, but also
about students’ achievements collectively
through program assessment activities i.e. a
strategy that has the potential to establish a
reciprocal learning and assessment relationship
for both candidate and faculty.

23. References






Imig, D., Perry, J.A., & Syed, S. (2009). Creating rubrics for the assessment of the
EdD: Narrative report to the Spencer Foundation. College Park, MD: Carnegie
Project on the Education Doctorate.
Jacobs, H. H. (1997). Mapping the big picture: Integrating curriculum and
assessment K-12. Alexandria, VA: Association for Supervision and Curriculum
Development.
Maki, P. L., & Borkowski, N.A. (2006) (Eds). The assessment of doctoral education:
Emerging criteria and new models for improving outcomes. Sterling, Virginia: Stylus
Publishing.
Wiggins, G. (1998). Educative assessment: Designing assessments to inform and
improve student performance. San Francisco: Jossey-Bass.
Perry, J.A., & Imig, D. (2010). Final Report: The Carnegie Project on the Education
Doctorate 2007-2010.
Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA:
Association for Supervisions and Curriculum Development.