Using a standards alignment model as a framework for doctoral candidate assessment
1. Using a Standards Alignment Model as
a Framework for Doctoral Candidate
Sketching a Road Map
CPED October 2013 Convening
Hosted by Rutgers University
Dr. Valerie A. Storey
2. Fall Convening, 2007
Hosted by Peabody College, Vanderbilt University
The question is…
“How do we create a framework for assessment
and accountability that takes advantage of our
diversity and yet helps us account for our
efforts to reclaim education’s doctorates
within and across programs, strands, and
3. CPED Meeting at American Association of Colleges for
Teacher Education (AACTE), New Orleans,
Imig & Perry, 2008
Program Design Concepts:
•Use of state standards
•Program design & leadership analysis
4. Spencer Foundation Grant, 2008
Preparation of Professional Practitioners
$75,000 funding to identify outcomes of phase 1
Consortium identified six characteristics of graduates
that should result from preparation in a CPEDinfluenced EdD program:
4. Inquiry stance
2. Commitment to continuous change
5. Community engagement/social
3. Leadership capabilities
6. Harnessing human capital
Ref: Imig, Perry, Syed, 2009
5. Palo Alto, CA Convening,
Goldring & Yinger challenged consortium
Reflect on the results of the outcome data derived from the
Spencer funded research;
Consider how outcomes would be tested both by program and
Members responded by developing principles of
best practice for CPED EdD programs.
Ref: Perry & Imig, 2010
6. Working Principles for the Professional Practice
Doctorate in Education
1.Is framed around questions of equity, ethics, and social justice to bring about
solutions to complex problems of practice.
2.Prepares leaders who can construct and apply knowledge to make a positive
difference in the lives of individuals, families, organizations, and communities.
3.Provides opportunities for candidates to develop and demonstrate collaboration
and communication skills to work with diverse communities and to build
4.Provides field-based opportunities to analyze problems of practice and use
multiple frames to develop meaningful solutions.
5.Is grounded in and develops a professional knowledge base that integrates both
practical and research knowledge, that links theory with systemic and systematic
6.Emphasizes the generation, transformation, and use of professional knowledge
7. CPED Meeting at American Association of Colleges
for Teacher Education (AACTE)
Atlanta Hilton - February 19, 2010
Program Design Concepts:
Need to operationalize conceptual ideas;
Need to stay faithful to CPED definitions as we move from the theoretical to the operational;
Need to develop an assessment model grounded on CPED principles, and test over time.
Identify what we have and what is the gap?
a student-generated data base that is evidence of how we work with CPED
Signature assessments: Define 3 that all institutions use and study their efficacy
Strong formative aspect and summative; focus on capacities of students as they
develop and then leave the program; student-tracking database;
Content analysis of capstone process; guiding question of what impacts our grads are
making in the field
8. This presentation-3 broad topics
How we decided as an institution to move
Process of program design
Development of “gatekeepers.”
Design of assessment activities to provide
ongoing data for continuous improvement
9. We relied on two inter-related
theories of action
Understanding institutional change as
individual and collective learning growth,
and capacity building grounded by CPED
critical friends ( Tharp, Estrada, Dalton,
Utilizing curriculum design as the
framework for planning for institutional
10. Starting with the End in Mind:
What Are Our Intended Results?
Outcomes: What changes in student attitudes, behaviors,
skills, status, and/or level of functioning do we intend?
Outputs: What changes in our program design do we
hypothesize will lead to these outcomes?
Impact: How will our outcomes enhance educational practice,
research, and policy in the metropolitan area that the
BUT FIRST-who are our candidates? Conduct needs assessment
11. Soul searching…
Putting the Needs Assessment data to work
Vision statement-What is the purpose of the program?
What is the rationale and educational purpose of each
element of the professional practice doctoral program?address themes, elements, and learning outcomes.
What evidence aids the answering of the above
Golde, Jones, Bueschel, Walker, 2006
12. Cognizant of …
More value in a department assessing the quality of its
program regularly and continuously when done in
reference to clearly agreed upon vision for the program
and its students;
Golde, Jones, Bueschel, Walker, 2006
Need to measure the growth and development of
students during the doctoral program.
Golde, Jones, Bueschel, Walker, 2006
University actively develops its outcomes assessment program, requires faculty in all
programs to develop statements of intended learning outcomes and gather assessment
data to evaluate whether or not the desired outcomes are being achieved;
Program Advisory Committee-Educational Leadership & Interdisciplinary faculty begin a
two-year process of identifying the knowledge, skills, competencies, and perspectives
that they wished all students to possess upon graduation;
Create a set of standards (i.e., rubrics) that describe the characteristics of excellent work
in each of the learning outcome areas.
Develop student and faculty guidelines for using LiveText to craft a Scholarly Portfolio
incorporating assignment rubrics, enabling students to monitor and shape their own
learning and, in the process, develop reflective and self-awareness skills.
14. 1. Program theory of action underpins program design and
identified assumptions on
What content is essential and why?
Which pedagogical practices are essential?
What learning experiences are created, and how are these developmental?
How are Laboratories of Practice designed and supported ?
What program supports (such as cohort structure, advisement, and alternative
delivery formats) facilitate program delivery?
What is the role of assessment in fostering candidate development?
What are the knowledge, skills, and program commitments of core faculty
How does the program recruit faculty, and what values guide faculty recruitment and
selection, as a means of pursuing the program’s theory of action?
What assumptions link these components (i.e., how do these components, taken
together, effectively prepare candidates who reflect the program’s core values)?
15. 2. Program Design & Program Standards
What are the key understandings (e.g., big ideas)?
What are the essential questions of the field that this course addresses?
What are students expected to know and be able to do as a result of this
What performance tasks will students do to demonstrate their knowledge
To what extent do readings engage candidates in connecting core research with the skills
necessary to support recommended practices?
How are candidates asked to measurably demonstrate competency?
How do these relate to the program’s theory of action?
17. Mapping to Program Standards
Exemplary (3 pts)
Proficient (2 pts)
Unacceptable (1 pt)
Analyzes an authentic problem of
practice in which the elements and
interactions of leadership, vision,
mission and goals can be maximized
and lead to an inclusive action plan or
agenda which is clear, easily translated
into work tasks, and evaluated when
Understands and, with minor difficulty,
analyzes an authentic problem of practice
exhibiting the inter-relatedness among
leadership, vision, mission, and goals and
Is not able to analyze an authentic
problem of practice or exhibit an
understanding of the interactive
nature between leadership, vision,
mission, and goals or translate them
into a coherent action plan or agenda.
Is able to identify the situational,
contextual, and cultural aspects of an
organization which are necessary to
attain balance and that will lead to
enhanced educational practice and
Has some understanding of the
situational, contextual and cultural
aspects of an organization relative to
balance and enhanced educational
practice and outcomes.
Is not able to identify or relate any
specific organizational context,
culture, or situationally unique aspects
of organizational balance in any
setting to enhance educational
practice and outcomes.
Standard 3-Leadership for Learning,
EDU 701- Leadership, Policy & Context , Critical Assignment Performance Rubric
18. Authentic Measure
Scholarly Practitioner Portfolio (SPP)
Electronic Web-based format i.e. LiveText.
Organization of the SPP framed by the program’s standards & learning
Summary data, curriculum vita etc
Individual reflection papers and artifacts matched against the dimensions of each
learning outcomes. Emphasis on meta cognitive development.
Demonstrate via portfolio evidence the ability to address practitioner problem with
Admission to candidacy is predicated on successful presentation of SPP to
19. Scholarly Practitioner Electronic Portfolio
Scholarly Practitioner Portfolio presentation:
Authentic assessment evaluating the desired learning
outcomes of the program, abilities and dispositions that
characterize a professional practice doctorate;
Authentic tests require the performance of exemplary
tasks, individual judgments, reflection, and dialogue.
20. Scholarly Practitioner Portfolio (SPP)
Presents selected assignment that addresses a program
Reflects and demonstrates via artifacts that program knowledge
applied to practitioner competencies;
SPP Faculty Committee:
Individual rubrics combined for overall assessment.
Verbal and later written feedback to student.
Identification of program standards met –strengths and
weakness in meeting learning outcomes;
Program review to address identified program weakness.
21. Dissertation in Practice
Designed to engage students in a consultative
relationship with an educational agency e.g. school
district, college, university, or other educational
Demonstrate the knowledge & competencies of a
Both the SPP & DiP designed to provide faculty
with data they could use to make judgments not
only about individual students’ learning, but also
about students’ achievements collectively
through program assessment activities i.e. a
strategy that has the potential to establish a
reciprocal learning and assessment relationship
for both candidate and faculty.
Imig, D., Perry, J.A., & Syed, S. (2009). Creating rubrics for the assessment of the
EdD: Narrative report to the Spencer Foundation. College Park, MD: Carnegie
Project on the Education Doctorate.
Jacobs, H. H. (1997). Mapping the big picture: Integrating curriculum and
assessment K-12. Alexandria, VA: Association for Supervision and Curriculum
Maki, P. L., & Borkowski, N.A. (2006) (Eds). The assessment of doctoral education:
Emerging criteria and new models for improving outcomes. Sterling, Virginia: Stylus
Wiggins, G. (1998). Educative assessment: Designing assessments to inform and
improve student performance. San Francisco: Jossey-Bass.
Perry, J.A., & Imig, D. (2010). Final Report: The Carnegie Project on the Education
Wiggins, G., & McTighe, J. (1998). Understanding by Design. Alexandria, VA:
Association for Supervisions and Curriculum Development.
Hinweis der Redaktion
Ellen Goldring & Dr. Robert Yinger (Professor of Education at the University of Cincinnatti School of Education).
Needed to conceptualize program development as a learning process involving the social construction of knowledge. In our view institutional change would rest on learning and growth of individuals within the organization. We worked first at the individual level before moving to the program level.
We began to ask ourslves the same questions we would as classroom teachers as we addressed curriculm design: program goals, an understading of adult learning, build aggreagted faculty capacity, and use aggregated evidence to make program decisions.
Important role of the Advisory Board
Below are discussion questions to help program faculty & the Advisory Board surface their program theory and the assumptions underlying the choices made in course requirements, assignments, field experiences, faculty, and assessments. The results should enable faculty to assess the degree
of agreement between their espoused and enacted program theory and to evaluate the efficacy of their program theory against research, documented exemplary program models, current trends and priorities in educational leadership, or ideal program models.
What is the program’s theory of action of how program features and experiences develop leadership capacity?
To guide a program in auditing its focus and coherence, answer the following:
Which standards are well covered and which are not?
To what extent do the courses and assignments build candidates’ understanding of and demonstration of competency in standards (i.e., do several courses all address one standard)?
Do the courses work together progressively or developmentally?