Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits

© 2013 Springer Publishing Company, LLC. 1
Chapter 18
Program Assessment
&Oermann Gaberson
Evaluation and Testing in Nursing Education
4th edition

Program Assessment
♦ Process of judging the worth or value of an
educational program
♦ Purposes
– Provide data on which to base decisions about the
program
– Provide evidence of educational effectiveness in
response to internal and external demands for
accountability

Program Assessment Models
♦ Many models available
♦ Three commonly used models in nursing
education programs
– Accreditation
– Decision oriented
– Systems oriented

Accreditation Models
♦ Examples
– NLNAC
– CCNE
– CASN
– The Joint Commission
♦ Assess whether the program meets external
standards of quality
♦ Use combination of internal and external assessment
methods

Decision-Oriented Models
♦ Examples
– CIPP model
– CQI model
♦ Provide information to decision makers for program
improvement purposes
– No guarantee that corrective action will be taken if it is
indicated
♦ Usually focus on internal standards of quality, value,
and efficacy

CIPP Model
♦ Context: What needs to be done?
– Program needs, problems, strengths, weaknesses
♦ Input: How should it be done?
– System capabilities, work plans, budgets
♦ Process: Is it being done?
– Monitor implementation, make changes as needed
♦ Product: Did it succeed?
– Impact, effectiveness, sustainability

CQI Model
♦ Based on assumption that those who are
closest to a process know it best
♦ 4 phases
– Plan
– Do
– Study
– Act

CQI Model
♦ Plan
– Define problems and opportunities for
improvement
– Collect and organize data
– Decide on improvement initiatives
♦ Do
– Identify needed resources
– Implement planned initiatives
(cont’d)

CQI Model (cont’d)
♦ Study
– Monitor progress of improvement initiative
– Observe the effects of the change
♦ Act
– Modify improvement as needed
– Incorporate change into program structure and
policies
– Educate stakeholders about the change

Systems-Oriented Models
♦ Assessment of inputs into the program
– Characteristics of students, teachers, administrators, other
participants
– Program resources
♦ Assessment of program processes, operations
– Program implementation
– Context within which the program is implemented
♦ Assessment of program outcomes
– Intended outcomes being achieved
– Stakeholder satisfaction
– Program quality and cost-effectiveness

Stakeholders
♦ Judge worth or value of an educational
program
– Affected directly or indirectly by the decisions
made
– Key stakeholders of nursing education programs
• Students
• Faculty and staff members
• Partners (health care and community agencies)
• Consumers

Stakeholders
♦ Assessment purpose determines stakeholders to involve
– Formative assessment (program improvement)
• Students
• Teachers
• Administrators
– Summative assessment (whether program should be continued,
revised, closed)
• Program participants
• Graduates and their employers
• Prospective students
• Health care and community agencies
• Consumers
• Legislative and regulatory bodies

External vs. Internal Evaluators
♦ Some program assessment models use both
♦ External evaluator
– May be more objective
– May not know program or its context well enough
– Additional expense
♦ Internal evaluator
– Better understanding of program operations and context
– Can provide ongoing feedback
– May be biased

Program Outcome Assessment
Tools
♦ Teacher-made or standardized?
♦ Must possess qualities of effective
measurement instruments
♦ Teachers must be able to make valid and
reliable interpretations of standardized test
results
♦ Should be chosen based on the outcomes to
be measured

Limitations of Outcomes
Assessment
♦ Some intended outcomes may not be important or
related to stakeholders’ needs
♦ May provide feedback only at the end of a program
♦ To contribute to program improvement, program
assessment also should examine
– program goals
– structure
– process
– “side effects”

Curriculum Assessment
♦ Focus is more narrow than program
assessment
– Focus on elements central to the course of studies
taken by students
– Program assessment includes additional elements
• Institutional support for the program
• Administrative structure
• Faculty qualifications and productivity
• Student support services

♦ Focus of curriculum assessment has changed
over time
♦ Classification of assessment models
– First generation
– Second generation
– Third generation
– Fourth generation

Focus of Curriculum Assessment
♦ First-generation model
– Measure to obtain student scores representing curriculum quality
♦ Second-generation model
– Describe congruence between student performance and curriculum
objectives
♦ Third-generation model
– Judge the merit of performance in relation to standards
♦ Fourth-generation model
– Develop holistic understanding of needed improvements based on
input from stakeholders

Curriculum Assessment Elements
♦ Curriculum outcome statements
– Congruent with professional and institutional standards?
– Appropriate to program level?
– Relevant to healthcare context?
♦ Curriculum design
– Good fit of components?
– Reflect curriculum foundation and goals?
– Congruent with program environment?
– Courses logically sequenced?
(cont’d)

Elements (cont’d)
♦ Curriculum outcomes
– Evidence of program outcome achievement?
– Success of program graduates?
♦ Courses
– Course goals congruent with curriculum goals?
– Learning activities consistent with the philosophical
framework and goals?
– Course content current, evidence-based, logically
organized?
(cont’d)

Elements (cont’d)
♦ Teaching-learning methods
– Congruent with philosophical framework?
– Help students to achieve outcomes?
– Respect student diversity?
– Student satisfaction with the quality of teaching?
(cont’d)

Elements (cont’d)
♦ Student assessment methods
– Assess all relevant types of learning?
– Accommodate student input, diversity, feedback
needs?
– Accommodate faculty expertise, preferences,
workload?
(cont’d)

Elements (cont’d)
♦ Resources (human, physical, fiscal)
– Sufficient numbers of appropriately credentialed faculty
members?
– Adequate number and type of staff members to support
the curriculum?
– Classrooms, offices, labs—adequate, comfortable,
appropriately equipped?
– Library holdings appropriate and sufficient?
– Sufficient quality and quantity of clinical placements?
(cont’d)

Elements (cont’d)
♦ Learning climate
– Effect of social, emotional, intellectual atmosphere
on faculty, student, staff quality of life?
– Satisfaction of faculty and students with
relationships, academic freedom, sense of
community?
– Promote diversity of perspective and foster
responsibility and accountability?
(cont’d)

Elements (cont’d)
♦ Policies
– Appropriate, reasonable, applied consistently?
– Well understood by faculty and students?
– Need for additional policies?

Assessment of Online Courses
♦ One element of curriculum assessment
♦ Involves many of the same criteria used to assess
courses offered in traditional classrooms
♦ Additional elements specific to online environment
– Technology
– Accessibility
– Instructional design
– Content
– Interactive activities

Assessment of Online Courses
♦ Challenges
– Online courses hidden from view of faculty
members and administrators who are not directly
involved in teaching them
• Limits the role that these colleagues can play in course
evaluation
• Difficult for outside reviewer to assess elements of
asynchronous learning
– Pace of the learning activities
– Timing of instructor feedback to students

Assessment of Teaching
♦ Focuses on quality of classroom and clinical teaching
♦ Also focuses on other dimensions of the faculty role
– Depend on goals and mission of the nursing education
program
– Examples
• Scholarship and research
• Service to the academic institution, nursing program, community,
nursing profession
• Clinical practice

♦ Research in nursing education: Qualities of effective
teaching in nursing
– Knowledge of the subject matter
– Clinical competence
– Teaching skill
– Interpersonal relationships with students
– Personal characteristics
♦ Consistent with studies about teacher effectiveness
in other fields

♦ NLN’s "Core Competencies of Nurse Educators
with Task Statements" (2005)
– Knowledge, skills, and attitudes required for
effectiveness in the role of nurse educator
– Each competency statement followed by
description of tasks that exemplify it
– Three of the core competencies focus on effective
performance in the role of teacher

Core Competencies of
Nurse Educator
1. Facilitate learning
– Uses variety of teaching strategies
– Communicates effectively
– Shows enthusiasm for teaching and nursing
– Demonstrates interest in learners
– Demonstrates personal attributes that facilitate
learning
– Maintains professional practice knowledge base

Nurse Educator
2. Facilitate learner development and
socialization
– Identifies individual learning needs and styles
– Provides appropriate learning resources
– Creates learning environments that promote
socialization to the role of nurse

Nurse Educator
3. Use assessment and evaluation strategies
– Uses a variety of methods to assess learning
– Gives timely and constructive feedback to learners
– Demonstrates skill in design and use of
assessment tools

Qualities of Effective Teaching
♦ Knowledge of subject matter
– Content area expert
– Understands theories and concepts relevant to
nursing practice
• Can assist students to apply to patient care
– Keeps current with nursing and other interventions,
new developments in areas of expertise, and research
– Necessary but not sufficient; must be able to
communicate knowledge to students

♦ Clinical competence
– If teaching in the clinical setting
– Make sound clinical judgments
– Expert clinical skills
– Can guide students in skill development

♦ Teaching skill
– Identify students’ learning needs
– Plan instruction
– Present material effectively
– Explain concepts and ideas clearly
– Demonstrate procedures effectively
– Use sound assessment practices
• Especially important in clinical evaluation

♦ Positive relationships with learners
– Shows respect for and confidence in students
– Is honest and direct
– Supports students
– Is approachable
– Treats students fairly
– Creates climate of mutual respect between
teacher and student

♦ Personal characteristics
– Enthusiasm
– Patience
– Sense of humor
– Friendliness
– Integrity
– Perseverance
– Courage
– Willingness to admit mistakes

Teaching Effectiveness
Data Sources
♦ Student ratings
– Students can make valid and reliable judgments
about
• teaching methods.
• Fairness.
• interest in students.
• enthusiasm for the subject.
– Class size, course format can affect ratings

Data Sources
♦ Student ratings
– Necessary but insufficient data source
• Students not the best source of data about accuracy,
depth, and scope of the teacher’s knowledge
– Can be obtained in classroom or online
• Assure anonymity and confidentiality

Data Sources
♦ Peer review
– Perspective of another teacher who knows the
content and who has experience working with
similar students
– Best used for formative evaluation
• Reviewer may not be a content or clinical expert in
same area as teacher
• Observations can be influenced by personal feelings
about the colleague
(cont’d)

Data Sources
♦ Peer review (cont’d)
– Should take place within a context of continuous
improvement of teaching–learning process
– Must be supported by adequate resources
• Faculty development
• Mentoring
• Modeling by master teachers
– Observation in classroom, clinical setting,
laboratory
(cont’d)

Data Sources
♦ Peer review (cont’d)
– Online environment
• Visit as guest user, review course materials
• Evidence of effective instruction
– Promotes interaction among students and teacher, active
learning, student engagement
– Communicates appropriate performance expectations; gives
prompt, rich feedback
– Respects diversity of views, learning styles, abilities
– Uses effective instructional and graphic design

Teaching Portfolio
♦ Collection of materials related to teaching
activities in classroom, clinical practice, online
environment, simulation center, or other
settings
♦ Materials selected by the faculty member
should be highly selective and organized to
create a cohesive professional narrative
♦ Variety of formats

Teaching Portfolio
♦ May reflect specific purpose
– Teaching improvement (formative use)
– Promotion and tenure review (summative use)
– Seeking faculty position (career portfolio)
♦ Variety of formats
– Print
– Electronic

Teaching Portfolio
♦ Contents
– Teaching materials
• e.g., syllabi, sample tests, student assignments
– Philosophy of teaching
• Should be reflected in the teaching material
– Additional content for instructional improvement
• Areas of teaching that need improvement
• Description of efforts to develop teaching skills
– Peer and administrator evaluations of teaching
– Self-evaluation of strengths and weaknesses

Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits

Ähnlich wie Chapter 18 ppt eval & testing 4e formatted 01.10 kg edits (20)

Mehr von stanbridge

Mehr von stanbridge (20)