An introduction to educational testing and measurement; a requirement in the course Educ243 - Educational Evaluation

2. Test are
Only
Tools Testing
Process:
Only a Part
of the
Assessment
The
Distinction
Between
Testing and
Assessment
Process
Recent
History of
Educational
Measurement
Current
Develop
ment and
Trends

3. Tests are
Only Tools

4. Concerns:
1. Tests are only tools, and tools can be
appropriately used, unintentionally misused and
intentionally abused.
2. Test, like other tools can be well designed or
poorly designed.
3. Both poorly designed tools and well-designed
tools in the hand of ill-trained and inexperienced
users can be dangerous.

5. Tests are Not Infallible: Usefulness Vary
Across Purposes and Persons
• The most important factor that influence a
tests usefulness is a test’s technical
adequacy
Technical Adequacy includes several
factors. Two that are of primary importance
are:
1. Test validity- performance on the
measure is related to what the measure is
designed to assess
2. Score reliability - it indicates how free the
measurement is from random error
-it generates consistent results

6. Different Purposes: Effects onTest Usefulness
Test’s usefulness can vary depending
on the purpose of testing. A test’s
usefulness , or validity, can be high for one
purpose and low for another.

7. Different Populations:Effects on Test Usefulness
• The evidence of test’s validity and reliability
can also vary depending on the characteristics
of the people the test is used with.
• When evidence of a test’s validity and
reliability is limited for the population being
tested, tests result should be interpreted very
cautiously. And, in such situations, the result
of a single test should never be used
independently to make important decisions.

8. Instead of relying on limited
“snapshot” of student achievement for
important decision making, we
recommend that test results should be
considered to be a part of assessment.

9. Testing Process: Only a Part of the
Assessment
• Educators mistakenly believe that testing and
assessment are synonymous.
• Some seem to have eliminated the word “test” from
their vocabularies and replaced it with the term
“assessment” because they believe that the use of
the word assessment is less evaluative, threatening,
or negative than the use of the word “testing”.

10. • Testing typically occurs on a specific day and
an assessment process may span an entire
semester or even the entire school year.
• Assessment process as a comprehensive
evaluation made up of many components. A
comprehensive assessment process will
include test results from a variety of other
measurement procedures (e.g., performance
and portfolio assessments, observations,
checklists, and rating scales)

11. Testing is one part of the
process of assessment,
but the assessment process
encompasses much more
than just testing.

12. The Distinction Between
Testing and Assessment
Process
Testing
Assessment
Process

13. Testing
• Tests are developed or selected, administered
to the class, and scored.
• Test results are then used to make decisions
about a pupil (assign a grade, recommend for
an advanced program), instruction (repeat,
review, move on), curriculum (replace, revise),
or other educational factors.

14. Assessment Process
Information is collected from tests and other
measurement instruments.
This information is critically evaluated and
integrated with relevant background and
contextual information.
The integration of critically analyzed test result
and other information results in a decision about a
pupil (assign a grade, recommend for an advanced
program), instruction (repeat, review, move on),
curriculum (replace, revise), or other educational
factors.

15. Recent History of Educational
Measurement
Late 1960s, a fairly strong anti-test sentiment
began to develop in our country.
Some decried tests as weapons wilfully used to
suppress minorities. To others, test represented
simplistic attempts to measure complex traits or
attributes.
From the classroom to the Supreme Court, testing
and measurement practice came under close
scrutiny. It seemed to some that tests were largely
responsible for many of our society’s ills.

16. Recent History of Educational
Measurement
Late 1980s, more tests than ever were being
administered.
The report A Nation at Risk by the National
Commission on Excellence in Education in 1983
documented shortcomings of the U.S. public
education system and led to increased calls for
education reform and accountability. Tests and
assessments have been the cornerstone of the
accountability aspects of the education reform
movement.

17. Recent History of Educational
Measurement
Test and assessment-based accountability
became a national priority and requirement
with the passage of the No Child Left Behind
Act (NCLB) in 2002, landmark legislation
intended to raise overall achievement, but
especially for economically disadvantaged
youth.
When tests are used to make such important
decisions, they are called “high-stakes” tests.

18. Recent History of Educational
Measurement
Measurement experts advocate that important
educational decisions should be made based on
the integrated findings from a process of
assessment that includes test scores rather than
from test scores alone.
After 20 years, most have come to realize that
abolishing testing will not be a remedy for the
problem of education and contemporary society.
If test were eliminated, these decisions would still
be made but would be based on non-test data that
might be subjective, opinionated and biased.

19. Recent History of Educational
Measurement
Essay test, knowledge organization
assessment, portfolios, and various
performance tests are increasingly being
utilized in addition to traditional multiple-
choice tests.
Performance and portfolio assessments –
are authentic assessments, a term which suggest
that these measures may be more accurate and
valid than traditional tests.

20. Recent History of Educational
Measurement
Authentic assessments represent the most
objective, valid, and reliable information that
can be gathered about individuals.
Disadvantages:
- costly
- time-consuming to administer & score
- it questions about the evidence for
validity and score reliability

21. Recent History of Educational
Measurement
For the foreseeable future, use of testing and
assessment in education will at least remain at
today’s elevated levels and may even increase.
Competency in educational testing and
assessment practice will enable informed
educators to recognize and engage in “best
practices” in the measurement area.

22. CURRENT DEVELOPMENT and
TRENDS
No Child Left Behind Act (NCLB)
The Rapid Spread of State High-Stakes
Testing Mandates
2004 Reauthorization of the Individuals
with Disabilities Education
Improvement Act (IDEIA-04)
The Need to Identify Pupils at-Risk for
Low Performance on Accountability
Measures
The Lake Wobegon Effect and High-
Stakes Testing
Performance and Portfolio Assessment
Competency Testing for Teachers

23. NoChild Left Behind Act(NCLB)
The reauthorization of the Elementary and
Secondary Education Act (ESEA) in January 2002
become the No Child Left Behind Act (NCLB)
commonly referred as “Nickel-B”.
Intention: To improve educational opportunities
for every American child– regardless of ethnicity,
income or background.

24. 4 Common-Sense Pillars of NCLB
Accountability for results
An emphasis on doing what
works based on scientific
research
Expanded parental options
Expanded local control and
flexibility

25. ImportantFederal Educational Programs of NCLB
• Education for the Disadvantaged
• Reading First/ Early Reading First
• 21st Century Community Learning Centers
• Safe and Drug-free Schools
• Bilingual and Migrant Education
• Education Technology
• Teacher Quality
• Rural Education
• Impact Aid

26. Key Features ofNCLB
Standards and Annual Assessments
Accountability/ Proficiency
Adequate Yearly Progress
Disaggregated Data/ Reporting

27. Standards and Annual Assessments
• Fundamental requirement of NCLB is annual
academic assessment for accountability
purposes.
• All annual assessment under NCLB must be:
- Tied to state standards,
- Applicable to all students
- Technically reliable
• All students in all school must be assessed.

28. Accountability/ Proficiency
NCLB requires states to set ever increasing
annual proficiency goals (e.g. passing scores),
with the requirement that 100% of students
who take the test must achieve the state
standard of proficiency by the 2003-2004 school
year.

29. Adequate Yearly Progress
It requires that sufficient yearly progress must
be made overall for the school and for
economically disadvantage students, students
from major racial and ethnic minority groups,
students with disabilities and limited English
proficient students enrolled in a school since the
beginning of school year.

30. Disaggregated Data/Reporting
Under NCLB schools will have to issue
annual “report cards” that describe students
achievement , the percentage of students
participating in annual assessments, changes in
achievement from previous years and
graduation rates. Results must be reported for
the overall school but broken down, or
disaggregated, for economically disadvantaged
students.

31. The Rapid Spread of State High-Stakes Testing
Mandates
• There now exist a wide variety of state
regulations that require the use of test result
entirely, or primarily to make annual “high-
stakes” about students (e.g., promotion,
graduation), school personnel (e.g., pay
increases and continued employment), and
even control of schools (e.g., state takeover of
low performing school).

32. The Need to Identify Pupils at-Risk for Low
Performanceon Accountability Measures
• Teachers today must prepare all their students
for one or more state, district, or school-wide
benchmark tests. Benchmark tests are used to
identify students at risk for failure on the annual
assessment.
• To identify students who may be at risk to be
low performers on the benchmark tests and the
annual high-stakes test, teachers may
increasingly be encouraged to used standardized
and formal teacher-made measures.

33. The Lake Wobegon Effectand High-Stakes
Testing
Teachers and district administrators
familiar with a standardized norm-
referenced test and it becomes enticing to
“teach to the test” . This is most likely to
occur when standardized test scores
become the only basis for high-stakes
decisions.

34. Performance and Portfolio Assessment
• Performance and portfolio assessment
referred to as authentic assessment began to
gain popularity in the 1990s.
• Test scholars no longer cling to the notion
that accurate assessments of behaviour can
be derived only from formal test but also
from performance and portfolio assessment.

35. –Under IDEIA, children must be evaluated
regularly to assess their ongoing progress in the
general education curriculum through
performance and portfolio assessments.
Purposes:
1. to provide parents with regular report
of progress.
2. to determine whether children with
disabilities as a group are progressing in the
general curriculum.

36. Competency Testingfor Teachers
• In the early 1980s, a number of states passed
legislation requiring teachers to pass paper
and pencil competency test of teaching for
the development of professional teaching
standards
Major Goals:
 To established high and rigorous standards
for what effective teachers should know
and be able to do;

37.  To develop and operate a national, voluntary
system to assess and certify teachers who meet
these standards;
 To advance related education reforms for the
purpose of improving student learning.
The classroom teacher who is trained in
educational testing procedures will be able to use
test results more efficiently and effectively and will
be less likely to misuse or abuse test results.