This document outlines Dr. Simon Phipps' presentation on language assessment in Turkish universities given at Melikşah University from January 29-31, 2014. The presentation covered several key topics: [1] challenges in English language assessment in Turkish universities including beliefs about language and testing; [2] principles of ethical testing including reliability, validity, and test misuse; [3] the importance of assessment literacy for teachers and test developers; and [4] using tests as part of the learning process with positive backwash. The presentation concluded with a discussion of institutional issues around long-term planning, disconnects between teaching and tests, and the need for external validation.

1. Dr. Simon Phipps
Language Assessment
in Turkey:
Issues and Challenges
MELİKŞAH UNIVERSITY, KAYSERI, 29-31 JANUARY, 2014

3. Outline
!
Overview of the 3 days
!
English language assessment in Turkish universities (1)
!
Ethical testing
!
Assessment literacy
!
Test specifications
!
Testing as part of the language learning process
!
English language assessment in Turkish universities (2)
!
Institutional issues and challenges

4. Some quotes from you
‘When you become a teacher you realise how important it is
to teach your students properly. When you become a test
developer you realise how important it is to write items
properly.’
‘Assessment should not be viewed differently from the
teaching and learning process as good language tests mirror
what is taught in class.’

5. Overview of the 3 days
Wednesday 29
Thursday 30
Friday 31
09.0010.00
Language assessment in
Turkey: Issues/challenges
Applying standards to
testing
Curriculum and
assessment
10.1511.15
Principles of test design
Test design & test writing
Test development
cycle
11.1512.00
Workshop: Tests of
grammar & vocabulary
W/shop: Tests of
receptive skills
W/shop: Test of
productive skills
13.0014.00
Groupwork
Groupwork
Groupwork
14.1515.15
Groupwork
Groupwork
Groupwork
15.1516.00
Group presentations &
feedback
Group presentations &
feedback
Group presentations &
feedback

6. 1
English language assessment in
Turkish universities: some challenges
!
Beliefs about language, learning, testing
!
KPDS, university entrance exam
validity
!
!
!
!
!
reliability
practicality
Multiple choice vs open-ended Qs
‘Value what we test’ vs ‘test what we value’
Speaking skills
Formative assessment
Negative backwash

7. Ethical testing
!
Mis-use of tests (Fulcher, 2010)
!
!
!
Importance of reliability
!
!
gate-keeping vs equality of opportunity
‘ecological sensitivity’
consistent level of tests
Importance of validity
!
what do scores say about learners
‘The task for the ethical language tester is to look into the future, to
picture the effect the test is intended to have, and to structure the test
development to achieve that effect.’ (Fulcher & Davidson 2007:144)
!
Guidelines for good practice – EALTA
http://www.ealta.eu.org/documents/archive/guidelines/English.pdf

8. Assessment literacy
the conscious understanding of principles of assessment,
as well as the necessary skills to design, mark and
give feedback on effective tests (Stiggins, 1995)
! ‘it
is rocket science’ (Fullan, 2001)
! Lack
of focus on assessment in training courses
! Teachers’
! Test
ability to choose/write appropriate tests
writers need to understand:
! Principles
! Levels
(CEFR), and curriculum
! Teaching
and learning (and language)
! Importance
! Test
of test design
of:
development, specifications, evaluation cycle

9. Test specifications
(more detail from Zoltan tomorrow)
! Customer
service – right of users to know
! Specifications
! Writers,
! Link
for:
users, validators
to curriculum and CEFR
! Measure
what’s important
! Beliefs about language learning
! formative?
! EAP?
! Weir
2005, validation framework

10. Testing as part of the learning process
! Use
of tests to motivate learners (Fulcher, 2010)
! Formative
assessment
" results
are locally meaningful
" results
may not be generalisable across contexts
! Positive
backwash
# test
across a range of objectives
# use
a range of different task types
# transparency

11. 2
English language assessment in
Turkish universities: some challenges
!
How to decide the assessment system
placement
intake
achievement
A1
A2
B1
proficiency
university study
B2
progress
!
!
!
!
!
What do we value? What do we want to test?
EAP?
Speaking?
Formative assessment?
Positive backwash?

12. Institutional issues & challenges
‘The basic problem is to develop tests which are valid and reliable,
which have a beneficial backwash on teaching, and which are
practical’ (Hughes, 2003:8).
!
Digital literacy
!
Competency of test writers
!
!
assuming test writing is easy
Long-term planning and investment
!
!
!
setting up a testing unit
baseline data and benchmarking
Assessment-driven educational system
!
!
teaching to the test
!
!
disconnect btw teaching/curriculum/tests
lack of validity and reliability = lack of trust
External validation

13. Some useful resources
!
EALTA
http://www.ealta.eu.org/
http://www.ealta.eu.org/resources.htm
http://www.ealta.eu.org/documents/archive/guidelines/English.pdf
!
!
!
IATEFL Testing & Assessment SIG http://tea.iatefl.org
Prep Programmes Testing SIG <zeynepu@sabanciuniv.edu>
International Language Testing Association
http://www.iltaonline.com/
http://www.iltaonline.com/index.php?
option=com_content&view=article&id=57&Itemid=47
!
Language Testing Resources
http://www.le.ac.uk/education/testing/ltr.html

14. Thank you!
Dr. Simon Phipps