Dimova: National oral language assessment in Denmark: Where are we now and where are we going?
1. Centre for Internationalisation and Parallel Language Use
National oral language
assessment in Denmark:
Where are we now and
where are we going?
Slobodanka Dimova
Centre for Internationalisation and Parallel
Language Use (CIP)
2. Centre for Internationalisation and Parallel Language Use
Introduction
Overview of Danish educational system
Oral English language exams in 9th grade -Folkeskolens Afgangsprøve (FSA)
A-level and B-level oral English exams in upper
secondary school (gymnasium)
Synergies and tensions between different aspects
of exam development, administration, and use
Future developments and research
3. Centre for Internationalisation and Parallel Language Use
Overview of the Danish education system
(http://eng.uvm.dk/Education/Overview-of-the-Danish-Education-System)
FSA
4. Centre for Internationalisation and Parallel Language Use
Foreign language examination system
New national structure to promote evaluation and assessment in
compulsory education
•Since 2006, creation of a new national structure to establish the
importance of evaluation and assessment in compulsory education
•Three national bodies with a role in the evaluation and assessment
of compulsory education: the Education Agency; the Quality and
Supervision Agency; and the Danish Evaluation Institute (EVA)
•The Quality and Supervision Agency is responsible for the
financial, institutional and pedagogical supervision of both
compulsory and upper secondary education. Major tasks include
developing and running the national tests and final
examinations in compulsory education, implementation of
international assessments, development of materials for schools in
the Evaluation portal.
5. Centre for Internationalisation and Parallel Language Use
FSA ORAL ENGLISH EXAM (9th grade)
The FSA oral English exam consists of two tasks:
1. Presentation on a selected topic (~5 min.)
2. Conversation/interview on specific topics (~8 min.)
Total examination time including evaluation
(~20min.)
Individual exam (though pupils can prepare in pairs or
groups)
Two examiners (teacher and external examiner)
Mandatory for all pupils
6. Centre for Internationalisation and Parallel Language Use
Before the exam day
• The teacher, together with pupils, develops a bibliography
with reading and video material organized around at least
three main themes. Each theme consists of several different
topics.
• The pupil chooses a topic under one of the themes.
• The pupil selects sources related to the chosen topic and
prepares the presentation. This is done with the teacher as
supervisor.
• The pupil prepares and submits an outline of his/her
presentation before the exam day.
• The teacher approves the student's outline with a signature.
The pupil also signs the outline.
7. Centre for Internationalisation and Parallel Language Use
Examples of themes and topics
Themes
Topics
Life in South Africa before and
after the Apartheid
Life in the townships
AIDS in SA
Apartheid
Racism
Sports in SA
Story/movie/poem related to the
topic
Gun violence in the USA
The American Constitution
The Ku Klux Klan
Gang life
Eminem’s lyrics
Fit or fat?
Fitness
Children and obesity
Eating disorders
What’s wrong with McDonald’s?
What’s healthy living?
8. Centre for Internationalisation and Parallel Language Use
An example of presentation outline
Pupil’s name and class:
Theme: Young People Today
My topic: Obesity
1. Why did I choose this topic?
a) My family background. b) Films I’ve seen. c) A personal interest
2. Different kinds of eating disorders
Obesity due to binge-eating; Anorexia, Bulimia
3. Who becomes obsessive – and why?
A case story
4. Whose problem is it – and what should be done about the problem?
A personal problem
”Society is to blame”- or?
School Lunch
TV-programs
Sources:
Morgan Spurlock’s film “Super Size Me”,
Jamie Oliver’s Diary : http://www.jamieoliver.com/diary/
British School Dinners:
http://www.channel4.com/life/microsites/J/jamies_school_dinners/fom/index.html ns
Student’s signature:
Teacher’s signature:
9. Centre for Internationalisation and Parallel Language Use
During the exam
Task 1: Pupils give a 5-min presentation on the selected topic.
• Internal-examiner (teacher) can ask questions related to
the topic
• External-examiner (censor) can ask only clarifying
questions
• Examiners can use the pupil’s presentation outline as the
basis for questions during the exam
Task 2: Pupils draw one card from a number of cards containing
a question and pictures related to the themes covered in the
bibliography.
• Pupils should take an initiative in a conversation with the
examiners
11. Centre for Internationalisation and Parallel Language Use
Grading criteria
One overall grade based on
four areas:
1.Communicative skills
2.Language use
3.Preparation (acquisition)
4.Culture and society
Teacher and censor each
assign a grade, but
final/reported grade is
decided through discussion.
12. Centre for Internationalisation and Parallel Language Use
After the exam
The external-examiner (censor) prepares a report
discussing:
•
•
•
•
•
•
•
•
•
Text selection
Themes and topics
Pupils’ outlines
Sources used for preparation
Questions in Task 2
Common mistakes
Examination process
Grading
Complaints
The report is published and used for improvement of
future examination procedures.
13. Centre for Internationalisation and Parallel Language Use
Synergies
• Dynamic collaboration among pupils, teachers,
and external-examiners, involvement of all
stakeholders in the process
• Direct relation to material covered in the
classroom, i.e. integrated teaching, learning,
and assessment
• Clear expectations about the examination tasks,
i.e. examination procedure transparency
and lower negative individual impact
14. Centre for Internationalisation and Parallel Language Use
Tensions
• The exam is in line with the Common Goals outlined by
the curriculum. However,
• The Common Goals (Fælles Mål) are broad and lack
specific description of the assesed construct.
____________
• FSA is nationally mandated and grades are used to a
certain extent for school and teacher comparison.
However,
• The local administration of the exam brings a large
degree of variability (e.g., topics, task 2, grading,
mentorship).
15. Centre for Internationalisation and Parallel Language Use
A-LEVEL ORAL ENGLISH EXAM (stx)
The A-level oral English exam consists of two
parts:
1.Presentation based on text analysis
2.Interview about the analysis
Preparation time (60min.)
Examination time including evaluation
(~30min.)
Two examiners (teacher and external examiner)
Only randomly selected pupils take it
16. Centre for Internationalisation and Parallel Language Use
Before the exam day
• Pupils prepare throughout all three years in
gymnasium by
• reading texts related to different topics
• analyzing texts
• learning different terminology used in text analysis
• Teachers prepare at least three new texts related to
the topics covered in class and send them to external
examiners (censors)
• Censors check the teacher selected texts for the
fulfillment of the requried criteria and approve them
•
Pupils are selected to take the exam by random draw
17. Centre for Internationalisation and Parallel Language Use
During the exam: Task (text analysis)
• Pupils are asked to prepare a presentation based on
a literary or non-literary text. In the presenation they
should,
• Analyze the text
• Interpret the text
• Relate the text to a topic discussed during the
semester
• The text specifications are as follows:
• 2-4 pages
• New (not discussed in class)
• 1300 letters (if a short story or book chapter)
• 30 lines from a poem
• 30 lines from a play
18. Centre for Internationalisation and Parallel Language Use
Task example
Theme: Urban Life Past and Present
Text: An excerpt from the novel Saturday by Ian McEwan,
Jonathan Cape 2005, p. 74-81.
The novel is set in London, 2003, and describes a day in
the life of Henry Perowne, a succesful neurosurgeon. In
this excerpt he is on his way to his weekly game of
squash. The streets are filled with anti-war protestors.
The occasion is the impending war against Iraq.
…the excerpt follows…
19. Centre for Internationalisation and Parallel Language Use
The emphasis is on:
1. Coherent (fluent) and mostly correct English
2. Presentation, analysis, understanding, interpretation and
discussion in relation to the knowledge gained in working with
the topic studied
3. Communicating one’s own views and arguments
20. Centre for Internationalisation and Parallel Language Use
Synergies
• Some collaboration among pupils, teachers, and
external-examiners, involvement of all
stakeholders in the process
• Direct relation to the material covered in the
classroom, i.e. some integration of teaching,
learning and assessment.
• Clear expectations about the examination tasks,
i.e. examination procedure transparency
and lower negative individual impact.
• Integration of two language skills reading
and speaking.
21. Centre for Internationalisation and Parallel Language Use
Tensions
• The exam is nationally mandated and grades are
used to a certain extent for school and teacher
comparison. However,
• The local administration of the exam brings a large
degree of variability.
______________
• Emphasis is on literary text analysis. However,
• Grades are used for admission in EMI programs
at university level.
22. Centre for Internationalisation and Parallel Language Use
Where are we now?
• Traditional oral exams—have existed a long time with
small revisions
• Practicality:
• FSA and A-level/B-level oral exams are administered
locally
• No additional resources needed except for censors
• Authenticity:
• Materials in prompts are authentic
• Authentic tasks?
23. Centre for Internationalisation and Parallel Language Use
Where are we now?
• Reliability:
• inter- and intra- grader reliability?
• administration reliability?
• pupil reliability?
• Validity:
• content validity
• construct underdefined, especially for FSA
• Possible construct-irrelevant variation?
• Consequences:
• stx scores use for admission in EMI
• FSA scores used for employment
• Washback:
• too much emphasis on literary texts
• 9th grade FSA focused on topic and presentation
development
25. Centre for Internationalisation and Parallel Language Use
Where are we going?
OPTION 1
• Keep traditional testing methods, but restrict
result interpretations to local contexts
• No generalization
• No school comparison
• Local use
• Advisory purposes
26. Centre for Internationalisation and Parallel Language Use
Where are we going?
OPTION 2
• Improve definition of the construct
• Standardize tasks
• Standardize examination procedures
• Monitor intra- and inter-rater reliability
• Conduct on-going validation investigations
• Re-think the relationship between tasks and
expectations for real-life language use
• Follow the consequences of exam changes
27. Centre for Internationalisation and Parallel Language Use
The right way…
…depends on many factors.
• If FSA and A-level exams become higher stakes
exams, quality measures have to be
implemented
• Tradition could still be maintained
• Balancing practicality and other exam qualities
will still greatly depend on the available
resources (financial, material, professional)
