Testing & Assessing Speaking

1. ASSESSING
SPEAKING
MUSFERA NARA VADIA
1300925
K4-13

2. BASIC TYPE OF SPEAKING
 IMITIATIVE, performance is the ability to simply parrot back
(imitative) a word or phrase or possibility a sentence. The only role
of listening here is in the short-term storage of a prompt, just long
enough to allow the speaker to retain the short stretch of
language that must be imitated.
 INTENSIVE, it employed in assessment contexts is the production of
short stretch of oral language designed to demonstrated
competence in a narrow band of grammatical, phrasal, lexical, or
phonological relationships. The speaker must be aware of
semantic properties to be able respond, but interaction with an
interlocutor or test administrator is minimal at best.

3.  RESPONSIVE, it includes interaction and test comprehension
but at the somewhat limited level or very short
conversation, standard greetings and small talk, simple
requests and comments, and the like.
 INTERACTIVE, the difference between responsive and
interactive speaking is in the length and complexity of the
interaction, which sometimes includes multiple exchanges
and/or multiple participants.
 EXTENSIVE (monologue), extensive oral production tasks
include speeches, oral presentations, and storytelling,
during which the opportunity for oral interaction from
listeners is either highly limited or ruled out altogether.

4. Micro- and Macro of Speaking
 The micro-skills refer to producing the smaller
chunks of language such s phonemes, morphemes,
words, collocations, and phrasal units.
 The macro-skills imply the speaker’s focus on the
larger elements: fluency, discourse, function, style,
cohesion, nonverbal communication, and strategic
options.

5. A. Designing Assessment Tasks:
Imitative Speaking
1# Word and sentence repetition tasks (L,S)
 Test-takers hear: repeat after me:
Beat pause bit pause
Bat pause vat pause
I bought a boat yesterday
The glow of the candle is growing
When did they go on vacation?
Do you like coffee?
 Test-takers repeat the stimulus.

6. Scoring scale for repetition tasks:
2 acceptable pronunciation
1 comprehensible, partially, correct pronunciation
0 silence, seriously, incorrect pronunciation
The longer the stretch of language, the more possibility
for error and therefore the more difficult it becomes to
assign a point system to the text.

7. 2# Versant
 Formerly called PhonePass.
 Among a number of speaking tasks on the test,
repetition of sentences occupies a prominent role.
 Versant has supported the construct validity of its
repetition tasks not just for a test-taker’s
phonological ability but also for discourse and
overall oral production ability.

8. B. Designing Assessment Tasks:
Intensive Speaking
1# Directed Responsive Tasks
 The administrator elicits a particular grammatical form or a
transformation of a sentence, but they do require minimal
processing of meaning in order to produce the correct
grammatical output.
 Test-takers hear (L,S)
Tell me he went home.
Tell me that like rock music.
Tell me that you aren’t interested in tennis.

9. 2# Read-Aloud tasks
 Intensive read-aloud tasks include reading beyond the sentence
level up to paragraph or two.
 Teachers listening to the recording would then rate students on a
number of phonological factors (vowels, diphthongs, consonants,
stress, and intonation) by completing a two-page diagnostic
checklist on which all error or questionable items were noted.
Some variations on the task of simply reading a short passage:
 Reading a scripted dialogue
 Reading sentences containing minimal pairs
 Reading information from a table or chart

10. There are several drawbacks to using this techniques for
assessing oral production:
Reading aloud is somewhat inauthentic in that we
seldom read anything aloud to someone else in the real
world.
Reading aloud calls on certain specialized oral abilities
that may not indicate one’s pragmatic ability to
communicate orally in face-to-face context.

11. 3# Sentence/Dialogue Completion Tasks and
Oral Questionnaires
 Test-takers are first given time to read through to get its gist
and to think about appropriate lines to fill in. then as the
tape, teacher, or test administrator produces one part orally,
the test-takers responds:
 The advantages of this technique lies in its moderate control
of the output of the test-takers. On the other hand, this
techniques is its reliance on literacy and an ability to transfer
easily from written to spoken English. In addition, it is
contrived, inauthentic nature of this task.

12.  Test-takers see:
o Interviewer : what did you do last weekend?
o Test-taker :
o Interviewer : what did you do after you graduated from this
program?
o Test-taker :
o Interviewer :
o Test-taker : I was in japan for two weeks.
o Interviewer :
o Test-taker : it’s ten thirty.
 Test-takers respond with an appropriate lines.

13. 4# Picture-Cued Tasks
 It is more popular ways to elicit oral language performance at both
intensive and extensive levels. Its stimulus requires a description from
the test-takers.
 Picture-cued elicitation of minimal pairs
 Picture-cued elicitation of comparatives
 Picture-cued elicitation of future tense
 Picture-cued elicitation of nouns, negative responses, numbers, and
location
 Picture-cued elicitation of responses and description
 Picture-cued elicitation of giving directions
 Picture-cued elicitation of multiple-choice description for two test-
takers

14. Scoring scale for intensive tasks
2 Comprehensible; acceptable target form
1 Comprehensible; partially correct target form
0 Silence, or seriously incorrect target form
Evaluating interview could be used:
 Grammar
 Vocabulary
 Comprehensible
 Fluency
 Pronunciation
 Task (accomplishing the objective of the
elicited task)

15. 5# Translation (of Limited Stretches of
Discourse)
Translation is a meaningful communicative device in contexts
in which the English-user is called on to be an interpreter.
The test-taker is given a native-language word, phrase, or
sentence and is asked to translate it.

16. C. Designing Assessment Tasks:
Responsive Speaking
1# Question & Answer
 Question & answer can consist of one or two questions from an
interviewer or they can make up a portion of a whole battery
of questions and prompts in an oral interview.
 The first question is intensive in its purpose; it is a display
question intended to elicit a predetermined correct response.
 Questions at the responsive level tend to be genuine
referential questions in which the test-taker is given more
opportunity to produce meaningful language in response.

17. Responsive question may take following forms:
Questions eliciting open-ended responses
Test takers hear:
 1. what do you think about the weather today?
 2. why did you choose your academic major?
 What kind of strategies have you used to help you learn
English?
 Test-takers respond with a few sentences at most

18. 2# Giving Instructions and Directions
 The technique is simple: the administrator poses the problem, and test-
taker responds. Scoring is based primarily on comprehensibility, and
secondary on other specified grammatical or discourse categories. The
choice of topics needs to be familiar enough so that the test is not
general knowledge but linguistic competence. Finally, the task should
require the test-taker to produce at least five or six sentences.
 Eliciting instructions or direction
Test-takers hear:
 Describe how to make a typical dish from your country?
 How do you access e-mail on a PC?
Test-takers respond with appropriate instruction.

19. 3# Paraphrasing
 The test-takers read or hear a short story or
description with a limited number of sentences
(perhaps two or five) and produce a paraphrase
of the story. The advantages is they elicit short
stretches of output and perhaps tap into test-
takers’ to practice the conversational art of
conciseness by reducing the output/ input ratio.

20. 4# Test- of Spoken English (TSE Test)
 The Test of Spoken English are designed to elicit oral
production in various discourse categories rather than
in selected phonological, grammatical, or lexical
targets. Tasks include description, narration, summary,
giving instruction, comparing and contrasting.
 From their findings, the researchers were be able to
report on the validity of the tasks, especially the match
between the intended task functions and the actual
output of both native and non-active speakers.

21. D. Designing Assessment Tasks:
Interactive Speaking
#1 Interview
 A test administrator and a test-taker sit down in a direct face-to-face
exchange and proceed through a protocol of questions and
directives.
Four level stages:
1. warm-up, preliminary small talk to make test-taker become
comfortable with the situation. No scoring of this phase takes place.
2. level check, a series of preplanned questions.
3. probe, probe questions and prompts challenge test-takers to go to
the heights of their ability, to extend beyond the limits of the
interviewer’s expectation through increasingly difficult questions.
4. Wind-down, a final phase of interview. No scoring for this part.

22. The success of an oral interview will depend on:
 Clearly specifying administrative procedure of the
assessment. (practically)
 Focusing the questions and probes on the purpose (validity)
 Appropriately eliciting an optimal amount and quality of oral
production from the test-taker (biased for best performance)
 Minimizing the possibly harmful effect of the power
relationship between interviewer and interviewee (biased for
best performance)
 Creating a consistent, workable scoring system (reliability)

23. #2 Role Play
 It frees students to be somewhat creative in their
linguistic output. In some versions, role play allows
some rehearsal time so that students can map out
what they are going to say. It also has the effect of
lowering anxieties as students can, even for few
moments, take on the persona of someone other
than themselves.
 The test administrator must determine the assessment
objectives of the role play then devise a scoring
technique that appropriately pinpoints those
adjectives.

24. #3 Discussion and Conversation
Discussion may be especially appropriate tasks through which
elicit and observe such abilities:
 Topic nomination, maintenance, and termination
 Attention getting, interrupting, control
 Clarifying, questioning, paraphrasing
 Comprehension signals
 Negotiating meaning
 Intonation patterns for pragmatic effect
 Kinesics, eye contact, proxemics, body language
 Politeness, and other sociolinguistics factors

25. #4 Games
 Assessment games:
 Tinkertoy game
 Crossword puzzles
 Information gap
 City maps
 As assessments, the key is to specify a set of criteria and a
reasonably practical and reliable scoring method.

26. 5# ACTFL Oral Proficiency Interview (OPI)
 Originally known as the Foreign Service Institute (FSI) test.
 In a series of the structured tasks, the OPI is carefully designed to
elicit pronunciation, fluency and integrative ability, sociolinguistic
and cultural knowledge, grammar and vocabulary.
 Valdman (1988) summed up the complaint:
“The OPI forces test-takers into a closed system where, because the
interviewer in endowed with full social control, they are unable to
negotiate a social world….. In short, the OPI can only inform us how
learners can deal with an artificial social imposition rather than
enabling us to predict how they would be likely to manage
authentic linguistic interactions with target language native
speaker.”

27. E. Designing Assessment Tasks:
Extensive Speaking
1# Oral Presentations
The rules for effective assessment must be involved:
 Specify the criterion
 Set appropriate tasks
 Elicit optimal output
 Establish practical, reliable scoring process
 For oral presentation, a checklist or grid is a common means of
scoring or evaluation. The wash back effect of a such checklist can
be enhanced by written comments from the teacher, a conference
with the teacher, peer evaluation using the same form, and self-
assessment.

28. 2# Picture-Cued Storytelling
 It considers a picture or a series of pictures as a stimulus for a
longer story or description.
3# Retelling a Story, News Event [L, R, S]
 Test-takers hear or read a story or news event that they are asked
to retell.
 The objectives in assigning is listening comprehension of the
original to production of a number of oral discourse features
(sequences and relationship of events, stress and emphasis
pattern), fluency, and interaction with the hearer.
 Scoring should meet the intended criteria.

29. 4# Translation (of Extended Prose)
 The longer texts are presented for the test-taker to read in the
native language and then translate into English. Those texts could
come in many forms: dialogue, directions for assembly product,
a synopsis of a story, etc.
 The advantage: control the content, vocabulary, and the
grammatical and discourse features.
 Disadvantage: the translation of longer texts is highly specialized
skill for which some individuals obtain post-baccalaureate
degrees!
 Criteria of scoring should take into account not only the purpose
of stimulating a translation but the possibility of errors that are
unrelated to oral production ability.