Group Presentation TBLA PDF: https://docs.google.com/fileview?id=0ByQvYzklJ2IuNzUwMWQ2YWQtOTE2NC00MTRmLTliMDItOTFkYWMwNzZmNDRm&hl=en Test: https://docs.google.com/fileview?id=0ByQvYzklJ2IuYTU1NzY2N2MtYzZkMS00MDQ5LTk5OWUtMDkxN2MxMzI3ZjA1&hl=en Listening comprehension: http://www.lancasterlinguistics.org/mod/forum/discuss.php?d=794

1. Task-basedLanguageAssessment(TBLA) A presentation by Ulrike Schuh-Fricke & Wolfgang Stadler 1

2. Group Presentation (7.2) Consists of A PowerPoint in which we, among others, Explain the thinking behind our test design Say what tasks the students will have to do Present and describe our assessment instrument Explain why it is task-based and how it relates to the textbook Address concerns raised in the literature on TBLA Reflect on how task-based tests relate to the issues of generalisability, task difficulty, and task variability A task-based language test for a group of students who have done unit 1 of the course book Widgets 2

3. Structure Background (Slide 4) Task-based language teaching (Slides 5–7) Course book Widgets, Unit 1 (Slides 8, 9) Tasks to do on Threshold Level (=B1, CEF) (Slides 10, 11) Concept behind performance tasks (Slide 12) Tasks achieved after Unit 1, Widgets (Slide 13) Task-based language testing (Slides 14–16) Performance (Assessment) Criteria (Slides 17–21) The Test (Slides 22–25) Critical reflection (Slide 26) Literature (Slide 27) 3

4. Background Communicative Language Ability (CLA): Focus on use of language in various situations (see David’s comment in 7.1) Action-orientedapproach: users and learners of a language are primarily ‘social agents’, i.e. members of society who have tasks to accomplish in a given set of circumstances, in a specific environment and within a particular field of action. (CEFR, ch. 2.1) Focus on the agents’ use of strategies linked to their competences and how they perceive or imagine the situation to be mastered with their language skills In accordance with the action-oriented approach, it is assumed that the language learner is in the process of becoming a plurilinguallanguage user developing interculturality(ibid.) 4

5. Task-based language teaching Tasks are a feature of everyday life in the personal, public, educational or occupational domains. A task may be quite simple or extremely complex and it may involve a greater or lesser number of steps or embedded sub-tasks. Tasks can be extremely varied in nature, and may involve language activities to a greater or lesser extent, for example: creative, skills based, problem solving, routine transactions, interpreting a role in a play, taking part in a discussion, giving a presentation, planning a course of action, reading and replying to (an e-mail) message, etc. Task accomplishment involves the strategic activation of specific competences in order to carry out a set of purposeful actions in a particular domain with a clearly defined goal and a specific outcome. (CEFR, ch. 7.1) 5

6. What is a language task? (Definition) “A task [is] any activity in which a person engages, given an appropriate setting, in order to achieve a specifiable class of objectives.“ (J.B. Carroll, 1993) According to CEFR, a task is defined as any purposeful action considered by an individual as necessary in order to achieve a given result in the context of a problem to be solved, an obligation to fulfil or an objective to be achieved (CEFR, ch. 2.1) Tasks are closely associated with specific situations are goal-oriented involve active participation of the language users 6

7. Developing a task (Checklist) According to Bachman/Palmer’s framework (1996), five aspects need to be considered when developing a task (checklist): Setting (physical characteristics, participants, time) Test rubric (instructions, structure, time allotment, scoring method) Input (format, language) Expected response (format, language) Relationship between input and response (reactivity, scope of relationship, directness of relationship) 7

8. Introducing Widgets Widgets – designed for an ideal class of 16-24 students – creates a “real-life” English-speaking environment in the classroom and focuses mainly on listening and speaking skills. Students imagine that they are new employees at an international company, Widgets Inc., where they must work together in small groups to perform various practical, yet fun, tasks. http://www.widgets-inc.com/index.php 8

9. CoursebookWidgets, Unit 1 In Unit 1 students learn about the course concept and objectives learn about Widgets Inc. (the company) explore customs about business cards, name tags and handshaking meet and greet each other in a casual business setting learn about Victoria Vanderhoff, the company’s president familiarize themselves with appropriate conversation topics 9

10. Tasks to do on Threshold Level(=B1, CEFR) Personal identification Learners can say who they are, spell their name, state their address, give their telephone number, say when and where they were born, state their age, sex, state whether they are married or not, state their nationality, say where they are from, what they do for a living, describe their family, state their religion, if any, state their likes and dislikes, say what other people are like; elicit/understand similar information from others. 10

11. Tasks to do on Threshold Level(=B1, CEFR), cont. Communication at work Seek work permits etc. as required; Enquire (e.g. from employment agencies) about the nature, availability and conditions of employment (e.g. job description, pay, laws of work, free time and holidays, length of notice); read employment advertisements; Write letters of application and attend interviews giving written or spoken information about own personal data, qualifications and experience and answer questions about them; Understand and follow joining procedures; Understand and ask questions concerning the tasks to be performed on starting work; Understand safety and security regulations and instructions; Report an accident and make an insurance claim; Make use of welfare facilities; Communicate appropriately with superiors, colleagues and subordinates; 11

12. Concept behind performance tasks Students perform authentic, real-life tasks, using the productive skills of speaking and/or writing but also combining skills. Teachers elicit authentic communication and may predict students’ future performances in target language use situations. If performance assessment is closely linked to the curriculum, a strong positive washbackeffect may be achieved. 12

13. What students can do afterUnit 1, Widgets Read and understand a letter of employment View an orientation video of the company and take notes Distinguish between business cards and name tags and see their purpose Make their own business cards and name tags Introduce themselves and shake hands (culture specific elements) Find missing information (in a CV) Distinguish between appropriate and inappropriate small talk questions and how difficult that can be depending on the speakers and the situation (sociolinguistic appropriateness) Listen in on a private conversation and enter a dialogue Lead a more private conversation (pragmatic competence: how to begin, interrupt, keep up, end a conversation) Describe, explain and sell a product Listen to a presentation and find out how a department works Describe one’s character 13

15. Authenticity of task

16. Set of criteria to judge the performance14

17. Task-based assessment forms 15

20. Decision on successful achievement of task

21. Integrated tasks

22. Task difficulty

23. Task characteristics’ impact on measures of fluency, complexity and accuracy

24. Rater training16

25. Performance (Assessment) Criteria A performance test is a ”test in which the ability of candidates to perform particular tasks, usually associated with job or study requirements, is assessed“ (Davies et al., 1999) “< … > the target language is used by the learner for a communicative purpose (goal) in order to achieve an outcome” (Willis, 1996) 17

26. Our selected set of criteria We opt for two steps of assessing tasks Although meaning is primary learners should not be too restricted in their use of language forms tasks should bear a relationship to real-world activities the priority is on achieving the goal of the task tasks are assessed based on their outcome (see http://www.widgets-inc.com/teacher/tblt.php) we want to assess language as well. 18

27. Two steps of assessment 19

28. Descriptors for a Pass 20

29. Descriptors for a Fail 21

30. The Test See attachment (PDF) It includes the task-based test, the listening tasks (recorded by Ulrike) and the Widgets rating scales. See the links in the description 22

31. Structure of the test 23

32. Test and tasks The test is a “task-based achievement test” based on similar tasks in the course book Tasks are authentic and goal oriented they allow for meaningful interaction between test taker and task test taker and interlocutor (simulation) test takers (role play) 24

33. Discussion of assessment criteriaand test design The main assumption of the rating process and the scale is: “… if an appropriate outcome was achieved, then we already know that the student’s pronunciation, vocabulary and grammar are at an “acceptable” level for the task, communicatively speaking. They would not have managed to complete the task otherwise.” Problems: What is meant by “appropriate outcome”? Can “task appropriacy” be measured and rated? If so, how? (see Ben’s thoughts on this in 7.1) 25

34. Critical reflection Are learners aware of what they are learning: general English or English for specific purposes? Do they have to be aware of this? Are learners able to transfer their competence and use of skills and strategies to new tasks and real-life situations? How will students who are used to task-based language learning perform in non task-based (proficiency) tests? Real-life tasks seem rather specific, whereas the language used can be fairly general or practical. Can task difficulty drastically be altered by changing task specifics (e.g. turn a face-to-face dialogue into a telephone conversation; change role of interlocutor) and how does this affect the outcome? Who chooses tasks and who decides whether the tasks have been successfully solved? Teachers or professionals? Rater training seems vital. Content validity: How to guarantee that tasks are representative of the TLU domain? Face validity and predictive validity seem to be crucial in TBLA (see also Ben in 7.1) . How can test developers tackle construct validity and its effect on rating scales? What about task-based interculturality in a multi-migrant classroom? (cf. Kate in 7.1) 26

35. Literature Bachman, L.F. & Palmer, A.S. (1996). Language testing in practice. Oxford: Oxford University Press (Chapter 3: Describing tasks: Language use in language tests) Brown, J. D., & Hudson, T. (1998). The alternatives in language assessment. TESOL Quarterly 32 (4), pp. 653-675. Colprin, M. & Gysen, S (2006). Developing and introducing task-based language tests. In K. Van den Branden (Ed.), Task-based language education – From theory to practice, Cambridge, pp. 152 -153. Common European Framework of Reference (CEFR). Eisenmann, M. (2009). FörderungderSprachproduktion. In Praxis Fremd-sprachenunterricht 05/09, pp. 4-8. Nunan, D. (2004). Task-based language teaching, Cambridge, pp.145-146. Wigglesworth, G. (2008). Task and performance based assessment. In S. Shohamy & N. H. Hornberger (Eds.), Encyclopedia of language and education (2nd Ed.), Volume 7: Language testing and assessment, pp.111-122. Widgets. A task-based course in practical English (Coursebook) Willis, J. (1996): A framework for task-based learning. Harlow: Longman. 27

36. © 2009 W. Stadler, U. Schuh-Fricke Thank you for your attention  28