1. Designing and Developing Online Course Assessments Day 1: Online Strategies for Assessment Design Dr. Veronica Diaz, Maricopa Community Colleges Dr. Patricia McGee, The University of Texas at San Antonio
2.
3. What are the foundations of online course assessment?
23. Transfer of learning Objectives? Outcomes? Monitor and Adjust What Students Know transfer of learning What students can do Facts Procedures Principles Strategies Critical thinking Problem finding/solving Creative thinking Typically measured by objective assessments Typically measured by performance assessments Homework, quizzes, tests, exams Projects, cases, problems, designs, experiments
Patricia – Assessment is aligned with evaluation process but serve different purposes. While assessment measures the achievement and progress of the individual, evaluation indicates the successful implementation of a course of instruction and how learning is transferred to other contexts.
Patricia – Assessments are conducted for a variety of purposes. In an online environment these are particularly crucial since evidence may not be readily available or evident to the course instructor.
Patricia
Patricia - Valid: directly reflects the learning outcome being assessed Reliable: inter-rater reliability when subjective judgments are made Actionable: results help faculty identify what students are learning well and what requires more attention Efficient and cost-effective in time and money Engaging to respondents, so they’ll demonstrate the extent of their learning Interesting to stakeholders, care about results and are willing to act on them Triangulation: multiple lines of evidence point to the same conclusion
Patricia Match learning objectives with assessments Take precautions to limit the possibility of cheating Communicate assessment tasks clearly Use formative assessment to promote deeper learning; consider alternative forms of assessment such as portfolios Use self-assessments to improve learning and self-awareness Have students conduct peer-assessments (may be particularly effective when used in conjunction with group work) Multiple sources including http://www.sjc.cc.nm.us/pages/2852.asp
Patricia Match learning objectives with assessments Take precautions to limit the possibility of cheating Communicate assessment tasks clearly Use formative assessment to promote deeper learning; consider alternative forms of assessment such as portfolios Use self-assessments to improve learning and self-awareness Have students conduct peer-assessments (may be particularly effective when used in conjunction with group work) Multiple sources including http://www.sjc.cc.nm.us/pages/2852.asp
Patricia
Patricia Performance feedback is challenging in an online environment because performance must be re-defined. Whether performance involved actions and productions that can be easily relayed online, feedback is key to the learner understanding “how they did.” Depending upon the type of course and the numbers of students there are a variety of strategies to provide feedback.
Patricia
Patricia Evaluations can inform you if the assessment are helping students understand what they are accomplishing. They can also identify other barriers or ‘invisible’ problems.
Patricia
Patricia
Patricia -When considering differences between face-to-face and online contexts for learning, several factors are in critical when designing, developing, and implementing online assessments. In a campus classroom the instructor is typically delivering instruction to a group of students, while similar in an online classroom, the students are separated and are realistically in a classroom on one. Students can ‘feel’ that instruction is one-on-one and the instructor is much more in tune with what they are experiencing. Students can rely on the instructor to keep track of everything they do, while in a classroom the instructor is more group focused. In a classroom assessments are scheduled whereas in an online classroom, assessments may be needed if there are disparate responses, lack of achievement, etc. In a classroom the instructor knows and seeks out ways of assessing student understanding in real time through multiple modalities and strategies. In a.n online class there is much ambiguity about understanding. In a classroom courses are typically designed around teaching methods: lecture, presentation, research, etc. In an online course, content is typically chunked into segments where the focus is more strategy-based: discussions, chats, group work, etc.
When considering differences between face-to-face and online contexts for learning, several factors are in critical when designing, developing, and implementing online assessments. In a campus classroom the instructor is typically delivering instruction to a group of students, while similar in an online classroom, the students are separated and are realistically in a classroom on one. Students can ‘feel’ that instruction is one-on-one and the instructor is much more in tune with what they are experiencing. Students can rely on the instructor to keep track of everything they do, while in a classroom the instructor is more group focused. In a classroom assessments are scheduled whereas in an online classroom, assessments may be needed if there are disparate responses, lack of achievement, etc. In a classroom the instructor knows and seeks out ways of assessing student understanding in real time through multiple modalities and strategies. In a.n online class there is much ambiguity about understanding. In a classroom courses are typically designed around teaching methods: lecture, presentation, research, etc. In an online course, content is typically chunked into segments where the focus is more strategy-based: discussions, chats, group work, etc.
Patricia
Patricia
Patricia - In thinking about assessment we need to start by thinking about what we expect students to know. Undergraduate students particularly are focused on the acquisition of foundational knowledge that are typically measured by objective assessments which work fairly well in an online environment. If a course is designed to have the learner transfer what they have learned into application, then performance measures and assessments must be designed to allow them authentic ways to perform. In a traditional classroom, I would argue, performance assessments are easier to implement because you have control over the environment and variables, and you can see the processes students use. In an online course, these processes are often invisible and if students are novices to higher education or distance learning, then their lack of sophistication and expertise may present challenges.
Patricia – progressive assessment is important in online contexts because the student may not always be aware of where they stand in accomplishing course objectives. In a ftf classroom, teachers typically view progressive assessment as a series of learning events, collection and interpretation of evidence judged against pre-determined criteria, culminating in a report. References Caldwell, B. & Carter, E. M. A. (1993). The return of the mentor: Strategies for workplace learning . Routledge Gardner, J. (2006). Assessment and learning. SAGE.
Patricia- Progressive assessment can also be viewed as a series of events. Here we may start with goals from which student either provide or generate evidence before or during a class. Then students and instructor determine criteria are used to judge achievement. The outcome of this judgment results in next steps for learning References Caldwell, B. & Carter, E. M. A. (1993). The return of the mentor: Strategies for workplace learning . Routledge Gardner, J. (2006). Assessment and learning. SAGE. .
Patricia – in an online environment, progressive assessment in learner-centered; learners keep track, participant in, discuss, and communicate their understanding, mastery, deficits, etc. with each other and the instructor.
Patricia – In an online environment, progressive assessment allows the learner to document and recognize their achievement of learning objectives
Patricia
Patricia
Patricia
Patricia
Patricia
Patricia - The relationship between formative and summative assessment can be viewed in several ways. Some course designs support a semester long progression of assignments the lead to a final, summative product, project, or final evaluative event – an exam, performance, presentation, etc.. In this model, formative assessments determine whether the learner has satisfactory met the criteria and is prepared to proceed.
Patricia - An example of such an approach is a writing or research-based course.
Patricia - Another approach is to have periodic assessment that measure student learning but do not indicate a progression towards a final summative event. In this design students can proceed through the materials once they have completed the assessment.
Patricia - One example of this approach in an online course design is where students take different approached to the assignment and proceed through activities without intervention if they haven’t mastered content.
Patricia
Veronica
Veronica
Do in chat
Veronica
Veronica
Veronica - Note: if objectives are poorly written, the assessment will be difficult to construct
Veronica
Veronica - Let’s go through each and in the chat make suggestions on how it could be improved, then suggest an an assessment tool that could be used to assess
Veronica
Veronica
Veronica
Veronica
Veronica - Alignment
Veronica
Veronica
Veronica
Veronica
Veronica
Patricia and Veronica
Patricia
Veronica
Veronica
Veronica
Veronica
Veronica
Veronica- Students or others report their perception of how well a given learning outcome has been achieved
Veronica
Veronica
Veronica
Patricia - handout
Patricia – Selecting criteria is important for either framework
Patricia – Selecting criteria is important for either framework
Patricia
Patricia Look at the objective (s) Think about the criteria for assessment