Moodle Quiz, Assignment, Workshop and our new pilot LTI ACJ tool

1. Tools for Online
Assessment
Craig Brown
Sarah Honeychurch
John Maguire

2. Moodle Quiz
Craig Brown

3. Why use Quizzes
• Immediate feedback
• Help student focus
• Give students time to act on feedback
• Not public to peers
• Multiple attempts
• Spot gaps in knowledge
• Save class time
• More individually focused
• Quickly gage progress

4. Tips for using quizzes
• Takes time so plan
• Make questions valid to the course
• Question banks
• Weight your questions
• Feedback for answers
• Review options
• Randomise
• Media and images
• Import question and reuse/repurpose

5. Quiz for formative assessment
• Test quiz
• One question per page
• Asynchronous
• Timing
• Attempts
• Security

6. Short-Answer
Pros
•Automatic marking
•Quick to write
Cons
•No real understanding required to answer

7. Essay question
Pros
•Gives opportunity to demonstrate deep knowledge
•Can help develop writing skills
Cons
•Time to mark
•Can result in poor quality writing if not properly timed

8. Drag and Drop
Pros
•Automatic marking
•Engaging
Cons
•Time
•Accessibility

9. Multiple choice questions (MCQ)
Pros
•Automatically marked
•Can cover a wide range of topics
Cons
•Time
•Can confuse students
•Students can ‘Guess’

10. Discussion
● When is it good to use a quiz?
● Do you think the quiz tool is a useful form of assessment?
● How successful do you think MCQ questions are?

11. Moodle Assignment
John Maguire

12. What is Moodle Assignment?
Assignment activity provides an area where students can submit work
electronically for teachers to grade and give feedback on.

13. Guides to assignment
Moodle.gla.ac.uk

14. Assignment
• File submissions
• Students can upload one or more files
• If pdf is uploaded it can be annotated in the browser.

15. Assignment
• Submission settings
• Submit button
• Important when using Urkund, and Group submission
• Submission statement
• Required for submissions (Senate Policy)
• Removes need for a paper copy

16. Assignment
• Group submissions
• Submit in groups
• Groups members will be able to see anything that is uploaded by others members
• 1 person submits, 1 submits others agree, all submit separate documents.
Ensure groups are setup before the assignment is created

17. Assignment
• Feedback types
• Comments
• Files
• Offline grading worksheet

18. Grading worksheet
CSV files do not retain formatting, important when using feedback comments
Enter grades in the Grade column
Ignore the Scale column, this simply displays the available scale

19. Assignment
• Blind marking
• Will hide the student’s name.
• If names are revealed they cannot be hidden again

20. Assignment
• Marker allocation
Can be used to allocate markers to specific submissions. This could be used
to ensure that subject specialists are assigned to mark questions about their
specialism.

21. Assignment
• Marking workflow
Marks will go through a series of workflow stages before being released to
students. This allows for multiple rounds of marking and allows marks to be
released to all students at the same time.

22. Assignment - Urkund
• Plagiarism detection
• Develop good academic practice
• Senate office policy (draft)
• Multiple or single submissions

23. Assignment - Urkund
• Plagiarism detection
•Develop good academic practice
• Senate office policy (draft)
• Multiple or single submissions
http://bit.ly/1TESSBQ

24. Discussion
What is an acceptable percentage when considering plagiarism reports?
Plagiarism or similarity checker?
http://www.gla.ac.uk/services/senateoffice/studentcodes/staff/plagiarism/plagiarismstatement/

25. Discussion
What is plagiarism?
A.Writing or copying a short piece from a source verbatim without stating who the original author is.
B.In collusion with your tutor/teacher, work further on a text that you have yourself written earlier.
C.Translate another author’s text into a different language and use the result without citing the source.
D.Make use of an existing text without stating who the original author is, but make small adjustments that alter the word sequence
and sentence structure, replace words with synonyms, remove or add individual words, and so on.
E.Reading several texts and reworking the contents of these into a new text that isn’t like any of the original sources other than a few
words or a sentence being the same, without citing any of the sources.
F.Transcribe a maximum 3-4 sentence-long text verbatim and state who the original author is in relation to the text.
G.Translate a text you have originated yourself into another language and make use of the result.
H.Build further on a text that has previously been graded without indicating which parts are old.

26. Online Peer Assessment & Peer Review
Sarah Honeychurch

27. Why use Peer Assessment/Review?
"..if we want students to develop critical thinking, judgement
and autonomy in assignment production they should be
provided with high-level evaluative experiences similar to
those of experts. Peer review, students evaluating and
commenting on each other's work, is one way to achieve
this.”
http://www.reap.ac.uk/PEER.aspx

28. Peer assessment or peer review?
• Definitions
– Assessment – gives grade
– Review – gives feedback but no grade
– Two aspects:
• Writing reviews of others’ work
• Receiving reviews on own work from peers
“… many students dislike being asked to mark other students work and receiving marks from other
students. There are many reasons for this, including a belief by students that their peers do not have the
expertise to mark reliably. In contrast, students are usually very positive about the benefits of peer
review processes where marking is not involved.”
http://www.reap.ac.uk/PEER/Research.aspx

29. Peer assessment?
• Benefits:
– Students (say that they) want more feedback
– Students (say that they) want prompt feedback
– Does not take (much) extra staff time
• Problems:
– Concerns about the reliability of student marking (quantity over
quality?)
– Students don’t like grading peers

30. Peer Review
• Benefits of receiving peer review
– Accessible language of peers
– Variety of feedback
– Conflict of feedback mirrors professional life (must process and decide what to accept/reject)
• Benefits of giving peer review
– Constructing feedback requires cognitive engagement
• Knowledge construction
• Engagement with assessment criteria
• Developing discipline specific writing skills
– Variety of approaches can stimulate self-reflection
• Issues
– Summative assessment?

31. Online Tools for peer
assessment/review
• Moodle Workshop
• New LTI tool: ACJ (Adaptive Comparative Judgement)
• Other options
– Moodle Wiki
– Moodle Forums

32. Moodle Workshop
• Similar to Moodle Assignment
• Options for self, peer and tutor review
• Options for review or assessment
• Option to assign tutor mark for submission and for review

33. Adaptive Comparative Judgement
• Thurstone (psychophysics); Pollitt
• Students are presented with (random) pairs of submissions and
judge better/worse
• Software uses iterative and adaptive algorithm to sort
submissions
• Staff can “seed” submissions to provide grade boundaries
Pollitt, A (2012) The method of Adaptive Comparative Judgement.
Assessment in Education: Principles, Policy & Practice. 19: 3, 1-
20. DOI:10.1080/0969594X.2012.665354

34. Adaptive Comparative Judgement
• Benefits
– Timely feedback
– Easier than peer assessment
– Little staff time
– Has inter and intra rater reliability
• Issues
– Trust in new system
– Complex criteria
– Rich feedback
http://www.cambridgeassessment.org.uk/Images/232694-investigating-the-
reliability-of-adaptive-comparative-judgment.pdf

35. Discussion
• Do you trust students to peer review responsibly?
– Why/not?
• Do you think peer assessment or review are preferable?
• What about ACJ
– Would you be interested in participating in a pilot next academic
session?

36. Feedback