1. Tests & Quizzes (aka SAMigo)
Christine Doherty, Stanford University

2. Agenda
• Feature Overview
• Demo of Features
• 2.6 Highlights
• 2.7 Roadmap
• Q&A
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 2

3. What are SAMigo’s Features:
• Assessment Authoring
• Assessment Publishing
• Assessment-Taking Management
• Assessment Grading
• Question Pools Management
• Assessment-Type Management
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 3

4. Pedagogical Flexibility
• Online test
• Self-study questions
• Homework, problem-sets
• Paper, code, or project submission
• Language drills
• Quick knowledge probes
• Surveys
• Others???
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 4

5. Organize & Publish Assessments
• Pending* Assessments * New in 2.6
• Starting page for assessment authoring
• Quick Create feature available OOTB *
• Can be exported/imported in QTI XML format
• Can create copies easily *
• When ready, published via Settings
• Published Assessments for student assessment-taking
• When students respond, Scores link appears
• Can edit published assessments and some settings *
• Can remove published assessments
• Last Modified column with name/date/time *
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 5

7. * New in 2.6

8. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 8

9. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 9

10. Question Types
Auto-scored
• Multiple Choice (single and multiple correct)
• Survey (multiple choice, different rating scales available)
• Fill in the Blank
• Numeric Response
• Matching
• True/False
Manually scored
• Short Answer/Essay
• Audio Recording
• File Upload
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 10

11. x

13. What’s New in Question Types
• Multiple Choice: Multiple Correct, Single Selection
• Full credit if student picks any of the possible correct answers
• Multiple Choice/Single Correct, True/False: Negative
Point Value for Incorrect Answer Selection
• Subtracts designated point value from total score
• Discourages random guessing
• Reset Selection option in delivery for students to clear radio
button selection
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 13

14. What’s New in Question Types
• Audio Recording: Improved Recording Applet
• Ability to display multiple audio recording questions on a page
without performance problems; link pops up applet in separate
window
• Save/Close button in applet for improved usability
• Improved recording length reliably saved (300 sec)
• Improved processing time to save recording
• Volume meter bar to provide live feedback
• Aesthetic enhancements (applet size, text size)
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 14

15. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 15

16. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 16

17. Rich Text & Attachments
• At all levels of assessment authoring:
• Assessment/Part Headers
• Questions
• Selections
• Feedback
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 17

18. Create an Assessment
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 18

23. * New in 2.6

47. Quick Create
• Contributed by Claremont McKenna College and UC Davis
• Available in 2.6 OOTB
• Efficiency created by reducing number of clicks, pages
loading and saving
• Useful for transferring assessments created in other
applications or LMS (e.g., MS Word, WebCT), no need to
paste into different fields
• Supports Multiple Choice, Fill in the Blank, Short
Answer/Essay, and True/False question types
• Rich text, attachments, and additional questions can be
added later, through assessment authoring page
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 47

53. Import/Export
• Supports QTI v 1.2
• Export types: IMS QTI-compliant XML or IMS
Content Packaging (for use with attachments)
• Useful for:
• Copying assessment to different site
• Archiving assessments offline
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 53

60. Settings
• Configures delivery of assessments (how they appear
to students)
• Options include:
• display/organization
• late handling
• number of submissions
• feedback/grading options
• high security and time limit for high stakes testing
• Some settings are not editable after publishing, so it
is very important for instructors to review these
carefully before publishing
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 60

64. What’s New in Settings
• Option to release assessment to one or more
groups/sections
• groups/sections need to be defined in Section Info or Site
Info > Manage Groups
• Mark for Review is configurable per assessment;
deselected by default
• Option to auto-submit saved assessments after due
date
• needs to be enabled in sakai.properties
(samigo.autoSubmit.enabled=true)
• quartz job needs to be created in Administration Workspace
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 64

65. * New in 2.6

66. * New in 2.6

67. * New in 2.6
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 67

68. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 68

69. Save Settings and Publish - to release to students
Save Settings - to make changes to pending assessment settings,
can publish assessment at later time

71. Publishing
• Release assessment to students
• Copy of pending assessment – pending assessment
can be modified without affecting published version
• Can set Available Date in future -- useful for
instructors who want to do bulk of setup early
• Can retract any time, making assessment
unavailable to all students
• Can remove published assessment
• Can edit published assessment *New in 2.6
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 71

72. Edit a Published Assessment (OOTB)
• OOTB 2.6 allows editing a published assessment
ONLY IF no students have started taking it
• Includes editing content of existing questions and
point values
• Does not include editing parts, reordering parts or
questions, adding or removing questions
• More editable settings than 2.5: Delivery Dates,
Timed Assessment, Assessment Organization,
Submissions, Feedback, Grading
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 72

73. Edit a Published Assessment (full)
• Option to edit a published assessment AFTER
students have started taking it
• needs to be enabled in sakai.properties
(samigo.editPubAssessment.restricted=false)
• Ability to regrade auto-scored questions for existing
submissions
• Ability to allow students to update their latest
submission
• Warning messages for students to notify them of
modified assessments
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 73

75. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 75

77. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 77

78. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 78

79. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 79

80. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 80

81. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 81

82. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 82

83. Published Inactive Assessments
• Retracted assessments will not appear to any
students
• Assessments that are past the due date will not
appear to students, with following exception:
• When late submission is allowed, students who have
not submitted will get “one last chance” to submit
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 83

85. Taking an Assessment (Students)
• Take an Assessment section
• List determined by available, due, and retract dates,
plus number of allowed submissions
• Submit for Grading and Save for Later options
• Table of Contents and Mark for Review features
available in non-linear assessment organization
• Submitted Assessments section
• Provides submission confirmation with timestamp
• Depending on settings, students can review scores,
feedback, grader’s comments, statistics
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 85

86. What’s New in Assessment Delivery
• Save for Later confirmation screen
• Submit for Grading confirmation screen
• Show anonymous users their test results
immediately after they submit assessment
• Disable Save for Later button for timed
assessment
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 86

92. * New in 2.6
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 92

95. * New in 2.6
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 95

99. Grading Workflows
• Ability to grade all questions for one student
or grade all responses to one question
• Manual score adjustments can be made to
individual questions or entire assessment
• Comments can be made for individual
questions or entire assessment
• Display of scores can be filtered by section
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 99

100. Other Grading Features
• Email students from Scores page
• Allow Retake for individual student
• Download All function for file upload
questions -- generates zip file of all student
submissions
• Statistics and histograms
• Export students' responses to Excel file
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 100

101. What’s New in Grading Features
• Download All function for file upload
questions – download by section/group
• Improved statistics – Item Analysis
• Export part scores as additional columns in
Excel spreadsheet
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 101

102. Grading Settings
• Anonymous grading OR show student
identities
• Send scores to Gradebook OR not
• Record highest score OR last score
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 102

111. xxxxxxx

112. xxxxxxx

113. xxxxxxx

114. * New in 2.6
xxxxxxx

116. * New in 2.6
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 116

122. Question Pools
• Organize questions into multiple subpools
• Move/copy/remove multiple questions at a time
• Author questions from within Pool Manager
• Import questions from IMS QTI-compliant XML file
• Randomly draw from pools for an assessment
• Question pools belong to an individual user, available
across all user’s sites
• Ability to share question pools with other site
members *New in 2.6
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 122

125. * New in 2.6

126. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 126

127. July 2009 10th Sakai Conference - Boston, MA, U.S.A. 127

128. Assessment Types
• Create assessment “types” for different
disciplines, different uses
• Hide complexity by selecting which settings
are visible to the instructor.
• Pre-set defaults by making setting choices for
the type (e.g., Self-study always has
immediate feedback)
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 128

129. Assessment Creation Workflow
July 2009 10th Sakai Conference - Boston, MA, U.S.A.

135. 2.6 Highlights
• Edit a published assessment
• Includes editing content of existing questions and most settings
• Does not include editing parts, reordering parts or questions, adding or
removing questions
• Ability to regrade auto-scored questions for existing submissions
• Copy a pending assessment
• Question Type enhancements:
• Multiple Choice/Multiple Correct/Single Selection
• Negative Point Value for Incorrect Answer Selection*
• Improved Recording Applet
• Submit for Grading*** and Save for Later confirmation screens
• Share question pools *
• Release assessment to one or more groups/sections **
• Statistics for item analysis / Export part scores **
*Contributed by Universidad Politecnica de Valencia
** Contributed by University of Cape Town *** Contributed by Rutgers University

136. 2.7 Roadmap
• Auto-Save and Save (on demand)*
• Auto-save interval configurable by system admin
• Save button allows students to save work without leaving page
• Significantly reduces loss of data
• Reconfiguration/Renaming of navigation buttons in delivery
• Submit for Grading button position changed
• Save for Later changed to Exit
• Email Notification when publishing/republishing assessments
• Respondus integration
• Print assessment**
* Based on contributions from Rutgers University
** Based on contributions from Universidad Politecnica de Valencia

137. Questions?
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 137

138. Contacts
Questions and feedback: samigo-dev@lists.stanford.edu
Lydia Li, Stanford University
Project Manager
lydial@stanford.edu
Homepage:
http://confluence.sakaiproject.org/confluence/display/SAM
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 138