Presented by Shaheen Miller, Administrative Coordinator, UCF College of Medicine At the UCF College of Medicine, we use ExamSoft Rubrics to assess first and second year medical students for our unique research program called FIRE (Focused Inquiry Research Experience). All students are required to complete a 2-year research project on a topic of their choice in the field of Medicine/Health. We use the ExamSoft Rubrics platform to assess students' papers in the first year and posters in the second year. Students receive formative (not graded) and summative (graded) feedback from peers, faculty, statistician, and librarian. This webinar will explore how to create and administer these assessments of student learning.

1. Thank you for joining.
The webinar will begin shortly.
Using Examsoft Rubrics to Assess
Student Medical Research
Shaheen Miller

2. Using Examsoft Rubrics to Assess
Student Medical Research

3. • We will discuss:
• Creating rubrics specific to reviewer expertise (Faculty, Peer,
Statistician, and Librarian)
• Creating the assessment, grader assignments, and creation of
anonymous reviewers
• Providing feedback to students and dealing with the
unexpected

4. You may be asking , how do I assess a Medical
Project on many elements that require specific
expertise and knowledge? Do I need to hire an
Einstein who can do it all or can I utilize the
resources (people) available to me? Do I need
to make multiple assignments?
4

5. EXAMSOFT RUBRICS!

6. At the UCF College of Medicine, we use ExamSoft Rubrics to assess first
and second year medical students for our unique research program
called FIRE (Focused Inquiry Research Experience). All students are
required to complete a 2-year research project on a topic of their choice
in the field of Medicine/Health. We use the ExamSoft Rubrics platform to
assess students' papers in the first year and posters in the second year.
Students receive formative (not graded) and summative (graded)
feedback from peers, faculty, statistician, and librarian. This webinar will
explore how to create and administer these assessments of student
learning

7. Lets Get Started!

8. Lets Login..
8
Sometimes an individual has trouble logging in because they are typing in their credentials in the Administrators/Faculty log-in instead of
the Exam Taker login (or vice versa). Checking to ensure the correct fields are filled out is a good first step before resetting the password.
Students log in here! They can view instructions, upload documents, peer grade, and access their
feedback from the student portal. They can click on “Lost Student ID or Password?” if they have
forgotten their password.
The (users) and faculty/librarians (graders) will log in
here. The administrator access to manage the
assignments, while the graders will only have the ability to
complete their assigned rubrics.

9. 9
Use these two tabs!

10. 10
Organize the rubrics into
folders. Rubrics can be
moved across folders to
keep organized.
Create a separate rubric for each type of grader, even if
the rubric itself is the same. This will allow students to be
able to distinguish between a peer’s versus a faculty
grader’s responses when viewing their feedback. The title
of the rubric will display on student reports.
Notice that this is a “Rubric Template”. You can create a template in
order to use it as the basis to create rubrics. A rubric template cannot
be used on an assessment.
Once a rubric has been used on an assessment, it can no longer be edited. A new rubric must be created. Because of this, it may be good to title rubrics with their
academic year, so to better organize the rubrics across years (e.g., 2016-17 FIRE-2 Summative Rubric – Faculty). The title of the rubric will display on student reports.
CREATING RUBRICS
This is the dashboard in the “Rubrics” tab. Rubrics will need to be created in this tab, and then linked to an assignment under the “Assignments” tab.

11. 11
Title the rubric the
name of specific type
of group of reviewers
or type of category that
will be assessed.
Ex: Statistics , Library ,
Faculty etc.
Adjust the rows and columns to match the size of the
Rubric you want to create ( Ex: 4 rows by 6 columns)
If you plan on creating a Rubric where you can assign
a point value to each column select “Points”.

12. 12
Creating Rubrics
If desired, the different
rows (dimensions)
could be assigned
different weights. This
will be reflected on the
student’s overall rubric
score on any
assignment this rubric
is attached to.

13. Assessments can be
moved across folders. A
possible folder structure
may be “A/Y Module”,
such as “2016-17 FIRE-
1”.
Select “Performance
Rubric”
Formative assessment? You can type in “0” points
here. (But it may be preferable to allow points, and just
inform students it’s a formative assignment. Removing
points takes scores away from the users as well.)Add rubrics, and re-order
them to the order you would
like them to appear to
students on their feedback
reports. Assign weights
across the rubrics based on
how their overall score
should be calculated.
13
Creating A New Assignment
Once you hit save, you will then
have the ability to “Post” the
assignment to the class.

14. 14
Posting An Assignment
Instructions are a document
that was previously built in
ExamSoft that students can
download through the portal.
If you would like students to
view the rubric(s) from a link
in the portal, you can pre-
release the rubric. Students will
receive an e-mail on the
“Release on” date/time that it
is available through the portal.
Upon hitting “Save Changes”, the assignment posts, and students receive an e-mail that the assignment is
available and their answer file is due by the date specified. To prevent an e-mail, check the “Suppress
Posting Emails” option. You can still manually send customized e-mails to the entire class or only select
students in the “Exam Taker Activity” tab, if desired. This will only suppress the auto-generated e-mails.
You can set the due date,
reminder date, and identify
if you will accept multiple
files or allow revisions.

15. 15
At the grading start time, all graders will receive an auto-e-mail that an assignment is ready for
grading. Set this date/time for the moment you want this e-mail to go out. Graders will be able
to grade and re-grade up until the grading end time.
Randomly or
manually assign
exam takers.
Check “Hide ET
Info from All
Graders”. Set a
reminder
date/time for the
system to send
this out.
Clicking on a student’s magnifying
glass will allow you to select who
that student will grade. (If a student
has already been assigned to
someone else for grading, s/he will
not be available to be selected again
if the settings to the left stipulate “1
Peer Grader per Exam Taker”.
This blue bar shows what % of students
have been assigned a grader.
AS S IGNING GRADERS

17. 17
?
?
?
?
?
One grader started grading,
so now grading assignments
cannot be assigned for any
other rubric.
ADDING GRADERS – S ET U P **PRIOR T O** T HE “GRADING S T ART ” T IMES
Important: This can be avoided by setting up all of the grader assignments prior to the “Grading Start” time for any one rubric. The
“Grading Start” time is when the system auto-sends an e-mail to the graders, notifying them that a rubric is now available for grading.

18. 18
Under the Reporting/Scoring tab, “Release Exam Taker
Results” will allow you to post the S&O reports to the
student portal (and send an e-mail that results are available, if
desired). The “Strengths and Opportunities” option will pull
the exact same report to PDF. One option releases the S&O
report to students, and one option opens S&O report to
PDF.
If you released the S&O report to the student portal, but it
was not customized as desired, you can fix the settings and
“Post to Exam Taker Portal” again, which will automatically
update what students see.
Customize this report to your
preference!
S T RENGT H AND OPPORT U NIT IES (S &O) REPORT

19. 19
S T RENGT H AND OPPORT U NIT IES (S &O) REPORT

20. 20
S T RENGT H AND OPPORT U NIT IES (S &O) REPORT Customize this report to your
preference! If you want it to
remain anonymous with detailed
comments check the include for
each section but leave “Grader
ID” check box blank.

22. 22

23. Contact Info:
Shaheen Miller
Office: 407-266-7102
Shaheen.Miller@ucf.edu

24. Thank You
Got Questions?