An overview of the essential nine steps in the Quality Matters review process including members of the review team and basic forms.

1. The Quality Matters
Review Process
Presentation by Michael Wilder

2. Introduction
The Quality Matters review process:
• Involves nine major steps.
• Requires communication between Faculty
Developer, Master Reviewer, Institutional
Representative, and at least two Peers
Reviewers.
• Involves a multitude of online and digital
forms.

3. Overview
Steps in a QM Course Review
1.
2.
3.
Instructor worksheet submitted
Reviewer team selected
Pre-review
•
4.
5.
Course review
Final report submitted
•
•
6.
7.
8.
9.
Pre-review conference call
Post-review conference call
Post-review survey
Communication with faculty developer as needed
Course amendment form submitted
Changes reviewed/approved
Course recognized

4. Steps

5. Step 1: Instructor worksheet submitted
From the Course Review Management System
(CRMS), the faculty developer initiates the
process by submitting a Course Review
Application.

7. Step 1: Instructor worksheet submitted
As part of the online application process, the
faculty developer creates a “Course Worksheet”
(also known as “Faculty Developer Worksheet”).
This worksheet identifies such details as course
title and course-level learning objectives.

8. Step 1: Instructor worksheet submitted
The QM Review Coordinator provides the
Faculty Developer with instructions to complete
the Faculty Developer Worksheet.
Once the Faculty Developer Worksheet and the
Course Review Tracking Spreadsheet have been
successfully completed, the Review Coordinator
makes the course available for review in the
CRMS.

9. Step 2: Reviewer team selected
Successful submission of the Course Review
Application also alerts the QM Review
Coordinator who identifies the Master Reviewer.
If the course to be reviewed is initiated by a
subscriber institution, it is possible that the
Master Reviewer has already been selected.

10. Step 2: Reviewer team selected
The Quality Matters Review Coordinator sends
the Course Review Tracking Spreadsheet to the
Master Reviewer when he or she confirms
availability.

11. Step 2: Reviewer team selected
The Master Reviewer uses the “Peer Reviewer
Search Tool” (located at MyQM) to identify
potential members of the review team per QM
standards. Once reviewers confirm
availability, the Master Reviewer informs the
Review Coordinator via an updated Course
Review Tracking Spreadsheet.

13. Step 2: Reviewer team selected
• The Master Reviewer is responsible for choosing
a team on a QM-managed Review.
• Subscriber-managed reviews generally have
chosen the team ahead of time.
• In either case, the Master Reviewer will receive
access to the Instructor Worksheet, and should
retrieve the forms from MyQM in the Master
Reviewer area to set up and conduct the prereview conference call (next step).

14. Step 2: Reviewer team selected
The Master Reviewer lets the Institutional
Representative and the faculty developer know
who is on the review team and who is the
subject matter expert.
This information should be posted on the
spreadsheet and sent to each team member.

15. Step 3: Pre-review
Using the “Email Introduction Template” as a
guide, the Master Reviewer sends customized
“Review Assignment Instructions” to
reviewers, the faculty developer, and the
Institutional Representative. The “Team
Calendar,” “QM Principles,” “QM Roles,” and
“Writing Recommendations” documents are
included as attachments.

16. Step 3: Pre-review
The Faculty Developer (or Institutional
Representative) opens the online course to the
Peer Reviewers and provides login information
to the Master Reviewer. Directions and login
information to enter the course are sent to the
Peer Review team by the Master Reviewer.

17. Step 3: Pre-review
The Master Reviewer develops the Team
Calendar, setting timelines for the pre-review
conference call, active review, post-review, and
revision procedures. Generally the entire
process takes about twenty weeks. The Master
Reviewer sends a copy of the Team Calendar to
the Peer Review team, the Faculty
Developer, and the Institutional Representative.

18. Step 3: Pre-review
Using the “Pre Review Conference Call Agenda
Template” as a guide, the Master Reviewer
initiates contact with the Peer Review
team, communicates instructions and
responsibilities for the pre-review
conference, shares contact information, and
requests that the review team enter the course
to ensure no immediate navigation difficulties.

19. Step 3: Pre-review
Peer review team members review the Faculty
Developer Worksheet prior to the Pre-Review
Conference Call.

20. Step 3: Pre-review
The Master Reviewer convenes the pre-review
conference. At this conference, the Master
Reviewer describes the review
process, discusses the Team
Calendar, communicates time
expectations, identifies LMS access
concerns, elicits questions regarding the Faculty
Developer Worksheet, explains change
limitations, and discusses confidentiality.

21. 4. Course review
Using the Quality Matters Rubric and the Course
Review Management System, the Review Team
individually provides feedback and
recommendations on the Course Worksheet.

23. 4. Course review
The Master Reviewer will have access to review
course worksheets in progress. It is the Master
Reviewer’s responsibility to communicate with the
other reviewers if recommendations are not
meeting the criteria for helpful feedback.
Recommendations should be
constructive, specific, measurable, sensitive and
balanced. Master Reviewers should coach peer
reviewers if changes need to be made.

24. 5. Final report submission
Once Peer Reviewers complete all
recommendations, but before the final report is
submitted, the Master Reviewer conducts a
post-review discussion. The Peer Review team
discusses differences in scoring and any
remaining questions for the Faculty Developer.

25. 5. Final report submission
The Master Reviewer schedules a "Post-Review"
Conference Call so the reviewers can discuss "essential”
standards that did not meet expectations. The MR will
encourage the reviewers to be sure each
recommendation is clear and detailed enough that the
faculty developer/instructor knows what needs to be
done and encourage team members to include positive
comments. Team members can still change their decision
(met or not met) at this point on any of the standards.

26. 5. Final report submission
Individual reviews are revised based on team
discussion and finalized.
Peer Reviewers submit their revised report in
CRMS after editing for spelling and clarity.
Reviewers may edit, save and resubmit up until
the final report is submitted.
The Master Reviewer submits the Final Report in
the CRMS.

27. 5. Final report submission
Once the Master Reviewer submits the final
Reviewer Report Form, the CRMS automatically
notifies the Faculty Developer, IR and QM review
coordinator with the review outcome and provides
directions to the Faculty Developer for completing
the online Faculty Response Form.
If QM standards are met, the process moves to Step
6.

28. 5. Final report submission
If QM standards are not yet met, the Faculty Developer
receives the outcome email instructing them to view the
report and submit the online Faculty Response Form within
two weeks of the final report submission indicating if changes
will be made to the course.
The submission of the Faculty Response Form will generate a
Course Amendment Form in the CRMS.
The CRMS will determine the amendment due date
depending upon the review start date. The re-review needs
to be completed within the 20-week window from review
start date unless the Institutional Representative petitions
Quality Matters for an extension to the amendment date.

29. 5. Final report submission
The Faculty Developer completes the online
Course Amendment Form to record the changes
he or she made in the course in order for it to
now meet expectations. Upon submission, the
Master Reviewer will be automatically notified
by email that the amendments are complete.

30. 5. Final report submission
The CRMS provides directions to the Faculty
Developer for completing the online Faculty
Response Form.
All reviewers, the faculty developer, and the
institution representative are asked to take a postreview survey. The CRMS will automatically send a
link to this survey once the final report is
submitted.
If the course has met QM standards, the process
moves to Step 9.

31. 6. Communication with faculty developer as
needed
The Institution Representative follows up with
the Faculty Developer to provide instructional
design resources and to ensure that the
amendment deadline is met.

32. 7. Course amendment form submitted
Once the amendments to the course are
completed, the Faculty Developer submits the
online Amendment Form in the CRMS.

33. 8. Changes reviewed/approved
The Master Reviewer reviews the changes in the course
and approves the online Amendment Form if the course
now meets expectations. Once amendments are
approved and submitted, the CRMS will notify the Faculty
Developer, IR and QM review Coordinator that the course
now meets standards (if applicable).
The Master Reviewer sends the completed Course Review
Tracking Spreadsheet to the QM Review Coordinator at
the close of the review.

34. 8. Changes reviewed/approved
If the course does not meet QM standards after
amendment, the Master Reviewer does not approve the
online Amendment Form.
The CRMS automatically notifies the Faculty Developer, IR and
QM review Coordinator that the course does not meets
standards.
The Master Reviewer sends the completed Course Review
Tracking Spreadsheet to the QM Review Coordinator at the
close of the review.

35. Step 9: Course recognized
Once the course meets
Quality Masters
Standards, it is formally
recognized in the QM
CRMS.

36. Questions?

37. This presentation made by:
Michael Wilder
Instructional Design Coordinator
E-mail | Web Site