1. Solving the OSP
Reporting Conundrum
Lynn Ward, Principal Systems Analyst
Indiana University

2. Indiana University
• 7 Campuses • Stats
• 2 large research • > 1K degree programs
campuses • > 5K FT, 2K PT Faculty
(Bloomington and > 11K staff
•
IUPUI)
• > 100K students
• 5 smaller regional
campuses (East, Fort
Wayne, Kokomo, • Sakai is our only CMS/
Northwest, South LMS
Bend, Southeast)
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 2

3. ePortfolio Timeline @ IU
• 1998: IUPUI campus adopts a competency or
ability-based approach to general education
• 2000: IUPUI campus initiates Eport project,
partnering with Ali Jafari (Epsilen)
• 2003: IU and rSmart partnership on OSP 2.0 Mellon
Grant; IUPUI abandons first homegrown tool
• 2004-05: early pilots with OSP
• 2006: moratorium on new portfolio projects
• 2007-present: growing interest and use on all
campuses
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 3

4. OSP Use @IU
• > 20 active portfolio projects underway,
many in professional schools
• Majority focus on outcomes assessment at
the program level
• Highly individualized approach to
assessment
• Individual differences accommodated via
customized scaffoldings and forms
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 4

5. OSP Support @ IU
• .5 – 2 FTE developers
• .10-20 FTE designer
• .25 FTE combined consultant, analyst, tech
writer, trainer, QA specialist
• Outsource some QA and development
work
• Growing knowledge among consultants in
campus teaching centers
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 5

6. The Conundrum
• Project coordinators need to aggregate,
analyze, and summarize evaluation (and other
types of) data
• Evaluation forms differ widely in terms of
• # of rating fields
• # rating scale
• # labels assigned to each point on scale
• XML Programmer required to develop custom
reports for each project – DOES NOT SCALE
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 6

7. Our Solution
• Define local form standards
• Flexible
• Predictable
• Publish and promote standards
• Define report definitions based on those
standards
• Reports “discover” the XSD structure on
the fly and render the results
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 7

8. Evaluation Form Standard
• Field name conventions:
• Rating fields (rating, rating 1, rating2, rating3 …
rating20)
• Comments (comments)
• Rating fields must be represented as a selection
field (drop-down menu)
• All rating fields in the form (except rating) must
use the same rating scale.
• Enumeration values must be a number, “n/a”, or
null
• Rating fields may not be repeatable
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 8

9. Customization Options
• Rating fields and comments fields may use any
display name
• Rating scales may have any number of points
• The display label for each point on the scale
may be any valid string
• Evaluation forms may have any valid name
• Each matrix cell may use a different evaluation
form.
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 9

10. SAMPLE REPORTS
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 10

11. Rating Summary
10th Sakai Conference - Boston, MA, U.S.A. 11

12. Rating Details
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 12

13. Interactive Report – Main (Matrix) View
Click to see
summary report
for cell
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 13

14. Interactive Report – Summary
View
Click to return to
main (matrix)
view.
Click to see
detailed report for
cell
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 14

15. Interactive Report – Details View
Click to return to
main (matrix) view.
Click to return to
summary view
Click to open cell
in new window.
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 15

16. Matrix Cell Status Summary
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 16

17. Matrix Cell Status Details
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 17

18. Matrix Link Summary
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 18

19. Next Steps
• Publish form templates and documentation
on how to customize
• Refine and augment existing reports based on
user feedback.
• Merge Reports tool enhancements to trunk
• Post report definitions to OSP library so other
institutions can use them.
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 19

20. For More Information
• Contact…
– Lynn Ward (leward@iupui.edu)
– Chris Maurer (chmaurer@iupui.edu)
– Bryan Holladay (bahollad@iupui.edu)
July 2009 10th Sakai Conference - Boston, MA, U.S.A. 20