Slides for ALMS Framework presentation.

1. Measuring the Difficulty of
Revising/Remixing OER

2. First Rule of Delphi Study

3. Slides Available
http://www.slideshare.net/OpenEdBlogger/
alms-analysis-presentation

4. 4R’s
1.Reuse
2.Revise
3.Remix
4.Redistribute

5. OER Revise/Remix
Photo by Crisitano Betta

6. Why?
Photo by Greg Westfall.

7. Categories of Difficulty
1. Content
2.Pedagogical
3. Technical

8. ALMS Analysis
1.Access to editing tools?
2.Level of expertise required to revise
or remix?
3.Meaningfully editable?
4.Source-file access?
Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The Four R’s of Openness and ALMS Analysis:
Frameworks for Open Educational Resources. Open Learning: The Journal of Open and Distance
Learning, 25(1), 37-44.

9. Rubric
1. Break down each part of the ALMS Analysis into parts
2. Requires some technical knowledge and training
3. Apply to a variety OER

10. Access
1 2 3 4
Is appropriate software No - - Yes N/A
application pre-installed
with the operating
system?
Is an appropriate software Not found Found Found Found
application downloadable through through through through
from the Internet? Internet significant some Internet
search research research search
easily

11. Access Part 2
1 2 3 4
How much does the Program cost Program Some No N/A
application cost? is expensive cost is program program
significant cost cost
How large is the File to be File to be Found Found
program to be downloaded is downloaded through through
downloaded? prohibitively a significant some Internet
large size research search
easily

12. Access Part 3
1 2 3 4
Does the program Program Program Program Program N/A
depend on other depends on a depends on depends on does not
software or large amount a significant some depend on
programming of libraries or amount of libraries or any
libraries? other libraries or other libraries or
programs other programs files
programs
Is appropriate No Access with Access with Access
functionality significant some with no
available from a restrictions restrictions restrictions
web-based service? or sign-up
only

13. Access Part 4
1 2 3 4
Does the program Program Program Program Program N/A
depend on other depends on a depends on depends on does not
software or large amount a significant some depend on
programming of libraries or amount of libraries or any
libraries? other libraries or other libraries or
programs other programs files
programs
Is appropriate No Access with Access with Access
functionality significant some with no
available from a restrictions restrictions restrictions
web-based service? or sign-up
only

14. Access Part 5
1 2 3 4
How much does the Service cost is Service cost Some No service N/A
service cost? expensive is significant service cost cost
Does the service Service Service Service Service
require the user to requires requires an requires an does not
have an account? extensive account account require an
account with with a with some account
major privacy significant or no
concerns amount of information
information

15. Access Part 6
1 2 3 4
Are you able to cut- No - - Yes N/A
and-paste the text
into a word
processor or text
editor?

16. Expertise
1 2.5 4
What degree of text Extensive text Some text editing Minimal text N/A
editing expertise is editing expertise required editing
required expertise (Example: footnotes, expertise
required headers) required
(Example: (Example:
multi-column bolding,
layout) headings)
What degree of Extensive Some image editing Minimal
image editing image editing expertise required image
expertise is expertise (Example: multiple editing
required? required layers, filters) expertise
(Example: required
masking, clone (Example:
stamp) cropping,
brightness/
contrast)

17. Expertise Part 2
1 2.5 4
What degree of Extensive Some audio editing Minimal N/A
audio editing audio editing expertise required audio editing
expertise is required expertise (Example: audio filters, expertise
required transitions) required
(Example: (Example:
remixing/revisi trimming and
ng multiple saving in
sources) appropriate
format)
What degree of Extensive Some video editing Minimal
video editing video editing expertise required video editing
expertise is expertise (Example: transitions, expertise
required? required filters) required
(Example: (Example:
remixing/revisi trimming and
ng multiple saving in
sources) appropriate
format)

18. Expertise Part 2
1 2.5 4
What degree of Extensive Some video editing Minimal video N/A
programming video editing expertise required editing
expertise is expertise (Example: functions, expertise
required? required if/then statements) required
(Example: data (Example:
structures such variable
as Object- manipulation,
Oriented basic
programming) arithmetic)
What degree of Extensive Some video editing Minimal video
specialty expertise is specialty expertise required editing
required? expertise (Example: intermediate expertise
required animation or 3D required
(Example: modeling) (Example:
Advanced basic
animation or animation or
3D modeling) 3D modeling)

19. Editable
1 2 3 4
What portion of the No text can Some text A significant All of the N/A
text in the OER is be edited can be amount of text can be
editable? edited text can be edited
(Example: Text stored edited
independently, not
embedded in an image)
What portion of the No images Some A significant All of the
images in the OER is can be images can amount of images can
editable? edited be edited the images be edited
(Example: Images can be
stored independently, edited
not embedded in a
PDF)

20. Editable continued
• What portion of the audio in the OER is
editable?
(Example: Audio stored independently, not embedded in a video)
• What portion of the video in the OER is
editable?
(Example: Video stored independently, not embedded in a Flash object)
• What portion of the other components in the OER
is editable?
(Example: 3D Model)

21. Source Files
1 2 3 4
What portion of the Text source Some text A significant All text N/A
source files for the files are not source files amount of source files
text in the OER is available are available the text are
available? source files available
(Example: a Word are available
document available for
a PDF)
What portion of the Image Some image A significant All image
source files for the source files source files amount of source files
images in the OER is are not are available the image are
available? (Example: a available source files available
Photoshop file with all are available
layers available)

22. Source Files continued
• What portion of the source files for the video in the OER
is available?
(Example: Video project files)
• What portion of the source files for the programming
code in the OER is available?
(Example: Source code downloadable from a website)
• What portion of the source files for the other
components of the OER is available?
(Example: FLA files for SWF Flash objects)

24. Calculations
Each category subtotal is the Total score divided
by the number of Non-N/A responses
Subtotal Score
Access to Editing Tools 3.75
Level of Expertise 3
Meaningfully Editable 4
Source Files Available 4
Average 3.69

25. Delphi Study

26. Criteria
1. Reliability
2. Desirability
3. Feasibility
Participants asked to rater each cell, some gave
an entire row the same score.

27. 1st Round Results
Reliability Desirability Feasibility Mean
Access 2.53 2.99 3.08 2.87
Level 2.97 3.09 2.93 3
Meaningful 3.17 3.41 3.35 3.31
Source 3.07 3.18 3.31 3.19
Total 2.93 3.17 3.17 3.09

28. Comments
• Concerns about cost of technical reusability.
• Concerns about technical reusability being only
one of many values.
• Concerns about properties of the OER itself
versus the context in which reuse is occurring.

29. Future Directions
1. Possible automated scoring of Access
component
2. Revising other components for future Delphi
study rounds
3. Rating a sample of OER for inter-rater
reliability

30. Photo by walknboston

31. Thank You
Seth Gurell
sethgurell@gmail.com
On Google+ or Facebook