This presentation discusses the current dilemma with respect to Open Educational Resources (OER) search. It introduces existing OER search methodologies and highlights their weaknesses. The Desirability framework for parametrically measuring the usefulness of an OER is also discussed. The desirability framework uses the D-index to measure the openness, accessibility and relevance of an OER. OERScout, a text mining based faceted search engine is introduced for improved OER search. It uses autonomously identified domain specific keywords, the D-index and faceted search to allow focused OER search.

1. The Current OER Search Dilemma
Guest Lecture
24th and 27th June 2013
Sukhothai Thammathirat Open University, Thailand
Ishan Abeywardena
MSc, MSc (Brunel), BSc (Bangalore), MIEEE, MBCS, MIET, MTA
Senior Lecturer, School of Science and Technology
Wawasan Open University
Penang, Malaysia

2. Acknowledgement
I express my gratitude to:
– Assoc. Prof Dr. Chailerd Pichitpornchai for
extending me this kind invitation;
– Ms. Rattip Phukkeson and all other colleagues at
STOU for having me here.

3. Lecture Outline
• Review of the current OER search dilemma.
• Parametrically measuring the Desirability of
OER using D-index.
• OERScout Technology Framework: A Novel
Approach to OER Search.

4. The Current OER Search Dilemma

5. Main Reference
Abeywardena, I.S., & Chan, C.S. (2013).
Review of the Current OER Search
Dilemma. Proceedings of the 57th World
Assembly of International Council on
Education for Teaching (ICET 2013),
Nonthaburi, Thailand.
Available at:
https://www.researchgate.net/publicatio
n/240310607_Review_of_the_Current_O
ER_Search_Dilemma?ev=prf_pub

6. Talking Points
• Current State of OER
• The Dilemma
• Some Existing Solutions
– Pearson’s Project Blue Sky
– GLOBE
– LRMI
– OERScout

7. Current State of OER

8. The Dilemma
So…how do I find the material I need for my
teaching

9. Literature
• ...The problem is in finding the resources, and more correctly finding the “right”
resources. Using a regular search engine like Google to find content is not always a
viable option as it will generate too many answers. There is, hence, a need to easily
find relevant content...” (Hatakka, 2009)
• “searching this way (using existing search engines such as Google) might be a long
and painful process as most of the results are not usable for educational purposes”
(Pirkkalainen & Pawlowski, 2010)
• No single search engine is still able to locate resources from all the OER
repositories (West & Victor, 2011)
• One of the major barriers to the use and re-use of OER is the difficulty of finding
quality OER matching a specific context (Dichev & Dicheva, 2012)
• “…the problem with open content is not the lack of available resources on the
Internet but the inability to locate suitable resources for academic use” (Unwin,
2005).

10. Some Existing Solutions
• Google: “…searching this way might be a long and
painful process as most of the results are not usable for
educational purposes” (Pirkkalainen & Pawlowski, 2010).
• Federated Search: BRENHET2; OpeScout; Global
Learning Object Brokered Exchange (GLOBE); and
Pearson’s Project Blue Sky.
• Semantic Search: OER-CC ontology; the “Assistant”
prototype; the “Folksemantic” project; and “Agrotags”.
Pirkkalainen, H., Pawlowski, J. (2010). Open Educational Resources and Social Software in Global E-Learning Settings. In Yliluoma, P. (Ed.)
Sosiaalinen Verkko-oppiminen. IMDL, Naantali, 23–40.

11. The Dilemma
Yeah…but which one do I choose

12. Pearson’s Project Blue Sky
http://www.pearsonlearningsolutions.com/pearson-bluesky/

13. GLOBE
http://www.globe-info.org/

14. LRMI
http://creativecommons.org/tag/learning-resource-metadata-initiative

15. OERScout
http://www.oerscout.org/

16. Questions

17. Desirability of OER using D-index

18. Main Reference
Abeywardena, I.S., Raviraja, R., & Tham, C.Y.
(2012). Conceptual Framework for
Parametrically Measuring the Desirability of
Open Educational Resources using D-index.
International Review of Research in Open and
Distance Learning, 13(2), 104-121 (ISI-
indexed publication).
Available at:
http://www.irrodl.org/index.php/irrodl/articl
e/view/1177/2142

19. Talking Points
• Usefulness of OER
• The Desirability of OER
• Measuring the Desirability using D-index

20. What is the Usefulness of an OER?
• The usefulness of an OER for a
particular teaching or learning need
can only be accurately assessed by
reading through the content.
• This aspect of use and re-use of
OER will remain a human function
regardless of the improvements in
technology.

21. Measuring the Usefulness OER
However, there are other aspects of a resource which are
fundamental to the usefulness of that particular resource and can be
parametrically identified by a software based mechanism:
– Whether a resource is relevant to a user’s needs;
– Whether the resource is open enough for using,
reusing, remixing and redistributing;
– Whether the resource is accessible with respect
to technology.

22. What is Desirability?
• If the user cannot easily use,
reuse and remix a resource
with available technology, the
resource becomes less useful
to the user.
• Less useful resources are less
desirable for teaching and
learning needs….

23. Measuring the Desirability of OER
Within the requirement of being able to use and reuse a
particular OER, the three parameters of Desirability can be
defined as:
– level of openness: the permission to use and reuse
the resource;
– level of access: the technical keys required to unlock
the resource;
– relevance: the level of match between the resource
and the needs of the user.

24. i.e. Desirability
D-index = (level of access x level of openness x relevance) / 256

25. Openness
Permission Value
Reuse 1
Redistribute 2
Revise 3
Remix 4
The level of openness based on the four R’s of openness
Mapping the CC licenses to the 4 R’s
Permission Creative Commons (CC) licence Value
Reuse None 1
Redistribute Attribution-NonCommercial-NoDerivs (CC BY-NC-ND)
Attribution-NoDerivs (CC BY-ND)
2
Revise Attribution-NonCommercial-ShareAlike (CC BY-NC-SA)
Attribution-ShareAlike (CC BY-SA)
3
Remix Attribution-NonCommercial (CC BY-NC)
Attribution (CC BY)
4

26. Access
Access to editing
tools
Level of expertise required to
revise or remix
Meaningfully
editable
Source-file access Value
LOW HIGH NO NO 01
LOW HIGH NO YES 02
LOW HIGH YES NO 03
LOW HIGH YES YES 04
LOW LOW NO NO 05
LOW LOW NO YES 06
LOW LOW YES NO 07
LOW LOW YES YES 08
HIGH HIGH NO NO 09
HIGH HIGH NO YES 10
HIGH HIGH YES NO 11
HIGH HIGH YES YES 12
HIGH LOW NO NO 13
HIGH LOW NO YES 14
HIGH LOW YES NO 15
HIGH LOW YES YES 16
ThelevelofaccessbasedontheALMSanalysis

27. Relevance
Search rank Value
Below the top 30 ranks of the search results 1
Within the top 21-30 ranks of the search results 2
Within the top 11-20 ranks of the search results 3
Within the top 10 ranks of the search results 4
The level of relevance based on search rank (Vaughan, 2004)
• Users will only consider the top ten ranked results for a particular search
as the most relevant;
• Users will ignore the results below the top 30 ranks.
Vaughan, L. (2004). New measurements for search engine evaluation proposed and tested. Information Processing and Management 40, 677–691.

28. Calculation
Original search results (OER Commons example)
Calculation of D-index
After application of D-index

29. Sample Search
Search Rank Title CC Lisence File Type
1 18.01 Single Variable Calculus CC BY-NC-SA PDF
2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text
3 18.01 Single Variable Calculus CC BY-NC-SA PDF
4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text
5 18.02 Multivariable Calculus CC BY-NC-SA PDF
6 Single Variable Calculus CC BY-NC-SA PDF
7 Calculus Online Textbook CC BY-NC-SA PDF
8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text
9 18.075 Advanced Calculus for Engineers CC BY-NC-SA PDF
10 MATH 140 - Calculus I, Summer 2007 CC BY-NC-SA Protected
Top 10 search results returned by MERLOT for the keyword “calculus”

30. Original Search Results
• The original top ten search results only contain
resources which are released under the CC BY-
NC-SA license.
• 6/10 resources returned are in PDF format which
make them difficult to reuse and remix.
• Resource ranked as number ten is a protected
resource which requires a specific username and
password to access.

31. Application of D-index
Rank After
Applying D-
index
Original
Search
Rank Title CC Lisence File Type D-index
1 2 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.75
2 4 18.013A Calculus with Applications CC BY-NC-SA HTML/Text 0.75
3 8 Calculus for Beginners and Artists CC BY-NC-SA HTML/Text 0.75
4 14 Multivariable Calculus CC BY HTML/Text 0.75
5 19
MATH 10250 - Elements of Calculus
I, Fall 2008 CC BY-NC-SA HTML/Text 0.56
6 20 18.022 Calculus CC BY-NC-SA PDF 0.56
7 22 Single-Variable Calculus I CC BY HTML/Text 0.50
8 25 Single-Variable Calculus II CC BY HTML/Text 0.50
9 15 Highlights of Calculus CC BY-NC-SA Video 0.42
10 21 Calculus I CC BY HTML/Text 0.38
Top 10 results when D-index is applied to the results returned by MERLOT

32. Results After Applying D-index
• 8/10 resources are in HTML/Text formats
which are the most accessible in terms of
reuse.
• 4/10 resources are available under the CC BY
licence which make them the most open
resources in the list.

33. Benefits of the D-index
The application of the D-index
would greatly improve the
effectiveness of the search with
respect to locating the most
suitable resources for use and
reuse.

34. Let’s discuss more over tea?

35. OERScout Technology Framework

36. Main Reference
Abeywardena, I.S., Chan, C.S., & Tham, C.Y.
(2013). OERScout Technology Framework: A
Novel Approach to Open Educational
Resources Search. International Review of
Research in Open and Distance Learning, In
press (ISI-indexed publication).
Available at:
Currently in press. Will be available at
http://www.irrodl.org

37. Talking Points
• Why not Google?
• Why not native search engines of
repositories?
• What is OERScout?
• Why use OERScout?

38. Google“AdvancedSearch”resultsforOERonChemistry(24thMay2012)

39. Why Not Repositories?
Identify which material to look for
(e.g. integration, C++ programming)
Identify the search queries
(e.g. “undergraduate mathematics”)
Locate repository(word of mouth, some link
somewhere, go to the more popular
repositories)
Run multiple queries to find resources
Read each resource to identify the usefulness
(openness, access, relevance)
Identify useful resources
Repeat steps 3-6 on multiple repositories
(hundreds to thousands…..)

40. The Declaration
i. Facilitate finding, retrieving and sharing of OER.
Encourage the development of user-friendly
tools to locate and retrieve OER that are
specific and relevant to particular needs.
(UNESCO Paris OER Declaration, 2012)
UNESCO. (2012). Paris OER Declaration, Retrieved September18, 2012 from
http://www.unesco.org/new/fileadmin/MULTIMEDIA/HQ/CI/CI/pdf/Events/Paris%20OER%20Declaration_01.pdf

41. Specific & Relevant
• Specific: the suitability of an OER for a particular
teaching need.
For example, an OER on physics from the final
year syllabus of a physics degree would not be
suitable for a high school physics class.
• Relevant: the match between the content of the
OER and the content needed for a particular
teaching need.
For example, physical chemistry is not relevant for
a teaching need in organic chemistry.

42. OERScout
A Text Mining Algorithm which
–READS text based OER;
–LEARNS which academic domain(s) and
sub-domain(s) they belonged to;
–RECCOMENDS Desirable material for a
particular academic need.

43. Keyword-Document Matrix (KDM)

45. Advantages of OERScout
• Using the KDM, the system generates ranked lists
of relevant OER from heterogeneous
repositories to suit a given search query;
• Incorporates the Desirability framework to
recommend useful resources for academic
purposes;
• Uses a faceted search approach to allow users to
quickly zero-in on the resources they are after.

46. Benefits to Content Creators
• No need to manually define metadata;
• No need to publicise the availability of material;
• No need to build custom search mechanisms for
repositories;
• More visibility and reach of material to a wider
audience.

47. Benefits to Users
• Provides a central location for finding
resources scattered across the globe hidden in
high volume repositories;
• Locate only the most relevant resources.

48. Conclusion
The ultimate benefit of OERScout
is that both content creators and
users now only need to
concentrate on the actual
content and not the searching
and location of relevant OER.

49. Questions

50. About…
Ishan Abeywardena
Senior Lecturer, School of Science and Technology, Wawasan Open University, Penang, Malaysia
• MSc in Wireless Enterprise Business Systems, Brunel University, UK.
• MSc in Engineering Management, Brunel University, UK.
• BSc in Computer Science, Bangalore University, India.
• PhD Candidate in Computer Science, University Malaya, Malaysia. Areas of specialisation: text mining,
metadata, faceted search
Professional Member of
– Institute of Electrical and Electronic Engineers (MIEEE)
– British Computer Society (MBCS)
– Institution of Engineering and Technology (MIET)
– Microsoft Technology Associate (MTA)
Official Profile: http://www.wou.edu.my/IshanAbeywardena.html
Professional Profile: http://www.linkedin.com/in/ishansa
Research Profile: http://www.researchgate.net/profile/Ishan_Abeywardena/
Tech Blog: http://www.ishantalks.com
E-mail: ishansa@wou.edu.my

51. References
• Dichev, C., & Dicheva, D. (2012). Open Educational Resources in Computer Science Teaching. SIGCSE’11, February 29–March
3, 2012, Raleigh, NC, USA.
• Caswell, T., Henson, S., Jenson, M., & Wiley, D. (2008). Open Educational Resources: Enabling universal education.
International Review of Research in Open and Distance Learning 9(1), 1-11.
• Hatakka, M. (2009). Build It and They Will Come? – Inhibiting Factors for Reuse of Open Content in Developing Countries,
EJISDC 37(5), 1-16.
• Hilton, J., Wiley, D., Stein, J., & Johnson, A. (2010). The four R‘s of openness and ALMS Analysis: Frameworks for open
educational resources. Open Learning: The Journal of Open and Distance Learning, 25(1), 37-44.
• Pirkkalainen, H., Pawlowski, J. (2010). Open Educational Resources and Social Software in Global E-Learning Settings. In
Yliluoma, P. (Ed.) Sosiaalinen Verkko-oppiminen. IMDL, Naantali, 23–40.
• Unwin, T. (2005). Towards a Framework for the Use of ICT in Teacher Training in Africa. Open Learning 20, 113-130.
• West, P., & Victor, L. (2011). Background and action paper on OER. Report prepared for The William and Flora Hewlett
Foundation.