The presentation I held at #ocg12, based on the paper "The case for an open science in technology enhanced learning" by P. Kraker, D. Leony, W. Reinhardt, and G. Beham

1. www.know-center.at
Open Science
@PeterKraker
Opencamp Graz #ocg12
gefördert durch das Kompetenzzentrenprogramm

2. Reference
Kraker P., Leony D., Reinhardt, W., Beham G.: “The case for
an open science in technology enhanced learning”, Int.
J. Technology Enhanced Learning 3(6), 643-654.
Postprint available from: http://is.gd/open_science
Blogpost with background information:
2
http://science20.wordpress.com
© Know-Center 2010

3. Problems in Science
Information overload
Exponential growth (Price 1961)
Barnaby Rich Effect: “It„s always the other
author(s) who publishes too much and
“pollutes“, “floods”, “eutroficates” the literature,
never me” (Braun and Zsindelay 1985)
Exaggerated and inflated claims (Young et al.)
Reproducibility of results
Datasets and source code not available
Methodological information is missing
(Knorr-Cetina 1981)
Price, 1961 3
extended by Leydesdorff (2008)
© Know-Center 2010

4. Problems in Science
Simultaneous/repeated discoveries
Formation of “invisible colleges”
Technology Enhanced Learning
 Disjoint scientific communities (Gillet et al. 2009)
 Low-cross citation rate
 Low cross-authorship rate (Kirby et al. 2005, Maurer
and Khan 2010)
 Multi-disciplinarity instead of inter-disciplinarity
Can an Open Science help?
4
© Know-Center 2010

5. What is Open Science?
“Open Science means opening up the research
process by making all of its outcomes, and the
way in which these outcomes were achieved,
publicly available on the World Wide Web”
Open Data Open Source
Open
Science
Open
Open Access
Methodology
5
© Know-Center 2010

6. Open Access
Budapest Open Access Initiative
Free availability of publications on the Internet
Rights of the author: integrity of the work, acknowlegement
E-Prints: Unpublished, Pre-Print, Post-Print
arXiv.org, TeLearn Archive
Journals
Green Road
Gold Road
Directory of Open Access Journals (DOAJ)
6
© Know-Center 2010

7. Open Data
Publishing the data sets collected in the research process
on the World Wide Web, without restricting their use
(Murray-Rust 2008)
Important for
Reproducibility
Reuse of data
Aggregation of data
GenBank (storing DNA data sequences) – Bermuda http://lod-cloud.net /
principles
DataShop: educational data resources
dataTEL Initiative
7
© Know-Center 2010

8. Open Source
Open Source means that software is made available under
a license that permits anyone to use, change, improve,
or derive from existing source code, and sometimes
even to distribute the software (Feller and Fitzgerald
2002)
Advantages
Reuse of prototypes
Easier transfer into practice
Larger pool of developers
The R Project for Statistical Computing (http://r-project.org)
moodle (http://moodle.org)
8
© Know-Center 2010

9. Open Methodology
Papers do not contain all the methodological information
needed to reproduce a certain research result (Knorr-
Cetina 1981)
Decontextualization, Typification
Procedural remarks are missing
Open Methodology complements paper with an explicit and
detailed procedure on how to analyse the data collected
and to generate the obtained results
E.g. experimental setups, scripts written for computer
simulations, and aggregation rules in qualitative data
analysis
myExperiment
The Stanford Exploration Project (SEP): Reproducing
numerical results with makefiles 9
© Know-Center 2010

10. An Open Science for TEL
Connect the disjoint communities in TEL
Exchange of research findings
Discussion on implementations
Discussion of approaches to collect and
analyse data
Enables reproducibility of research
http://jtelsummerschool.eu /
Increases value of research
Enables researchers to build on each other‟s work
Efficiency: reduces redundant design and development
Comparability: which approach fits best, effectiveness
10
© Know-Center 2010

11. An Open Science for TEL
Benefits Stakeholders (e.g. teachers)
Research prototypes become more widely available
Can be used in practice much earlier
Greater impact on practice and more visibility for TEL
research.
Fosters Open Innovation
 moodle (originally PhD research project)
11
© Know-Center 2010

12. Issues
Legal and technical issues
Guidelines, standardized formats, appropriate licenses, and
proper citation methods.
Social issues
Issues among computer scientists - data, code (Stodden 2009)
 The time it takes to clean up and document for release
 The possibility that code/data may be used without citation
 Legal barriers, such as copyright
 Potential loss of future publications
 Competitors may get an advantage
 Privacy constraints
Reputation 12
© Know-Center 2010

13. Recommendations for implementing an Open
Science
Open Science is a community effort
Reproducibility and comparability as standard reviewing
criteria
Journals and conferences: making the submission of
source code, data, and methodological descriptions
together with the paper mandatory
Bermuda principles: DNA sequences should be rapidly
released into the public domain (GenBank)
Conferences and journals themselves should in turn commit
to making the papers openly accessible
Technical problems: review existing initiatives (DataCite;
Stodden 2010)
13
Standing problem: awareness
© Know-Center 2010

14. More to come
Barcamp Graz: May 11-13, 2012
Politcamp, Wissenscamp, Designcamp, iCamp, Geocamp
http://barcamp-graz.at
Special Track Science 2.0 (#STS) at i-KNOW 2012: Sep 5,
2012
Open Science
Recommendation
Analysis of Science
Change in scientific practice
http://i-know.tugraz.at/i-science/science-2-0 14
© Know-Center 2010

15. References
Brase, J. 2009. DataCite - A global registration agency for research data in Fourth International Conference on
Cooperation and Promotion of Information Resources in Science and Technology. IEEE, pp. 257–261.
Drachsler, H. et al. 2010. Issues and considerations regarding sharable data sets for recommender systems in
technology enhanced learning. Procedia Computer Science, 1(2), pp.2849-2858.
Feller, J. & Fitzgerald, B., 2002. Understanding Open Source Software Development, Addison-Wesley.
Gillet, D., Scott, P. & Sutherland, R. 2009. STELLAR European research network of excellence in technology
enhanced learning in International Conference on Engineering Education & Research
Kirby, J., Hoadley, C. & Carr-Chellman, A. 2005. Instructional Systems Design and the Learning Sciences: A Citation
Analysis. Educational Technology Research and Development, 53(1), pp.37-48.
Knorr-Cetina, K., 1981. The Manufacture of Knowledge: An Essay on the Constructivist and Contextual Nature of
Science, Pergamon Press
Maurer, H. & Khan, M.S. 2010. Research trends in the field of e-learning from 2003 to 2008: A scientometric and
content analysis for selected journals and conferences using visualization. Interactive Technology and Smart
Education, 7(1), pp.5-18.
Murray-Rust, P., 2008. Open Data In Science. Serials Review, 34(1), pp.52-64.
Price, D.J.D.S. 1963. Little science, big science. Columbia Univ. Press.
Reinhardt, W., Meier, C., Drachsler, H. & Sloep, P, 2011. Analyzing 5 years of EC-TEL proceedings. In C. D. Kloos,
D. Gillet, R. M. C. Garcìa, F. Wild, & M. Wolpers, eds. Towards Ubiquitous Learning. Proceedings of the 6th
European Conference on Technology Enhanced Learning, pp 531–536. Springer Berlin/Heidelberg.
Stodden, V., 2009. The Legal Framework for Reproducible Scientific Research: Licensing and Copyright. Computing
in Science & Engineering, 11(1), pp.35-40.
Stodden, V., 2010. The Scientific Method in Practice: Reproducibility in the Computational Sciences. Sloan School
Working Paper, 4773-10, MIT. Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1550193
[Accessed April 10, 2011].
Young, N.S., Ioannidis, J.P. a & Al-Ubaydli, O. 2008. Why current publication practices may distort science. PLoS
medicine, 5(10), p.e201.
15
© Know-Center 2010

16. www.know-center.at
Thank you!
@PeterKraker
pkraker@know-center.at
http://science20.wordpress.com
http://mendeley.com/profiles/peter-kraker-2
gefördert durch das Kompetenzzentrenprogramm