The document discusses tools and methods for improving reproducibility in research, including open data and open source tools. It summarizes that less than 30% of published studies are reproducible due to lack of sharing of data, code, and workflows. It promotes hosting research objects like data, images, and workflows to make them accessible and citable. Specific tools mentioned include GigaDB for data, OMERO for images, Galaxy and executable documents for workflows, and virtual machines for replicable computational environments.

1. Tools for:
Open-Source
Open-Data
Rob L Davidson about.me/rob.davidson
www.slideshare.net/RobertDavidson6/g3-talk-rld2

2. The problem
reproducibility.cs.arizona.edu
•515 papers (429 conf, 86 journal)
•<30% reproducible

3. The problem
reproducibility.cs.arizona.edu

4. The Cause
• Stodden 2010
– 638 registrant at NIPS
• 30% share code
• 20% share data
http://web.stanford.edu/~vcs/papers/SMPRCS2010.pdf

5. Publishers must provide!
Hosting
Curating
Citations for everything:
data, tools + workflows

6. Tools for Reproducibility
• Data: GigaDB
• Images: OMERO
• Workflows
– Galaxy
– Executable Docs
– VMs

7. GigaDB
github.com/gigascience/gigadb-cogini

8. Hosting all data

9. Hosting all research objects

10. Impact for research objects
• Host
• Curate
• Share
• Cite - DOI

11. Even more accessible, transparent data?
Hosting image data with OMERO

12. Re-producing Images
· Image LIMS
· Keeps metadata with image
· Means the image can be
found later!
· Image can be understood
· Also some processing
options
http://www.openmicroscopy.org/site/products/omero

13. Accessible, transparent Images
· Embed in web
· Full res
· View without special
software
· Adjust contrast etc
· Link all images to pub!
· No cherry picking!
http://www.openmicroscopy.org/site/products/omero

14. Cyber-Centipedes! Phenotyping
NO

15. Accessible Cyber-Centipede images
OMERO: providing
access to imaging data
View, filter, measure raw
images with direct links
from journal article.
See all image data, not
just cherry picked
examples.
Download and reprocess.

16. OMERO: Adding value

17. The alternative...
...look but don't touch

18. Workflows
1. Galaxy
galaxyproject.org

19. galaxy.cbiit.cuhk.edu.hk

20. Implement workflows in a community-accepted format
http://galaxyproject.org
Open source
Over 45,000 main
Galaxy server users
Over 1,000 papers
citing Galaxy use
Over 55 Galaxy
servers deployed

21. Implement workflows in an intuitive format
Tool list TooCol ppyraighrta NmBAFe-Bt 2e0r1i3sation Results panel

22. Visualising Workflows

23. Birmingham Metabo-Galaxy Workflow

24. Birmingham Metabo-Galaxy
Tools wrapped in Python and XML
User sees web form (easy!)
Data stored centrally (secure!)
Work done centrally (easy update)

26. Hosting Workflows

27. Hosting Workflows
1) Test data
2) Software files
3) Instructions
+ Galaxy implementation

28. Can we reproduce results? SOAPdenovo2 S. aureus pipeline

29. Galaxy
Most accessible
Easy to share (galaxy toolshed)
Quite a bit of work
Doesn't include publication explanations

30. Workflows
2. Executable Docs

31. Open lab books, dynamic documents
• Facilitate reuse and sharing with tools like: Knitr, Sweave,
iPython Notebook
Sweave
• Working towards executable papers…

32. E.g.

33. E.g.

34. Some testimonials for Knitr
Authors (Wolfgang Huber)
“I do all my projects in Knitr. Having the textual explanation,
the associated code and the results all in one place really
increases productivity, and helps explaining my analyses to
colleagues, or even just to my future self.”
Reviewers (Christophe Pouzat)
“It took me a couple of hours to get the data, the few custom
developed routines, the “vignette” and to REPRODUCE
EXACTLY the analysis presented in the manuscript. With few
more hours, I was able to modify the authors’ code to change
their Fig. 4. In addition to making the presented research
trustworthy, the reproducible research paradigm definitely
makes the reviewer’s job much more fun!

35. Executable docs:
Completely reproduce paper!
May require some code-skills

36. Workflow accessibility:
VMs

37. Why VMs?
• OS settings
• Dependencies
– Versions
– e.g. python!
• Data + Code linked
• Download or run in
cloud

38. VMs in GigaDB

39. VMs:
Can host Galaxy
Can hold KnitR code
Provides 'snapshot' of working system

40. Summary

41. Share data in GigaDB
Share all images in GigaDB
-View images via OMERO
Share code in GigaDB!
Share pipeline using:
Executable docs!
Galaxy!
VMs!

42. Improve
reproducibility!
Give us data, papers &
pipelines*
Contact us:
scott@gigasciencejournal.com
editorial@gigasciencejournal.com
database@gigasciencejournal.com
* APC’s currently generously covered
by BGI until 2015
www.gigasciencejournal.com

43. team: Our collaborators: Case study:
Ruibang Luo (BGI/HKU)
Shaoguang Liang (BGI-SZ)
Tin-Lap Lee (CUHK)
Qiong Luo (HKUST)
Senghong Wang (HKUST)
Yan Zhou (HKUST)
Thanks to:
Funding from: CBIIT
@gigascience
facebook.com/GigaScience
blogs.biomedcentral.com/gigablog/
Peter Li
Huayan Gao
Chris Hunter
Jesse Si Zhe
Nicole Nogoy
Laurie Goodman
Amye Kenall (BMC)
Marco Roos (LUMC)
Mark Thompson (LUMC)
Jun Zhao (Lancaster)
Susanna Sansone (Oxford)
Philippe Rocca-Serra (Oxford)
Alejandra Gonzalez-Beltran (Oxford)
www.gigadb.org
galaxy.cbiit.cuhk.edu.hk
www.gigasciencejournal.com