Repeat after me: Is our research reproducible (enough)?

•Als PPTX, PDF herunterladen•

2 gefällt mir•1,788 views

A short talk I gave to colleagues at the Centre for Ecological and Evolutionary Synthesis (CEES) Dept. of Biosciences, University of Oslo

Technologie

Repeat after me:
Is our research reproducible (enough)?
Lex Nederbragt

whoami

@lexnederbragt

flxlexblog.wordpress.com

What is ‘Reproducible research’?
“Reproducibility is the ability of an entire
experiment or study to be reproduced

either by the researcher or by someone else
working independently.
It is one of the main principles of the scientific
method.”

Wikipedia

The worst case

https://www.sciencemag.org/content/334/6056/579.full.pdf

A reproducibility crisis?

Retraction rates are on the rise Murat Cokol, Fatih Ozbay & Raul Rodriguez-Esteban
EMBO reports (2008) 9, 2 doi:10.1038/sj.embor.7401143

An example from our own work

Are all these figures/tables reproducible?

Supplementary information, page 19-63 of 72

An exercise in reproducibility
Master student project at informatics Sabba Ifzal

Solutions

Flick: http://farm4.staticflickr.com/3319/3265787219_bfbc654b5e_o.jpg

Wikimedia commons

Levels of reproducibility

Loose descriptions
that are very
incomplete, no
data/materials

All
data, protocols, code
etc

Levels of reproducibility
Minimum effort needed to
reproduce meaningfully
Loose descriptions
that are very
incomplete, no
data/materials

All
data, protocols, code
etc

Reproducibility and reusability

http://upload.wikimedia.org/wikipedia/commons/4/48/Recycle.jpg

How to make your research reproducible
Document your work

How to make your research reproducible

http://howdoitradestocks.com/wp-content/uploads/2011/12/share-ideas1.jpg

How to make your research reproducible

https://twitter.com/vsbuffalo/statuses/377651287133003776

How to make your research reproducible

https://github.com/lexnederbragt

How to make your research reproducible
Version control

https://www.atlassian.com/git/workflows

How to make your research reproducible

http://software-carpentry.org

https://twitter.com/ctitusbrown/status/383730337811296256

Weitere ähnliche Inhalte

Ähnlich wie Repeat after me: Is our research reproducible (enough)?

Slides for a talk held at BioScience Seminar at Dept. of Pharmaceutical BioSciences at Uppsala University on December 16, 2016. The event webpage: http://www.farmbio.uu.se/calendar/kalendarium-detaljsida/?eventId=22496 Structure of the talk: Reproducibility in Scientific Data Analysis ... ● What is it? ● Why is it important? ● Why is it a problem? ● What can we do about it? ● What does pharmb.io do about it?

Reproducibility in Scientific Data Analysis - BioScience Seminar

Samuel Lampa

Chemical Heritage Open Notebook Science

Jean-Claude Bradley

Exaptation

Jean Baptiste

OA Week 2012 Miami U: How Open Scholarship is Changing Research

William Gunn

Science Blogging Conference Bradley Talk

Jean-Claude Bradley

My presentation to Chemists Without Borders on September 6, 2007. I introduce the basics of the UsefulChem project and Open Notebook Science and why malaria is a particularly good target for Open Science. This includes the CombiUgi project and our collaborations with ChemSpider, Rajarshi Guha, Phil Rosenthal, Dan Zaharevitz and Tsu-Soo Tan. The final slide shows the presentation of our work on Second Life at a SciFoo Lives On session.

Open Notebook Science and Malaria

Jean-Claude Bradley

I2e Version5

the nciia

Reproducibility by Other Means: Transparent Research Objects

Timothy McPhillips

25. idiom, vp 7 quiz, newspaper presentation, 4 a text

IECP

Reproducibility

Philip Bourne

Here is my talk "Open Notebook Science using Blogs and Wikis" at the American Chemical Society meeting in Chicago on March 27, 2007 at the Communicating Chemistry Symposium. The first half is basically a condensed summary of how we are using UsefulChem to do Open Science. I then demonstrate for the first time Dave's Excel code to compute kinetics from JCAMP NMR reaction profiles and the building in Second Life where Beth and Eloise have help me to set up a poster room with NMR spectra, molecules and an organic chemistry quiz.

Open Notebook Science at ACS March07

Jean-Claude Bradley

Keynote at CLIR Workshop (Webinar): Torward Open, Reproducible, and Reusable Research. February 10, 2021. https://reusableresearch.com/ ABSTRACT. The “reproducibility crisis” has resulted in much interest in methods and tools to improve computational reproducibility. FAIR data principles (data should be findable, accessible, interoperable, and reusable) are also being adapted and evolved to apply to other artifacts, notably computational analyses (scientific workflows, Jupyter notebooks, etc.). The current focus on computational reproducibility of scripts and other computational workflows sometimes overshadows a somewhat neglected and arguably more important issue: transparency of data analysis, including data wrangling and cleaning. In this talk I will ask the question: What information is gained by conducting a reproducibility experiment? This leads to a simple model (PRIMAD) that aims to answer this question by sorting out different scenarios. Finally, I will present some features of Whole-Tale, a computational platform for reproducible and transparent computational experiments.

Computational Reproducibility vs. Transparency: Is It FAIR Enough?

Bertram Ludäscher

Results may vary: Collaborations Workshop, Oxford 2014

Carole Goble

Web Science, SADI, and the Singularity

Mark Wilkinson

Keynote given by Carole Goble on 23rd July 2013 at ISMB/ECCB 2013 http://www.iscb.org/ismbeccb2013 How could we evaluate research and researchers? Reproducibility underpins the scientific method: at least in principle if not practice. The willing exchange of results and the transparent conduct of research can only be expected up to a point in a competitive environment. Contributions to science are acknowledged, but not if the credit is for data curation or software. From a bioinformatics view point, how far could our results be reproducible before the pain is just too high? Is open science a dangerous, utopian vision or a legitimate, feasible expectation? How do we move bioinformatics from one where results are post-hoc "made reproducible", to pre-hoc "born reproducible"? And why, in our computational information age, do we communicate results through fragmented, fixed documents rather than cohesive, versioned releases? I will explore these questions drawing on 20 years of experience in both the development of technical infrastructure for Life Science and the social infrastructure in which Life Science operates.

ISMB/ECCB 2013 Keynote Goble Results may vary: what is reproducible? why do o...

Carole Goble

An introduction to the problem statement

teachner

Quantified Self and Learning Analytics

Erik Duval

How to Unleash the Transformative Potential of Local Communities / Ralph Piot...

Sitra Kestävä talous -foorumi

Reproducibility 1

Khalid Belhajjame

The beauty of workflows and models

myGrid team

Ähnlich wie Repeat after me: Is our research reproducible (enough)? (20)

Reproducibility in Scientific Data Analysis - BioScience Seminar

Chemical Heritage Open Notebook Science

Exaptation

OA Week 2012 Miami U: How Open Scholarship is Changing Research

Science Blogging Conference Bradley Talk

Open Notebook Science and Malaria

I2e Version5

Reproducibility by Other Means: Transparent Research Objects

25. idiom, vp 7 quiz, newspaper presentation, 4 a text

Reproducibility

Open Notebook Science at ACS March07

Computational Reproducibility vs. Transparency: Is It FAIR Enough?

Results may vary: Collaborations Workshop, Oxford 2014

Web Science, SADI, and the Singularity

ISMB/ECCB 2013 Keynote Goble Results may vary: what is reproducible? why do o...

An introduction to the problem statement

Quantified Self and Learning Analytics

How to Unleash the Transformative Potential of Local Communities / Ralph Piot...

Reproducibility 1

The beauty of workflows and models

Mehr von Lex Nederbragt

Coding & Best Practice in Programming in the NGS era

Lex Nederbragt

Why of version control

Lex Nederbragt

Assembly: before and after

Lex Nederbragt

Improving and validating the Atlantic Cod genome assembly using PacBio

Lex Nederbragt

A different kettle of fish entirely: bioinformatic challenges and solutions f...

Lex Nederbragt

Combining PacBio with short read technology for improved de novo genome assembly

Lex Nederbragt

Updated: New High Throughput Sequencing technologies at the Norwegian Sequenc...

Lex Nederbragt

New High Throughput Sequencing technologies at the Norwegian Sequencing Centr...

Lex Nederbragt

How and why I use blogging

Lex Nederbragt

How to sequence a large eukaryotic genome

Lex Nederbragt

Assembly of metagenomes

Lex Nederbragt

NGS techniques and data

Lex Nederbragt

NGS: bioinformatic challenges

Lex Nederbragt

Mehr von Lex Nederbragt (13)

Coding & Best Practice in Programming in the NGS era

Why of version control

Assembly: before and after

Improving and validating the Atlantic Cod genome assembly using PacBio

A different kettle of fish entirely: bioinformatic challenges and solutions f...

Combining PacBio with short read technology for improved de novo genome assembly

Updated: New High Throughput Sequencing technologies at the Norwegian Sequenc...

New High Throughput Sequencing technologies at the Norwegian Sequencing Centr...

How and why I use blogging

How to sequence a large eukaryotic genome

Assembly of metagenomes

NGS techniques and data

NGS: bioinformatic challenges

Kürzlich hochgeladen

This presentations targets students or working professionals. You may know Google for search, YouTube, Android, Chrome, and Gmail, but did you know Google has many developer tools, platforms & APIs? This comprehensive yet still high-level overview outlines the most impactful tools for where to run your code, store & analyze your data. It will also inspire you as to what's possible. This talk is 50 minutes in length.

Powerful Google developer tools for immediate impact! (2023-24 C)

wesley chun

MINDCTI Revenue Release Quarter One 2024

MIND CTI

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Manulife - Insurer Transformation Award 2024

The Digital Insurer

MS Copilot expands with MS Graph connectors

Nanddeep Nachan

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

MadyBayot

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

Zilliz

ICT role in 21st century education and its challenges

rafiqahmad00786416

Boost Fertility New Invention Ups Success Rates.pdf

sudhanshuwaghmare1

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Edi Saputra

Whatsapp Number Escorts Call girls 8617370543 Available 24x7 Navi Mumbai Call Girls Service Offer Genuine VIP Model Escorts Call Girls in Your Budget. Navi Mumbai Call Girls Service Provide Real Call Girls Number. Make Your Sexual Pleasure Memorable with Our Navi Mumbai Call Girls at Affordable Price. Top VIP Escorts Call Girls, High Profile Independent Escorts Call Girls, Housewife Women Escorts Call Girl, College Girls Escorts Call Girls, Russian Escorts Call girls Service in Your Budget.

Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model

Deepika Singh

The Good, the Bad and the Governed - Why is governance a dirty word? David O'Neill, Chief Operating Officer - APIContext Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

apidays

In this session, we will delve into strategic approaches for optimizing knowledge management within Microsoft 365, amidst the evolving landscape of Copilot. From leveraging automatic metadata classification and permission governance with SharePoint Premium, to unlocking Viva Engage for the cultivation of knowledge and communities, you will gain actionable insights to bolster your organization's knowledge-sharing initiatives. In this session, we will also explore how to facilitate solutions to enable your employees to find answers and expertise within Microsoft 365. You will leave equipped with practical techniques and a deeper understanding of how there is more to effective knowledge management than just enabling Copilot, but building actual solutions to prepare the knowledge that Copilot and your employees can use.

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Drew Madelung

Corporate and higher education. Two industries that, in the past, have had a clear divide with very little crossover. The difference in goals, learning styles and objectives paved the way for differing learning technologies platforms to evolve. Now, those stark lines are blurring as both sides are discovering they have content that’s relevant to the other. Join Tammy Rutherford as she walks through the pros and cons of corporate and higher ed collaborating. And the challenges of these different technology platforms working together for a brighter future.

Corporate and higher education May webinar.pptx

Rustici Software

Architecting Cloud Native Applications

WSO2

Scaling API-first – The story of a global engineering organization Ian Reasor, Senior Computer Scientist - Adobe Radu Cotescu, Senior Computer Scientist - Adobe Apidays New York 2024: The API Economy in the AI Era (April 30 & May 1, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

apidays

Effective data discovery is crucial for maintaining compliance and mitigating risks in today's rapidly evolving privacy landscape. However, traditional manual approaches often struggle to keep pace with the growing volume and complexity of data. Join us for an insightful webinar where industry leaders from TrustArc and Privya will share their expertise on leveraging AI-powered solutions to revolutionize data discovery. You'll learn how to: - Effortlessly maintain a comprehensive, up-to-date data inventory - Harness code scanning insights to gain complete visibility into data flows leveraging the advantages of code scanning over DB scanning - Simplify compliance by leveraging Privya's integration with TrustArc - Implement proven strategies to mitigate third-party risks Our panel of experts will discuss real-world case studies and share practical strategies for overcoming common data discovery challenges. They'll also explore the latest trends and innovations in AI-driven data management, and how these technologies can help organizations stay ahead of the curve in an ever-changing privacy landscape.

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

TrustArc

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

Kürzlich hochgeladen (20)

Powerful Google developer tools for immediate impact! (2023-24 C)

MINDCTI Revenue Release Quarter One 2024

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

2024: Domino Containers - The Next Step. News from the Domino Container commu...

A Year of the Servo Reboot: Where Are We Now?

Manulife - Insurer Transformation Award 2024

MS Copilot expands with MS Graph connectors

EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER

"I see eyes in my soup": How Delivery Hero implemented the safety system for ...

ICT role in 21st century education and its challenges

Boost Fertility New Invention Ups Success Rates.pdf

Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving

Navi Mumbai Call Girls 🥰 8617370543 Service Offer VIP Hot Model

Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...

Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...

Corporate and higher education May webinar.pptx

Architecting Cloud Native Applications

Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe

TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery

Data Cloud, More than a CDP by Matt Robison

Repeat after me: Is our research reproducible (enough)?

1. Repeat after me: Is our research reproducible (enough)? Lex Nederbragt

2. whoami @lexnederbragt flxlexblog.wordpress.com

3. whatdoido

4. What is ‘Reproducible research’? “Reproducibility is the ability of an entire experiment or study to be reproduced either by the researcher or by someone else working independently. It is one of the main principles of the scientific method.” Wikipedia

5. Reproducibility: why?

6. The worst case https://www.sciencemag.org/content/334/6056/579.full.pdf

7. A reproducibility crisis? Retraction rates are on the rise Murat Cokol, Fatih Ozbay & Raul Rodriguez-Esteban EMBO reports (2008) 9, 2 doi:10.1038/sj.embor.7401143

8. Does this affect us?

9. An example from our own work 2011

10. An example from our own work Are all these figures/tables reproducible? Supplementary information, page 19-63 of 72

11. An exercise in reproducibility Master student project at informatics Sabba Ifzal

12. Solutions Flick: http://farm4.staticflickr.com/3319/3265787219_bfbc654b5e_o.jpg Wikimedia commons

13. Levels of reproducibility Loose descriptions that are very incomplete, no data/materials All data, protocols, code etc

14. Levels of reproducibility Minimum effort needed to reproduce meaningfully Loose descriptions that are very incomplete, no data/materials All data, protocols, code etc

15. Reproducibility and reusability http://upload.wikimedia.org/wikipedia/commons/4/48/Recycle.jpg

16. How to make your research reproducible Document your work

17. How to make your research reproducible http://howdoitradestocks.com/wp-content/uploads/2011/12/share-ideas1.jpg

18. How to make your research reproducible https://twitter.com/vsbuffalo/statuses/377651287133003776

19. How to make your research reproducible https://github.com/lexnederbragt

20. How to make your research reproducible Version control https://www.atlassian.com/git/workflows

21. How to make your research reproducible

22. How to make your research reproducible http://software-carpentry.org

23. https://twitter.com/ctitusbrown/status/383730337811296256

Repeat after me: Is our research reproducible (enough)?

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Repeat after me: Is our research reproducible (enough)?

Ähnlich wie Repeat after me: Is our research reproducible (enough)? (20)

Mehr von Lex Nederbragt

Mehr von Lex Nederbragt (13)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Repeat after me: Is our research reproducible (enough)?