Meta-analysis of 93 research papers reporting findings based on mining GitHub repositories. We report concerns on several aspects: dataset collection, replicability, sampling techniques and methodologies.

1. Valerio Cosentino, Javier L. Cánovas Izquierdo, Jordi Cabot
Flickr/BenNuttall

2. Motivation

3. Motivation

4. Motivation
Empirical
Methods
Employed
Dataset
Used
Limitations
Reported
Methodology

5. Motivation
Empirical
Methods
Employed
Dataset
Used
Limitations
Reported
Methodology
Discussion

6. Methodology

7. Methodology
Title || Abstract || Keywords || IndexTerms
INCLUDES
“GitHub” OR “Git hub” OR “github”

8. Results
Flickr/mararle

9. Empirical Methods Employed

10. Datasets Used

11. Datasets Used

12. Datasets Used

13. Limitations Reported

14. Limitations Reported

15. Discussion
Flickr/KristinaAlexanderson

16. Discussion
Data Collection
Dataset Size
Replicability
Sampling
Longitudinal
Studies
Variety of
methodologies

17. Discussion
Data Collection
Dataset Size
Replicability
Sampling
Longitudinal
Studies
Variety of
methodologies
Freshness vs. Curation
Small-medium size
> 2/3 not providing dataset access
Most use non-probaility sampling
Scarcely used
Replication? Comparisons?

18. Flickr/JimRafferty
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International license.
Thanks!
http://tinyurl.com/GitHub-SystRev-Papers
Some works might
have been ignored
Subjetivity issues

19. Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International license.
Discussion
Data
Collection
Dataset Size
Replicability
Sampling
Longitudinal
Studies
Variety of
methodologies
Thanks!
http://tinyurl.com/GitHub-SystRev-Papers