RAMP: Collaborative challenge with code submission
•
1 gefällt mir•570 views
We describe the RAMP, a rapid data challenge format and tool we developed at the Paris-Saclay Center for Data Science.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Andere mochten auch
Andere mochten auch (13)
Ähnlich wie RAMP: Collaborative challenge with code submission
Ähnlich wie RAMP: Collaborative challenge with code submission (20)
Mehr von Balázs Kégl
Mehr von Balázs Kégl (8)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
RAMP: Collaborative challenge with code submission
- 1. Center for Data Science Paris-Saclay1 CNRS & University Paris Saclay TektosData BALÁZS KÉGL RAPID ANALYTICS AND MODEL PROTOTYPING (RAMP) COLLABORATIVE CHALLENGE WITH CODE SUBMISSION
- 2. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay2 A bit of context
- 3. Center for Data Science Paris-Saclay3 UNIVERSITÉ PARIS-SACLAY 19 founding partners
- 4. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay4 UNIVERSITÉ PARIS-SACLAY + horizontal multi-disciplinary and multi-partner initiatives to create cohesion
- 5. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay5 Center for Data Science Paris-Saclay A multi-disciplinary initiative to define, structure, and manage the data science ecosystem at the Université Paris-Saclay http://www.datascience-paris-saclay.fr/ Biology & bioinformatics IBISC/UEvry LRI/UPSud Hepatinov CESP/UPSud-UVSQ-Inserm IGM-I2BC/UPSud MIA/Agro MIAj-MIG/INRA LMAS/Centrale Chemistry EA4041/UPSud Earth sciences LATMOS/UVSQ GEOPS/UPSud IPSL/UVSQ LSCE/UVSQ LMD/Polytechnique Economy LM/ENSAE RITM/UPSud LFA/ENSAE Neuroscience UNICOG/Inserm U1000/Inserm NeuroSpin/CEA Particle physics astrophysics & cosmology LPP/Polytechnique DMPH/ONERA CosmoStat/CEA IAS/UPSud AIM/CEA LAL/UPSud 250researchers in 35laboratories Machine learning LRI/UPSud LTCI/Telecom CMLA/Cachan LS/ENSAE LIX/Polytechnique MIA/Agro CMA/Polytechnique LSS/Supélec CVN/Centrale LMAS/Centrale DTIM/ONERA IBISC/UEvry Visualization INRIA LIMSI Signal processing LTCI/Telecom CMA/Polytechnique CVN/Centrale LSS/Supélec CMLA/Cachan LIMSI DTIM/ONERA Statistics LMO/UPSud LS/ENSAE LSS/Supélec CMA/Polytechnique LMAS/Centrale MIA/AgroParisTech machine learning information retrieval signal processing data visualization databases Domain science human society life brain earth universe Tool building software engineering clouds/grids high-performance computing optimization Domain scientistSoftware engineer datascience-paris-saclay.fr LIST/CEA
- 6. Center for Data Science Paris-Saclay Data domains energy and physical sciences health and life sciences Earth and environment economy and society brain Data scientist Data trainer Applied scientist Domain scientistSoftware engineer Data engineer Data science statistics machine learning information retrieval signal processing data visualization databases Tool building software engineering clouds/grids high-performance computing optimization • (The lack of) manpower • especially at the interfaces • industrial brain-drain • Incentives • data scientists are not incentivized to work on domain science • scientists are not incentivized to work on tools • Access • no well-developed channels to identify the right experts for a given problem • Tools • few tools that can help domain scientists and data scientists to collaborate efficiently 6 CHALLENGES https://medium.com/@balazskegl
- 7. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay7 TWO ANALYTICS TOOLS FOR INITIATING DOMAIN-DATA SCIENCE INTERACTIONS RAPID ANALYTICS AND MODEL PROTOTYPING (RAMP) DATA CHALLENGES
- 8. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay8 DATA CHALLENGES
- 9. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay9 DATA CHALLENGES • The HiggsML challenge on Kaggle • https://www.kaggle.com/c/higgs-boson
- 10. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay10 HUGE PUBLICITY CLASSIFICATION FOR DISCOVERY 14
- 11. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay11 SIGNIFICANT IMPROVEMENT OVER THE BASELINE CLASSIFICATION FOR DISCOVERY 15
- 12. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay12 HUGE PUBLICITY SIGNIFICANT IMPROVEMENT OVER THE BASELINE yet partially missing the objectives
- 13. Center for Data Science Paris-Saclay • Challenges are useful for • generating visibility in the data science community about novel application domains • benchmarking in a fair way state-of-the-art techniques on well- defined problems • finding talented data scientists • Limitations • not necessary adapted to solving complex and open-ended data science problems in realistic environments • no direct access to solutions and data scientist • emphasizes competition 13 DATA CHALLENGES
- 14. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay14 HUGE PUBLICITY We decided to design something better
- 15. Center for Data Science Paris-Saclay • Prototyping • Training • Human resources • Collaboration building, networking • Social science observatory 15 RAPID ANALYTICS AND MODEL PROTOTYPING (RAMP)
- 16. Center for Data Science Paris-Saclay RAMPS 16 • Single-day coding sessions • 20-40 participants • preparation is similar to challenges • Goals • focusing and motivating top talents • promoting collaboration, speed, and efficiency • solving (prototyping) real problems
- 17. Center for Data Science Paris-Saclay17 ANALYTICS TOOLS TO PROMOTE COLLABORATION AND CODE REUSE
- 18. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay18 ANALYTICS TOOL TO PROMOTE COLLABORATION AND CODE REUSE
- 19. Center for Data Science Paris-Saclay RAMPS 19 www.ramp.studio software + management backend is open source: https://github.com/camillemarini/datarun
- 20. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay 2015 Jan 15 The HiggsML challenge 20 RAPID ANALYTICS AND MODEL PROTOTYPING
- 21. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay 2015 Apr 10 Classifying variable stars 21 RAPID ANALYTICS AND MODEL PROTOTYPING
- 22. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay22 VARIABLE STARS
- 23. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay23 VARIABLE STARS accuracy improvement: 89% to 96%
- 24. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay 2015 June 16 and Sept 26 Predicting El Nino 24 RAPID ANALYTICS AND MODEL PROTOTYPING
- 25. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay25 RMSE improvement: 0.9˚C to 0.4˚C RAPID ANALYTICS AND MODEL PROTOTYPING
- 26. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay26 2015 October 8 Insect classification RAPID ANALYTICS AND MODEL PROTOTYPING
- 27. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay27 accuracy improvement: 30% to 70% RAPID ANALYTICS AND MODEL PROTOTYPING
- 28. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay28 2016 February 10 Macroeconomic agent-based models RAPID ANALYTICS AND MODEL PROTOTYPING
- 29. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay29 f1-score improvement: 0.57 to 0.63 RAPID ANALYTICS AND MODEL PROTOTYPING
- 30. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay30 2016 February 13 Epidemium cancer survival rate RAPID ANALYTICS AND MODEL PROTOTYPING
- 31. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay31 RMSE improvement: 3000 to 300 RAPID ANALYTICS AND MODEL PROTOTYPING
- 32. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay32 2016 May 11 Drug identification from spectra RAPID ANALYTICS AND MODEL PROTOTYPING
- 33. Center for Data Science Paris-Saclay Center for Data Science Paris-Saclay33 Drug identification error improvement: 9% to 3% Drug concentration accuracy improvement: 20% to 12% RAPID ANALYTICS AND MODEL PROTOTYPING
- 34. Center for Data Science Paris-Saclay • Fast development of analytics solutions • Teaching support • Networking and HR support • Support for collaborative team work • Commercialized through TektosData 34 THE RAMP TOOL A prototyping tool for collaborative development of data science workflows
- 35. Center for Data Science Paris-Saclay • We have a cool tool for collaborative data analytics • designing workflows beyond scikit-learn predictors • Data management/munging is a big part of the data analytics workflow, we need tools • preparing a RAMP takes two weeks to six months • Big data is rare: our problems are more about flexible organization of heterogeneous data 35 TAKE HOME MESSAGES
- 36. Center for Data Science Paris-Saclay36 THANK YOU!