Pharo DataFrame: Past, Present, and Future

E
ESUGESUG
Pharo DataFrame:
Past, Present, and Future
Larisa Safina, Oleksandr Zaitsev, Cyril Ferlicot-Delbecque,
Papa Ibrahima Sow
International Workshop on Smalltalk Technologies 2023, Lyon, France
Agenda
What is Data Frames and what they are good for
Evolution of Data Frames in Pharo
Data Frames outside of Pharo
2
Data Frames
Tabular structure
Columns with named headers
Columns/Rows index
Mixed data types in rows
Primitive data types
*The table is courtesy of Zaitsev et al (ESUG22)
3
Data Frames API
Data Import/Export
Grouping and Aggregation
Missing Data Handling
Statistical Operations
Time Series Analysis
Visualization
4
*The table is courtesy of Zaitsev et al (ESUG22)
Pharo DataFrame. Past
Started as a GSoC 2017 project (by Oleks)
Focus of the two followed-up GSoC
Contributions from external developers
5
Pharo DataFrame. Past
Problems:
- No stable maintenance
- Lack of functionality
- Low performance
- Incomplete coherence with Pharo collections
- Lack of detailed documentation
6
Pharo DataFrame. Present
Stable project with a permanent developer (Cyril)
Code optimization
- code quality
- speed and volume improvements
Adding new functionality
7
Awesome
Data Frames
8
Kyle Mitchell (jcmkk3) and
Uwe L. Korn (xhochy)
https://github.com/jcmkk3
/awesome-dataframes
Pharo
DataFrame
vs
Pandas
9
Pharo DataFrame. Future
✨ Better performance ✨
Functionality Enhancements
Better synchronisation with PolyMath and pharo-ai
Big Data Support
Evaluation ❤
Toy Story 1995
10
Do you want to contribute?
https://github.com/PolyMathOrg/DataFrame
11
12
Student project → Mature project with engineers
Better performance
Functionality Enhancements
Evaluation
1 von 12

Recomendados

External Master Data in Alfresco: Integrating and Keeping Metadata Consistent... von
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...
External Master Data in Alfresco: Integrating and Keeping Metadata Consistent...ITD Systems
2.2K views49 Folien
A Finnish perspective on FAIRsFAIR outputs von
A Finnish perspective on FAIRsFAIR outputsA Finnish perspective on FAIRsFAIR outputs
A Finnish perspective on FAIRsFAIR outputsJessica Parland-von Essen
249 views30 Folien
Science and data manipulation in Pharo von
Science and data manipulation in PharoScience and data manipulation in Pharo
Science and data manipulation in PharoESUG
64 views21 Folien
ALM Search Presentation for the VSS Arch Council von
ALM Search Presentation for the VSS Arch CouncilALM Search Presentation for the VSS Arch Council
ALM Search Presentation for the VSS Arch CouncilSunita Shrivastava
256 views25 Folien
Flink Apachecon Presentation von
Flink Apachecon PresentationFlink Apachecon Presentation
Flink Apachecon PresentationGyula Fóra
3.6K views45 Folien
Neo4j Vision and Roadmap von
Neo4j Vision and Roadmap Neo4j Vision and Roadmap
Neo4j Vision and Roadmap Neo4j
39 views31 Folien

Más contenido relacionado

Similar a Pharo DataFrame: Past, Present, and Future

Dalton Sergio Leonardo Eng Resume 20160803 von
Dalton Sergio Leonardo Eng Resume 20160803Dalton Sergio Leonardo Eng Resume 20160803
Dalton Sergio Leonardo Eng Resume 20160803Dalton Sergio Leonardo
220 views4 Folien
Overview of Modern Graph Analysis Tools von
Overview of Modern Graph Analysis ToolsOverview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis ToolsKeiichiro Ono
2.4K views35 Folien
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO von
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGOOpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGOOpenNebula Project
246 views7 Folien
Thinking About Guideline for Data Interoperability - Design concept and workf... von
Thinking About Guideline for Data Interoperability - Design concept and workf...Thinking About Guideline for Data Interoperability - Design concept and workf...
Thinking About Guideline for Data Interoperability - Design concept and workf...Open Cyber University of Korea
595 views9 Folien
SiLA: Making the standard fit for the future and adapting an open-source coll... von
SiLA: Making the standard fit for the future and adapting an open-source coll...SiLA: Making the standard fit for the future and adapting an open-source coll...
SiLA: Making the standard fit for the future and adapting an open-source coll...Gáspár Incze
225 views32 Folien
The AmeriFlux Network Data Management System von
The AmeriFlux Network Data Management SystemThe AmeriFlux Network Data Management System
The AmeriFlux Network Data Management SystemIntegrated Carbon Observation System (ICOS)
73 views20 Folien

Similar a Pharo DataFrame: Past, Present, and Future(20)

Overview of Modern Graph Analysis Tools von Keiichiro Ono
Overview of Modern Graph Analysis ToolsOverview of Modern Graph Analysis Tools
Overview of Modern Graph Analysis Tools
Keiichiro Ono2.4K views
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO von OpenNebula Project
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGOOpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO
OpenNebulaConf2017EU: FairShare Scheduling by Valentina Zaccolo, INDIGO
OpenNebula Project246 views
SiLA: Making the standard fit for the future and adapting an open-source coll... von Gáspár Incze
SiLA: Making the standard fit for the future and adapting an open-source coll...SiLA: Making the standard fit for the future and adapting an open-source coll...
SiLA: Making the standard fit for the future and adapting an open-source coll...
Gáspár Incze225 views
MuseoTorino, first italian project using a GraphDB, RDFa, Linked Open Data von 21Style
MuseoTorino, first italian project using a GraphDB, RDFa, Linked Open DataMuseoTorino, first italian project using a GraphDB, RDFa, Linked Open Data
MuseoTorino, first italian project using a GraphDB, RDFa, Linked Open Data
21Style1.4K views
Data Enthusiasts London: Scalable and Interoperable data services. Applied to... von Andy Petrella
Data Enthusiasts London: Scalable and Interoperable data services. Applied to...Data Enthusiasts London: Scalable and Interoperable data services. Applied to...
Data Enthusiasts London: Scalable and Interoperable data services. Applied to...
Andy Petrella1.3K views
Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d... von Dataconomy Media
Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d...Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d...
Andy Petrella_Med@Scale by Data Fellas: Scalable and Interoperable Genomics d...
Dataconomy Media582 views
Productive Development with APEX von Simon Boorsma
Productive Development with APEXProductive Development with APEX
Productive Development with APEX
Simon Boorsma856 views
2015 Data Science Summit @ dato Review von Hang Li
2015 Data Science Summit @ dato Review2015 Data Science Summit @ dato Review
2015 Data Science Summit @ dato Review
Hang Li976 views
System Architecture_BPM TIBCO_Network Inventory von Ela Pamieta
System Architecture_BPM TIBCO_Network InventorySystem Architecture_BPM TIBCO_Network Inventory
System Architecture_BPM TIBCO_Network Inventory
Ela Pamieta202 views
How to expand the Galaxy from genes to Earth in six simple steps (and live sm... von Raffaele Montella
How to expand the Galaxy from genes to Earth in six simple steps (and live sm...How to expand the Galaxy from genes to Earth in six simple steps (and live sm...
How to expand the Galaxy from genes to Earth in six simple steps (and live sm...
Raffaele Montella760 views
Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ... von Hirofumi Iwasaki
Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ...Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ...
Java EE 7 with Apache Spark for the World’s Largest Credit Card Core Systems ...
Hirofumi Iwasaki1.1K views
Apache Flink: Past, Present and Future von Gyula Fóra
Apache Flink: Past, Present and FutureApache Flink: Past, Present and Future
Apache Flink: Past, Present and Future
Gyula Fóra739 views
Tripal within the Arabidopsis Information Portal - PAG XXIII von Vivek Krishnakumar
Tripal within the Arabidopsis Information Portal - PAG XXIIITripal within the Arabidopsis Information Portal - PAG XXIII
Tripal within the Arabidopsis Information Portal - PAG XXIII
Vivek Krishnakumar1.2K views

Más de ESUG

Workshop: Identifying concept inventories in agile programming von
Workshop: Identifying concept inventories in agile programmingWorkshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programmingESUG
12 views16 Folien
Technical documentation support in Pharo von
Technical documentation support in PharoTechnical documentation support in Pharo
Technical documentation support in PharoESUG
31 views39 Folien
The Pharo Debugger and Debugging tools: Advances and Roadmap von
The Pharo Debugger and Debugging tools: Advances and RoadmapThe Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and RoadmapESUG
56 views44 Folien
Sequence: Pipeline modelling in Pharo von
Sequence: Pipeline modelling in PharoSequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in PharoESUG
86 views22 Folien
Migration process from monolithic to micro frontend architecture in mobile ap... von
Migration process from monolithic to micro frontend architecture in mobile ap...Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...ESUG
22 views35 Folien
Analyzing Dart Language with Pharo: Report and early results von
Analyzing Dart Language with Pharo: Report and early resultsAnalyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early resultsESUG
107 views30 Folien

Más de ESUG(20)

Workshop: Identifying concept inventories in agile programming von ESUG
Workshop: Identifying concept inventories in agile programmingWorkshop: Identifying concept inventories in agile programming
Workshop: Identifying concept inventories in agile programming
ESUG12 views
Technical documentation support in Pharo von ESUG
Technical documentation support in PharoTechnical documentation support in Pharo
Technical documentation support in Pharo
ESUG31 views
The Pharo Debugger and Debugging tools: Advances and Roadmap von ESUG
The Pharo Debugger and Debugging tools: Advances and RoadmapThe Pharo Debugger and Debugging tools: Advances and Roadmap
The Pharo Debugger and Debugging tools: Advances and Roadmap
ESUG56 views
Sequence: Pipeline modelling in Pharo von ESUG
Sequence: Pipeline modelling in PharoSequence: Pipeline modelling in Pharo
Sequence: Pipeline modelling in Pharo
ESUG86 views
Migration process from monolithic to micro frontend architecture in mobile ap... von ESUG
Migration process from monolithic to micro frontend architecture in mobile ap...Migration process from monolithic to micro frontend architecture in mobile ap...
Migration process from monolithic to micro frontend architecture in mobile ap...
ESUG22 views
Analyzing Dart Language with Pharo: Report and early results von ESUG
Analyzing Dart Language with Pharo: Report and early resultsAnalyzing Dart Language with Pharo: Report and early results
Analyzing Dart Language with Pharo: Report and early results
ESUG107 views
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6 von ESUG
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
Transpiling Pharo Classes to JS ECMAScript 5 versus ECMAScript 6
ESUG37 views
A Unit Test Metamodel for Test Generation von ESUG
A Unit Test Metamodel for Test GenerationA Unit Test Metamodel for Test Generation
A Unit Test Metamodel for Test Generation
ESUG53 views
Creating Unit Tests Using Genetic Programming von ESUG
Creating Unit Tests Using Genetic ProgrammingCreating Unit Tests Using Genetic Programming
Creating Unit Tests Using Genetic Programming
ESUG46 views
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes von ESUG
Threaded-Execution and CPS Provide Smooth Switching Between Execution ModesThreaded-Execution and CPS Provide Smooth Switching Between Execution Modes
Threaded-Execution and CPS Provide Smooth Switching Between Execution Modes
ESUG52 views
Exploring GitHub Actions through EGAD: An Experience Report von ESUG
Exploring GitHub Actions through EGAD: An Experience ReportExploring GitHub Actions through EGAD: An Experience Report
Exploring GitHub Actions through EGAD: An Experience Report
ESUG17 views
Pharo: a reflective language A first systematic analysis of reflective APIs von ESUG
Pharo: a reflective language A first systematic analysis of reflective APIsPharo: a reflective language A first systematic analysis of reflective APIs
Pharo: a reflective language A first systematic analysis of reflective APIs
ESUG57 views
Garbage Collector Tuning von ESUG
Garbage Collector TuningGarbage Collector Tuning
Garbage Collector Tuning
ESUG20 views
Improving Performance Through Object Lifetime Profiling: the DataFrame Case von ESUG
Improving Performance Through Object Lifetime Profiling: the DataFrame CaseImproving Performance Through Object Lifetime Profiling: the DataFrame Case
Improving Performance Through Object Lifetime Profiling: the DataFrame Case
ESUG43 views
thisContext in the Debugger von ESUG
thisContext in the DebuggerthisContext in the Debugger
thisContext in the Debugger
ESUG36 views
Websockets for Fencing Score von ESUG
Websockets for Fencing ScoreWebsockets for Fencing Score
Websockets for Fencing Score
ESUG18 views
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript von ESUG
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScriptShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ShowUs: PharoJS.org Develop in Pharo, Run on JavaScript
ESUG46 views
Advanced Object- Oriented Design Mooc von ESUG
Advanced Object- Oriented Design MoocAdvanced Object- Oriented Design Mooc
Advanced Object- Oriented Design Mooc
ESUG85 views
A New Architecture Reconciling Refactorings and Transformations von ESUG
A New Architecture Reconciling Refactorings and TransformationsA New Architecture Reconciling Refactorings and Transformations
A New Architecture Reconciling Refactorings and Transformations
ESUG28 views
BioSmalltalk von ESUG
BioSmalltalkBioSmalltalk
BioSmalltalk
ESUG415 views

Último

Short_Story_PPT.pdf von
Short_Story_PPT.pdfShort_Story_PPT.pdf
Short_Story_PPT.pdfutkarshsatishkumarsh
5 views16 Folien
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols von
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - DolsDSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - DolsDeltares
9 views23 Folien
Ports-and-Adapters Architecture for Embedded HMI von
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMIBurkhard Stubert
21 views19 Folien
Headless JS UG Presentation.pptx von
Headless JS UG Presentation.pptxHeadless JS UG Presentation.pptx
Headless JS UG Presentation.pptxJack Spektor
8 views24 Folien
MS PowerPoint.pptx von
MS PowerPoint.pptxMS PowerPoint.pptx
MS PowerPoint.pptxLitty Sylus
5 views14 Folien
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h... von
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...Deltares
9 views31 Folien

Último(20)

DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols von Deltares
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - DolsDSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols
DSD-INT 2023 European Digital Twin Ocean and Delft3D FM - Dols
Deltares9 views
Ports-and-Adapters Architecture for Embedded HMI von Burkhard Stubert
Ports-and-Adapters Architecture for Embedded HMIPorts-and-Adapters Architecture for Embedded HMI
Ports-and-Adapters Architecture for Embedded HMI
Burkhard Stubert21 views
Headless JS UG Presentation.pptx von Jack Spektor
Headless JS UG Presentation.pptxHeadless JS UG Presentation.pptx
Headless JS UG Presentation.pptx
Jack Spektor8 views
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h... von Deltares
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
DSD-INT 2023 Exploring flash flood hazard reduction in arid regions using a h...
Deltares9 views
360 graden fabriek von info33492
360 graden fabriek360 graden fabriek
360 graden fabriek
info33492122 views
Copilot Prompting Toolkit_All Resources.pdf von Riccardo Zamana
Copilot Prompting Toolkit_All Resources.pdfCopilot Prompting Toolkit_All Resources.pdf
Copilot Prompting Toolkit_All Resources.pdf
Riccardo Zamana10 views
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with... von sparkfabrik
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
20231129 - Platform @ localhost 2023 - Application-driven infrastructure with...
sparkfabrik7 views
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports von Ra'Fat Al-Msie'deen
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug ReportsBushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
BushraDBR: An Automatic Approach to Retrieving Duplicate Bug Reports
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated... von TomHalpin9
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...
Dev-HRE-Ops - Addressing the _Last Mile DevOps Challenge_ in Highly Regulated...
TomHalpin96 views
Navigating container technology for enhanced security by Niklas Saari von Metosin Oy
Navigating container technology for enhanced security by Niklas SaariNavigating container technology for enhanced security by Niklas Saari
Navigating container technology for enhanced security by Niklas Saari
Metosin Oy14 views
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx von animuscrm
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
2023-November-Schneider Electric-Meetup-BCN Admin Group.pptx
animuscrm15 views
.NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra... von Marc Müller
.NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra....NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra...
.NET Developer Conference 2023 - .NET Microservices mit Dapr – zu viel Abstra...
Marc Müller40 views
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ... von Donato Onofri
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...
Unmasking the Dark Art of Vectored Exception Handling: Bypassing XDR and EDR ...
Donato Onofri860 views
Airline Booking Software von SharmiMehta
Airline Booking SoftwareAirline Booking Software
Airline Booking Software
SharmiMehta6 views
Dapr Unleashed: Accelerating Microservice Development von Miroslav Janeski
Dapr Unleashed: Accelerating Microservice DevelopmentDapr Unleashed: Accelerating Microservice Development
Dapr Unleashed: Accelerating Microservice Development
Miroslav Janeski10 views
DSD-INT 2023 Process-based modelling of salt marsh development coupling Delft... von Deltares
DSD-INT 2023 Process-based modelling of salt marsh development coupling Delft...DSD-INT 2023 Process-based modelling of salt marsh development coupling Delft...
DSD-INT 2023 Process-based modelling of salt marsh development coupling Delft...
Deltares7 views

Pharo DataFrame: Past, Present, and Future

  • 1. Pharo DataFrame: Past, Present, and Future Larisa Safina, Oleksandr Zaitsev, Cyril Ferlicot-Delbecque, Papa Ibrahima Sow International Workshop on Smalltalk Technologies 2023, Lyon, France
  • 2. Agenda What is Data Frames and what they are good for Evolution of Data Frames in Pharo Data Frames outside of Pharo 2
  • 3. Data Frames Tabular structure Columns with named headers Columns/Rows index Mixed data types in rows Primitive data types *The table is courtesy of Zaitsev et al (ESUG22) 3
  • 4. Data Frames API Data Import/Export Grouping and Aggregation Missing Data Handling Statistical Operations Time Series Analysis Visualization 4 *The table is courtesy of Zaitsev et al (ESUG22)
  • 5. Pharo DataFrame. Past Started as a GSoC 2017 project (by Oleks) Focus of the two followed-up GSoC Contributions from external developers 5
  • 6. Pharo DataFrame. Past Problems: - No stable maintenance - Lack of functionality - Low performance - Incomplete coherence with Pharo collections - Lack of detailed documentation 6
  • 7. Pharo DataFrame. Present Stable project with a permanent developer (Cyril) Code optimization - code quality - speed and volume improvements Adding new functionality 7
  • 8. Awesome Data Frames 8 Kyle Mitchell (jcmkk3) and Uwe L. Korn (xhochy) https://github.com/jcmkk3 /awesome-dataframes
  • 10. Pharo DataFrame. Future ✨ Better performance ✨ Functionality Enhancements Better synchronisation with PolyMath and pharo-ai Big Data Support Evaluation ❤ Toy Story 1995 10
  • 11. Do you want to contribute? https://github.com/PolyMathOrg/DataFrame 11
  • 12. 12 Student project → Mature project with engineers Better performance Functionality Enhancements Evaluation