SlideShare ist ein Scribd-Unternehmen logo
1 von 23
Downloaden Sie, um offline zu lesen
Parallel Computing
      (with GPars)




                     NOV 2011
Evolution

● Old Days
  ○ threads
  ○ synchronize
  ○ locks
  ○ wait-notify
Evolution

● Recent enhancements
  ○ java.util.concurrent.*
  ○ Future task
  ○ Executor Pools
  ○ Blocking Queues
  ○ ConcurrentMap
Evolution

● Emerging Technologies
  ○ Map-Reduce
  ○ API's (Google Collections, JSR-166)
  ○ Non-blocking I/O
  ○ Parallel Friendly Languages (Scala, Clojure, Groovy)
Groovy Parallel Systems

High level abstractions for writing concurrent code!

 ● Code-level Helpers
    ○ Parallel Collections
    ○ Asynchronous Processing
    ○ Fork/Join - Map/Reduce

 ● Architecture-level Concepts
    ○ Dataflows
    ○ Actors

 ● Protecting Shared Mutable State
    ○ Agents
Groovy Parallel Systems

High level abstractions for writing concurrent code!

 ● Code-level Helpers
    ○ Parallel Collections
    ○ Asynchronous Processing
    ○ Fork/Join - Map/Reduce

 ● Architecture-level Concepts
    ○ Dataflows
    ○ Actors (For next time...)

 ● Protecting Shared Mutable State
    ○ Agents (For next time...)
DEMO 1
 (Parallel encryption)
Optimal Thread Pool Size




   Pool size = ???
Optimal Thread Pool Size




   Pool size = Ncores
Optimal Thread Pool Size




   Pool size = Ncores + 1
Optimal Thread Pool Size




   Pool size = Ncores + Block. Op. Ratio
Optimal Thread Pool Size




   Pool size = Ncores + wait time
                         cpu time
DEMO 2
"Dado un rango obtener la cantidad de
nr. primos agrupados por él ultimo digito"
Danger



 Don't mess up with mutable shared data!
DEMO 3
 (Facebook Search)
Collections
 ● eachParallel()
 ● eachWithIndexParallel()
 ● collectParallel()
 ● findAllParallel()
 ● findAnyParallel
 ● findParallel()
 ● everyParallel()
 ● anyParallel()
 ● grepParallel()
 ● groupByParallel()
 ● foldParallel()
 ● minParallel()
 ● maxParallel()
 ● sumParallel()
 ● splitParallel()
 ● countParallel()
 ● foldParallel()
DEMO 4
 (Facebook Singles)
Fork-Join




Map-Reduce
DEMO 5
 (API Calls)
DEMO 6
  (Multiget)
Multiget
+ Info
  ● http://gpars.org/0.12/guide/guide/single.html
  ● http://www.slideshare.net/paulk_asert/concurrency-with-
    gpars
  ● http://gee.cs.oswego.edu/dl/papers/fj.pdf
  ● http://static.googleusercontent.
    com/external_content/untrusted_dlcp/labs.google.
    com/en/us/papers/mapreduce-osdi04.pdf
  ● Java Concurrency in Practice (ISBN-0321349601)


Pics Credits
  ● Stone age: http://tinyurl.com/bvnlv74
  ● Steam train: http://tinyurl.com/6lp3krq
  ● Speed train: http://tinyurl.com/7jjc3v5
  ● CPU: http://www.flickr.com/photos/olivander/149865626/
https://github.com/pablomolnar/gpars_samples




                                    @pmolnar

Weitere ähnliche Inhalte

Was ist angesagt?

Lofar python meetup jan9 2013
Lofar python meetup jan9 2013Lofar python meetup jan9 2013
Lofar python meetup jan9 2013
Gijs Molenaar
 
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Eric Evans
 
証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議
Hiroki Mizuno
 

Was ist angesagt? (19)

[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator[COSCUP 2018] uTensor C++ Code Generator
[COSCUP 2018] uTensor C++ Code Generator
 
Your data isn't that big @ Big Things Meetup 2016-05-16
Your data isn't that big @ Big Things Meetup 2016-05-16Your data isn't that big @ Big Things Meetup 2016-05-16
Your data isn't that big @ Big Things Meetup 2016-05-16
 
Toy Model Overview
Toy Model OverviewToy Model Overview
Toy Model Overview
 
Compiler basics: lisp to assembly
Compiler basics: lisp to assemblyCompiler basics: lisp to assembly
Compiler basics: lisp to assembly
 
.NET Memory Primer (Martin Kulov)
.NET Memory Primer (Martin Kulov).NET Memory Primer (Martin Kulov)
.NET Memory Primer (Martin Kulov)
 
OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel
OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel
OSDC 2019 | Fast log management for your infrastructure by Nicolas Frankel
 
Fantastic caches and where to find them
Fantastic caches and where to find themFantastic caches and where to find them
Fantastic caches and where to find them
 
Lofar python meetup jan9 2013
Lofar python meetup jan9 2013Lofar python meetup jan9 2013
Lofar python meetup jan9 2013
 
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)Time Series Data with Apache Cassandra (ApacheCon EU 2014)
Time Series Data with Apache Cassandra (ApacheCon EU 2014)
 
Syslog-ng Performance
Syslog-ng Performance Syslog-ng Performance
Syslog-ng Performance
 
Всеволод Поляков (DevOps Team Lead в Grammarly)
Всеволод Поляков (DevOps Team Lead в Grammarly)Всеволод Поляков (DevOps Team Lead в Grammarly)
Всеволод Поляков (DevOps Team Lead в Grammarly)
 
bup backup system (2011-04)
bup backup system (2011-04)bup backup system (2011-04)
bup backup system (2011-04)
 
Linux-Permission
Linux-PermissionLinux-Permission
Linux-Permission
 
証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議証明駆動開発のたのしみ@名古屋reject会議
証明駆動開発のたのしみ@名古屋reject会議
 
“Show Me the Garbage!”, Understanding Garbage Collection
“Show Me the Garbage!”, Understanding Garbage Collection“Show Me the Garbage!”, Understanding Garbage Collection
“Show Me the Garbage!”, Understanding Garbage Collection
 
High Performance Systems Without Tears - Scala Days Berlin 2018
High Performance Systems Without Tears - Scala Days Berlin 2018High Performance Systems Without Tears - Scala Days Berlin 2018
High Performance Systems Without Tears - Scala Days Berlin 2018
 
Is It Faster to Go with Redpanda Transactions than Without Them?!
Is It Faster to Go with Redpanda Transactions than Without Them?!Is It Faster to Go with Redpanda Transactions than Without Them?!
Is It Faster to Go with Redpanda Transactions than Without Them?!
 
Restinio - header-only http and websocket server
Restinio - header-only http and websocket serverRestinio - header-only http and websocket server
Restinio - header-only http and websocket server
 
JSON's big problem android_taipei_201709
JSON's big problem android_taipei_201709JSON's big problem android_taipei_201709
JSON's big problem android_taipei_201709
 

Ähnlich wie Parallel computing with GPars

Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)
Geoffrey De Smet
 
NetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapNetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmap
Ruslan Meshenberg
 

Ähnlich wie Parallel computing with GPars (20)

The Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization OpportunitiesThe Parquet Format and Performance Optimization Opportunities
The Parquet Format and Performance Optimization Opportunities
 
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
“Show Me the Garbage!”, Garbage Collection a Friend or a Foe
 
Distributed real time stream processing- why and how
Distributed real time stream processing- why and howDistributed real time stream processing- why and how
Distributed real time stream processing- why and how
 
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
Ehtsham Elahi, Senior Research Engineer, Personalization Science and Engineer...
 
Real-time analytics with Druid at Appsflyer
Real-time analytics with Druid at AppsflyerReal-time analytics with Druid at Appsflyer
Real-time analytics with Druid at Appsflyer
 
Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)Developing applications with rules, workflow and event processing (it@cork 2010)
Developing applications with rules, workflow and event processing (it@cork 2010)
 
Journey through high performance django application
Journey through high performance django applicationJourney through high performance django application
Journey through high performance django application
 
SystemML - Datapalooza Denver - 05.17.16 MWD
SystemML - Datapalooza Denver - 05.17.16 MWDSystemML - Datapalooza Denver - 05.17.16 MWD
SystemML - Datapalooza Denver - 05.17.16 MWD
 
Scio - A Scala API for Google Cloud Dataflow & Apache Beam
Scio - A Scala API for Google Cloud Dataflow & Apache BeamScio - A Scala API for Google Cloud Dataflow & Apache Beam
Scio - A Scala API for Google Cloud Dataflow & Apache Beam
 
Stripe CTF3 wrap-up
Stripe CTF3 wrap-upStripe CTF3 wrap-up
Stripe CTF3 wrap-up
 
Handout: 'Open Source Tools & Resources'
Handout: 'Open Source Tools & Resources'Handout: 'Open Source Tools & Resources'
Handout: 'Open Source Tools & Resources'
 
Flink Forward SF 2017: Kenneth Knowles - Back to Sessions overview
Flink Forward SF 2017: Kenneth Knowles - Back to Sessions overviewFlink Forward SF 2017: Kenneth Knowles - Back to Sessions overview
Flink Forward SF 2017: Kenneth Knowles - Back to Sessions overview
 
Sorry - How Bieber broke Google Cloud at Spotify
Sorry - How Bieber broke Google Cloud at SpotifySorry - How Bieber broke Google Cloud at Spotify
Sorry - How Bieber broke Google Cloud at Spotify
 
Developing games and graphic visualizations in Pascal
Developing games and graphic visualizations in PascalDeveloping games and graphic visualizations in Pascal
Developing games and graphic visualizations in Pascal
 
MacGyver Learns Spark
MacGyver Learns SparkMacGyver Learns Spark
MacGyver Learns Spark
 
NetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmapNetflixOSS meetup lightning talks and roadmap
NetflixOSS meetup lightning talks and roadmap
 
Machine learning at Scale with Apache Spark
Machine learning at Scale with Apache SparkMachine learning at Scale with Apache Spark
Machine learning at Scale with Apache Spark
 
Gpu workshop cluster universe: scripting cuda
Gpu workshop cluster universe: scripting cudaGpu workshop cluster universe: scripting cuda
Gpu workshop cluster universe: scripting cuda
 
GSoC2014 - Uniritter Presentation May, 2015
GSoC2014 - Uniritter Presentation May, 2015GSoC2014 - Uniritter Presentation May, 2015
GSoC2014 - Uniritter Presentation May, 2015
 
24 uses for perl6
24 uses for perl624 uses for perl6
24 uses for perl6
 

Kürzlich hochgeladen

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Kürzlich hochgeladen (20)

Cyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdfCyberprint. Dark Pink Apt Group [EN].pdf
Cyberprint. Dark Pink Apt Group [EN].pdf
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 

Parallel computing with GPars

  • 1. Parallel Computing (with GPars) NOV 2011
  • 2. Evolution ● Old Days ○ threads ○ synchronize ○ locks ○ wait-notify
  • 3. Evolution ● Recent enhancements ○ java.util.concurrent.* ○ Future task ○ Executor Pools ○ Blocking Queues ○ ConcurrentMap
  • 4. Evolution ● Emerging Technologies ○ Map-Reduce ○ API's (Google Collections, JSR-166) ○ Non-blocking I/O ○ Parallel Friendly Languages (Scala, Clojure, Groovy)
  • 5. Groovy Parallel Systems High level abstractions for writing concurrent code! ● Code-level Helpers ○ Parallel Collections ○ Asynchronous Processing ○ Fork/Join - Map/Reduce ● Architecture-level Concepts ○ Dataflows ○ Actors ● Protecting Shared Mutable State ○ Agents
  • 6. Groovy Parallel Systems High level abstractions for writing concurrent code! ● Code-level Helpers ○ Parallel Collections ○ Asynchronous Processing ○ Fork/Join - Map/Reduce ● Architecture-level Concepts ○ Dataflows ○ Actors (For next time...) ● Protecting Shared Mutable State ○ Agents (For next time...)
  • 7. DEMO 1 (Parallel encryption)
  • 8. Optimal Thread Pool Size Pool size = ???
  • 9. Optimal Thread Pool Size Pool size = Ncores
  • 10. Optimal Thread Pool Size Pool size = Ncores + 1
  • 11. Optimal Thread Pool Size Pool size = Ncores + Block. Op. Ratio
  • 12. Optimal Thread Pool Size Pool size = Ncores + wait time cpu time
  • 13. DEMO 2 "Dado un rango obtener la cantidad de nr. primos agrupados por él ultimo digito"
  • 14. Danger Don't mess up with mutable shared data!
  • 15. DEMO 3 (Facebook Search)
  • 16. Collections ● eachParallel() ● eachWithIndexParallel() ● collectParallel() ● findAllParallel() ● findAnyParallel ● findParallel() ● everyParallel() ● anyParallel() ● grepParallel() ● groupByParallel() ● foldParallel() ● minParallel() ● maxParallel() ● sumParallel() ● splitParallel() ● countParallel() ● foldParallel()
  • 17. DEMO 4 (Facebook Singles)
  • 19. DEMO 5 (API Calls)
  • 20. DEMO 6 (Multiget)
  • 22. + Info ● http://gpars.org/0.12/guide/guide/single.html ● http://www.slideshare.net/paulk_asert/concurrency-with- gpars ● http://gee.cs.oswego.edu/dl/papers/fj.pdf ● http://static.googleusercontent. com/external_content/untrusted_dlcp/labs.google. com/en/us/papers/mapreduce-osdi04.pdf ● Java Concurrency in Practice (ISBN-0321349601) Pics Credits ● Stone age: http://tinyurl.com/bvnlv74 ● Steam train: http://tinyurl.com/6lp3krq ● Speed train: http://tinyurl.com/7jjc3v5 ● CPU: http://www.flickr.com/photos/olivander/149865626/