Dataflow: the concurrency/parallelism architecture you need

•

1 gefällt mir•1,182 views

This document discusses dataflow architecture. It defines dataflow as an architecture where data flows between operators through channels. Each operator executes only in response to data arriving on its input channels. The document discusses how dataflow allows for concurrency and parallelism. It provides examples of dataflow frameworks in Scala, Akka, and Java and notes their limitations. GPars is highlighted as a framework that can create true dataflow networks by using DataflowQueue. The document uses calculating mean and standard deviation as a problem to demonstrate coding in dataflow.

Technologie

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow:
Russel Winder
@russel_winder
http://www.russel.org.uk
russel@winder.org.uk
The Concurrency/Parallelism
Architecture You Need

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
What is Dataflow?

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
What are (in computing†
):
Concurrency:
Structuring solution and code
such that multiple parts may
execute independently and
possibly even at the same
time.
Parallelism:
Execute multiple parts of a
system at the same time on
different processors so as to
get things working faster.
†
In natural language these words have very different meanings.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
What is Dataflow?
An architecture comprising channels allowing data to flow from
one operator to another, where each operator has multiple
input channels and multiple output channels, and executes
code only in response to the arrival of data on the inputs.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Historically
Dataflow computers:
– Values flowing between…
–…operators that calculate…
–…new values to pass to…
–…other operators.
Dataflow hardware didn't take
off, but the architecture works
at various scales.
The Manchester Prototype Dataflow Computer
J R Gurd, C C Kirkham, I Watson
CACM 28(1), 1985-01.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow diagrams have been an
integral part of analysis and design of
information systems since the 1970s
T de Marco, Structured Analysis and Systems Specification,
Yourdon Press, NY, 1978.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow and Functional
Operators seem like they
might be pure functions,
but…
…they are not necessarily,
operators may have internal
state.
Operators may be referentially
transparent, but they may be
not.
Operators may even have side
effects.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow is an
event-based
architecture

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow systems are
(possibly)
reactive systems.
Which would make them exceedingly
trendy even if the idea is very old.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow systems have
no†
shared memory.
†
or at least should have no.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
operator
channel

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow systems are
message passing systems.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Each operator must†
be single threaded.
†
or at least should.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow Frameworks
Scala:
–Future
Akka:
–Dataflow variables, aka
Promise
–Deprecated in favour of Async
Java:
–Pre-8, Future
–8+, CompletableFuture, aka
Promise

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Architectural Issue
Each of the aforementioned frameworks assumes that each
operator creates a single value. Communication is by dataflow
variables: each dataflow variable is a thread-safe single
assignment variable.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
GPars…
Has dataflow variables
(promises) and tasks and so
can do everything Akka and
Java can offer.
Has DataflowQueue, and so
can create real dataflow
networks.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
One does like to code…
…doesn't one.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
We need a problem…

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
A Problem
Calculate mean and standard deviation of a data sample.
¯x =
1
n
∑i=0
n
xi
s =
√ 1
n−1
∑i=0
n
(xi−¯x)2

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Amend the Problem
s =
√ 1
n−1 ((∑i=0
n
xi
2
)−n¯x ¯x)
¯x =
1
n
∑i=0
n
xi

@YourTwitterHandle@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
C
od
e

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Switch to using an IDE for this.Switch to using an IDE for this.
Code Example

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Summary
Dataflow is an architecture:
Event-driven, single-threaded
operators communicating by
message passing using
channels.
Dataflow is an easement:
Synchronization is inherent in
the model, and there is no
shared memory, so all
deadlocks are trivial.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow is a way of harnessing
concurrency and parallelism
in easy to program ways.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
GPars is usable from Java
as well as Groovy.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Testing is really Groovy with Spock.

@russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder
Dataflow is an architecture of
code you need to know.

Weitere ähnliche Inhalte

Ähnlich wie Dataflow: the concurrency/parallelism architecture you need

Dataflow, the Forgotten Way - Russel Winder

JAXLondon2014

GPars Remoting

Russel Winder

Shared memory multi-threading is an operating system technique which has no place in tool and application programming. Actors, dataflows, Communicating Sequential Processes (CSP): these are the tools for application and tool construction in this increasingly multiple multicore processor world. The Java platform has some useful tools in java.util.concurrent, but it is still focused on lower-level layers of abstraction. GPars builds on these and brings actors, dataflows, CSP, agents, etc. to people working with Java and Groovy. In this session we will look at various examples of how to use these tools to easily create concurrent and parallel programs without all the fears of deadlock and livelock inherent in shared memory multi-threaded code.

It's all about processes communicating - Russel Winder

JAX London

It's All About Processes Communicating

Russel Winder

RTI Connext 5.1.0

Jan Van Bruaene

Njug presentation

iwrigley

The Case for Kotlin and Ceylon

Russel Winder

Apache Cassandra is the leading distributed database in use at thousands of sites with the world’s most demanding scalability and availability requirements. Apache Spark is a distributed data analytics computing framework that has gained a lot of traction in processing large amounts of data in an efficient and user-friendly manner. The joining of both provides a powerful combination of real-time data collection with analytics. After a brief overview of Cassandra and Spark, this class will dive into various aspects of the integration.

Big Data Analytics with Spark

DataStax Academy

View the recording of the meet up, including the live demos, here: https://www.youtube.com/watch?v=uaJWB3K8lkg Data science holds tremendous potential for organizations to uncover new insights and drivers of revenue and profitability. Big Data has brought the promise of doing data science at scale to enterprises, however this promise also comes with challenges for data scientists to continuously learn and collaborate. Data Scientists have many tools at their disposal such as notebooks like Juypter and Apache Zeppelin & IDEs such as RStudio with languages like R, Python, Scala and frameworks like Apache Spark. Given all the choices how do you best collaborate to build your model and then work through the development lifecycle to deploy it from test into production? Why Data Science on Big Data? In this meetup you will cover the attributes of a modern data science platform that empowers data scientists to build models using all the data in their data lake and foster continuous learning and collaboration. We will show a demo of Apache Zeppelin, Apache Spark, Apache Livy and Apache Hadoop with the focus on integration, security and model deployment and management. Data Science at Scale DEMO The demo will cover the Data Science life cycle: develop model in team environment, train the model with all the data on a Hadoop cluster, deploy model into production. The model will be a Spark ML model Practical ML with Apache Spark To deliver machine learning solutions data scientists not only need to fit models but also do familiar tasks data collection & wrangling, labelling, feature extraction and transformation, model tuning and evaluation, etc. Apache Spark provide provides a unified solution for all this under the same framework. For example, one can use Spark SQL to generate training data from different sources and then pass it directly to MLlib for feature engineering and model tuning, instead of using Hive/Pig for the first half and then downloading the data to a single machine to train models in R. The latter is actually very common in practice but painful to maintain. Spark MLlib makes life easier for data scientists and machine learning engineers so that they can focus on building better ML models and applications. We will discuss the underlying principles required to develop practical machine learning and data science pipelines and show some hands-on experience using Apache Spark to solve typical machine learning and data science problem. We will also have a short discussion about how Spark MLlib faces challenges from other machine learning libraries such as TensorFlow and XGBoost.

Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017

Hortonworks

Analyzing Hadoop Data Using Sparklyr 

Cloudera, Inc.

Proxy Deep Dive Voxxed Belgrad 2015

Sven Ruppert

Everybody knows the pattern proxy, but how can you use it effectively? What kind of proxy patterns are available, and how can you build patterns more effectively with it? Why is reflection needed for this? Importantly, we need only the core JDK in most cases. This tutorial starts from the basics and continues on to DynamicProxies, DynamicObjectAdapter and DynamicStaticProxies at runtime, StaticObjectAdapters, and more. The session, based on the German book Dynamic Proxies, by Heinz Kabutz and the session’s presenter, takes a deep dive into this pattern group.

Proxy deep-dive java-one_20151027_001

Sven Ruppert

Introduction to Hadoop and Cloudera, Louisville BI & Big Data Analytics Meetup

iwrigley

GPars Workshop

Russel Winder

( Apache Spark Training: https://www.edureka.co/apache-spark-scala-training ) ( Hadoop Training: https://www.edureka.co/hadoop ) This Edureka Hadoop vs Spark video will help you to understand the differences between Hadoop and Spark. We will be comparing them on various parameters. We will be taking a broader look at: 1. Introduction to Hadoop 2. Introduction to Apache Spark 3. Spark vs Hadoop - Performance Ease of Use Cost Data Processing Fault tolerance Security 4. Hadoop Use-cases 5. Spark Use-cases

Hadoop vs Spark | Which One to Choose? | Hadoop Training | Spark Training | E...

Edureka!

ACCU 2012: Go, D, C++ and The Multicore Revolution

Russel Winder

Rust, Redis, and Protobuf - Oh My!

Nell Shamrell-Harrington

Big Data Infrastructure

Trivadis

Empowering Agile Development with Containers

Kellyn Pot'Vin-Gorman

Applications on Hadoop

markgrover

Ähnlich wie Dataflow: the concurrency/parallelism architecture you need (20)

Dataflow, the Forgotten Way - Russel Winder

GPars Remoting

It's all about processes communicating - Russel Winder

It's All About Processes Communicating

RTI Connext 5.1.0

Njug presentation

The Case for Kotlin and Ceylon

Big Data Analytics with Spark

Enterprise Data Science at Scale Meetup - IBM and Hortonworks - Oct 2017

Analyzing Hadoop Data Using Sparklyr 

Proxy Deep Dive Voxxed Belgrad 2015

Proxy deep-dive java-one_20151027_001

Introduction to Hadoop and Cloudera, Louisville BI & Big Data Analytics Meetup

GPars Workshop

Hadoop vs Spark | Which One to Choose? | Hadoop Training | Spark Training | E...

ACCU 2012: Go, D, C++ and The Multicore Revolution

Rust, Redis, and Protobuf - Oh My!

Big Data Infrastructure

Empowering Agile Development with Containers

Applications on Hadoop

Mehr von Russel Winder

On Concurrency and Parallelism in the JVMverse

Russel Winder

On the Architectures of Microservices: the next layer

Russel Winder

Fast Python? Don't Bother

Russel Winder

Making Python computations fast

Russel Winder

Tales from the Workshops

Russel Winder

Making Computations Execute Very Quickly

Russel Winder

Java is Dead, Long Live Ceylon, Kotlin, etc

Russel Winder

Java is dead, long live Scala, Kotlin, Ceylon, etc.

Russel Winder

Spocktacular testing

Russel Winder

Spocktacular Testing

Russel Winder

Is Groovy static or dynamic

Russel Winder

Java is dead, long live Scala Kotlin Ceylon etc.

Russel Winder

Are Go and D threats to Python

Russel Winder

Is Groovy as fast as Java

Russel Winder

Who needs C++ when you have D and Go

Russel Winder

Java 8: a New Beginning

Russel Winder

Why Go is an important programming language

Russel Winder

GPars: Groovy Parallelism for Java

Russel Winder

GroovyFX: or how to program JavaFX easily

Russel Winder

Switch to Python 3…now…immediately

Russel Winder

Mehr von Russel Winder (20)

On Concurrency and Parallelism in the JVMverse

On the Architectures of Microservices: the next layer

Fast Python? Don't Bother

Making Python computations fast

Tales from the Workshops

Making Computations Execute Very Quickly

Java is Dead, Long Live Ceylon, Kotlin, etc

Java is dead, long live Scala, Kotlin, Ceylon, etc.

Spocktacular testing

Spocktacular Testing

Is Groovy static or dynamic

Java is dead, long live Scala Kotlin Ceylon etc.

Are Go and D threats to Python

Is Groovy as fast as Java

Who needs C++ when you have D and Go

Java 8: a New Beginning

Why Go is an important programming language

GPars: Groovy Parallelism for Java

GroovyFX: or how to program JavaFX easily

Switch to Python 3…now…immediately

Kürzlich hochgeladen

Created by Mozilla Research in 2012 and now part of Linux Foundation Europe, the Servo project is an experimental rendering engine written in Rust. It combines memory safety and concurrency to create an independent, modular, and embeddable rendering engine that adheres to web standards. Stewardship of Servo moved from Mozilla Research to the Linux Foundation in 2020, where its mission remains unchanged. After some slow years, in 2023 there has been renewed activity on the project, with a roadmap now focused on improving the engine’s CSS 2 conformance, exploring Android support, and making Servo a practical embeddable rendering engine. In this presentation, Rakhi Sharma reviews the status of the project, our recent developments in 2023, our collaboration with Tauri to make Servo an easy-to-use embeddable rendering engine, and our plans for the future to make Servo an alternative web rendering engine for the embedded devices industry. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://ossna2024.sched.com/event/1aBNF/a-year-of-servo-reboot-where-are-we-now-rakhi-sharma-igalia

A Year of the Servo Reboot: Where Are We Now?

Igalia

Finology Group – Insurtech Innovation Award 2024

The Digital Insurer

Histor y of HAM Radio presentation slide

vu2urc

[2024]Digital Global Overview Report 2024 Meltwater.pdf

hans926745

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

BooK Now Call us at +918448380779 to hire a gorgeous and seductive call girl for sex. Take a Delhi Escort Service. The help of our escort agency is mostly meant for men who want sexual Indian Escorts In Delhi NCR. It should be noted that any impersonator will get 100 attention from our Young Girls Escorts in Delhi. They will assume the position of reliable allies. VIP Call Girl With Original Photos Book Tonight +918448380779 Our Cheap Price 1 Hour not available 2 Hours 5000 Full Night 8000 TAG: Call Girls in Delhi, Noida, Gurgaon, Ghaziabad, Connaught Place, Greater Kailash Delhi, Lajpat Nagar Delhi, Mayur Vihar Delhi, Chanakyapuri Delhi, New Friends Colony Delhi, Majnu Ka Tilla, Karol Bagh, Malviya Nagar, Saket, Khan Market, Noida Sector 18, Noida Sector 76, Noida Sector 51, Gurgaon Mg Road, Iffco Chowk Gurgaon, Rajiv Chowk Gurgaon All Delhi Ncr Free Home Deliver

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

Delhi Call girls

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

CNv6 Instructor Chapter 6 Quality of Service

giselly40

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Delhi Call girls

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

08448380779 Call Girls In Friends Colony Women Seeking Men

Delhi Call girls

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Neo4j

Microsoft's Threat Matrix for Kubernetes helps organizations understand the attack surface a Kubernetes deployment introduces to their environments. This ensures that adequate detections and mitigations are in place. By covering over 40 different attacker techniques, defenders can learn about Kubernetes-specific mitigations and controls to deploy to their environments. In this session, we will explore the MS-TA9013 Host Path Mount technique, which is commonly used by attackers to perform privilege escalation in a Kubernetes cluster. Attendees will learn how attackers and defenders can: * Escape the container's host volume mount to gain persistence on an underlying node * Move laterally from the underlying node into the customer's cloud environment * Analyze Kubernetes audit logs to detect pods deployed with a hostPath mount * Deploy an admission controller that prevents new pods from using a hostPath mount

Breaking the Kubernetes Kill Chain: Host Path Mount

Puma Security, LLC

Tata AIG General Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Enterprise Knowledge’s Urmi Majumder, Principal Data Architecture Consultant, and Fernando Aguilar Islas, Senior Data Science Consultant, presented "Driving Behavioral Change for Information Management through Data-Driven Green Strategy" on March 27, 2024 at Enterprise Data World (EDW) in Orlando, Florida. In this presentation, Urmi and Fernando discussed a case study describing how the information management division in a large supply chain organization drove user behavior change through awareness of the carbon footprint of their duplicated and near-duplicated content, identified via advanced data analytics. Check out their presentation to gain valuable perspectives on utilizing data-driven strategies to influence positive behavioral shifts and support sustainability initiatives within your organization. In this session, participants gained answers to the following questions: - What is a Green Information Management (IM) Strategy, and why should you have one? - How can Artificial Intelligence (AI) and Machine Learning (ML) support your Green IM Strategy through content deduplication? - How can an organization use insights into their data to influence employee behavior for IM? - How can you reap additional benefits from content reduction that go beyond Green IM?

Driving Behavioral Change for Information Management through Data-Driven Gree...

Enterprise Knowledge

Kürzlich hochgeladen (20)

A Year of the Servo Reboot: Where Are We Now?

Finology Group – Insurtech Innovation Award 2024

Histor y of HAM Radio presentation slide

[2024]Digital Global Overview Report 2024 Meltwater.pdf

A Domino Admins Adventures (Engage 2024)

Data Cloud, More than a CDP by Matt Robison

08448380779 Call Girls In Diplomatic Enclave Women Seeking Men

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

Axa Assurance Maroc - Insurer Innovation Award 2024

CNv6 Instructor Chapter 6 Quality of Service

08448380779 Call Girls In Greater Kailash - I Women Seeking Men

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

08448380779 Call Girls In Friends Colony Women Seeking Men

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Workshop - Best of Both Worlds_ Combine KG and Vector search for enhanced R...

Breaking the Kubernetes Kill Chain: Host Path Mount

Tata AIG General Insurance Company - Insurer Innovation Award 2024

Driving Behavioral Change for Information Management through Data-Driven Gree...

Dataflow: the concurrency/parallelism architecture you need

1. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow: Russel Winder @russel_winder http://www.russel.org.uk russel@winder.org.uk The Concurrency/Parallelism Architecture You Need

3. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder What are (in computing† ): Concurrency: Structuring solution and code such that multiple parts may execute independently and possibly even at the same time. Parallelism: Execute multiple parts of a system at the same time on different processors so as to get things working faster. † In natural language these words have very different meanings.

4. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder What is Dataflow? An architecture comprising channels allowing data to flow from one operator to another, where each operator has multiple input channels and multiple output channels, and executes code only in response to the arrival of data on the inputs.

5. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Historically Dataflow computers: – Values flowing between… –…operators that calculate… –…new values to pass to… –…other operators. Dataflow hardware didn't take off, but the architecture works at various scales. The Manchester Prototype Dataflow Computer J R Gurd, C C Kirkham, I Watson CACM 28(1), 1985-01.

6. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow diagrams have been an integral part of analysis and design of information systems since the 1970s T de Marco, Structured Analysis and Systems Specification, Yourdon Press, NY, 1978.

8. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow and Functional Operators seem like they might be pure functions, but… …they are not necessarily, operators may have internal state. Operators may be referentially transparent, but they may be not. Operators may even have side effects.

15. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow Frameworks Scala: –Future Akka: –Dataflow variables, aka Promise –Deprecated in favour of Async Java: –Pre-8, Future –8+, CompletableFuture, aka Promise

16. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Architectural Issue Each of the aforementioned frameworks assumes that each operator creates a single value. Communication is by dataflow variables: each dataflow variable is a thread-safe single assignment variable.

19. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder GPars… Has dataflow variables (promises) and tasks and so can do everything Akka and Java can offer. Has DataflowQueue, and so can create real dataflow networks.

27. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Summary Dataflow is an architecture: Event-driven, single-threaded operators communicating by message passing using channels. Dataflow is an easement: Synchronization is inherent in the model, and there is no shared memory, so all deadlocks are trivial.

33. @russel_winder#devoxxuk #dataflowrules Copyright © 2014 Russel Winder Dataflow: Russel Winder @russel_winder http://www.russel.org.uk russel@winder.org.uk The Concurrency/Parallelism Architecture You Need

Dataflow: the concurrency/parallelism architecture you need

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Dataflow: the concurrency/parallelism architecture you need

Ähnlich wie Dataflow: the concurrency/parallelism architecture you need (20)

Mehr von Russel Winder

Mehr von Russel Winder (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Dataflow: the concurrency/parallelism architecture you need