Suche senden
Hochladen
Your Code is Wrong
•
29 gefällt mir
•
31,503 views
N
nathanmarz
Folgen
My keynote at NoSQL Now! on August 21st, 2013
Weniger lesen
Mehr lesen
Technologie
Melden
Teilen
Melden
Teilen
1 von 106
Jetzt herunterladen
Downloaden Sie, um offline zu lesen
Empfohlen
The Epistemology of Software Engineering
The Epistemology of Software Engineering
nathanmarz
Using Simplicity to Make Hard Big Data Problems Easy
Using Simplicity to Make Hard Big Data Problems Easy
nathanmarz
The inherent complexity of stream processing
The inherent complexity of stream processing
nathanmarz
Storm
Storm
nathanmarz
Storm: Distributed and fault tolerant realtime computation
Storm: Distributed and fault tolerant realtime computation
Ferran Galí Reniu
Introduction to Storm
Introduction to Storm
Eugene Dvorkin
Learning Stream Processing with Apache Storm
Learning Stream Processing with Apache Storm
Eugene Dvorkin
Storm presentation
Storm presentation
Shyam Raj
Empfohlen
The Epistemology of Software Engineering
The Epistemology of Software Engineering
nathanmarz
Using Simplicity to Make Hard Big Data Problems Easy
Using Simplicity to Make Hard Big Data Problems Easy
nathanmarz
The inherent complexity of stream processing
The inherent complexity of stream processing
nathanmarz
Storm
Storm
nathanmarz
Storm: Distributed and fault tolerant realtime computation
Storm: Distributed and fault tolerant realtime computation
Ferran Galí Reniu
Introduction to Storm
Introduction to Storm
Eugene Dvorkin
Learning Stream Processing with Apache Storm
Learning Stream Processing with Apache Storm
Eugene Dvorkin
Storm presentation
Storm presentation
Shyam Raj
Apache Storm Internals
Apache Storm Internals
Humoyun Ahmedov
Storm Real Time Computation
Storm Real Time Computation
Sonal Raj
Storm
Storm
Pouyan Rezazadeh
Multi-tenant Apache Storm as a service
Multi-tenant Apache Storm as a service
Robert Evans
Apache Storm
Apache Storm
masifqadri
Apache Storm based Real Time Analytics for Recommending Trending Topics and S...
Apache Storm based Real Time Analytics for Recommending Trending Topics and S...
Humoyun Ahmedov
Spark vs storm
Spark vs storm
Trong Ton
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
DataWorks Summit/Hadoop Summit
Analysis big data by use php with storm
Analysis big data by use php with storm
毅 吕
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)
Brian Brazil
Introduction to Apache Storm
Introduction to Apache Storm
Tiziano De Matteis
Introduction to Apache Storm - Concept & Example
Introduction to Apache Storm - Concept & Example
Dung Ngua
Apache Storm and twitter Streaming API integration
Apache Storm and twitter Streaming API integration
Uday Vakalapudi
Storm and Cassandra
Storm and Cassandra
T Jake Luciani
Storm: The Real-Time Layer - GlueCon 2012
Storm: The Real-Time Layer - GlueCon 2012
Dan Lynn
Apache Spark
Apache Spark
masifqadri
Jan 2012 HUG: Storm
Jan 2012 HUG: Storm
Yahoo Developer Network
Using Hystrix to Build Resilient Distributed Systems
Using Hystrix to Build Resilient Distributed Systems
Matt Jacobs
Storm-on-YARN: Convergence of Low-Latency and Big-Data
Storm-on-YARN: Convergence of Low-Latency and Big-Data
DataWorks Summit
Real-Time Analytics with Apache Storm
Real-Time Analytics with Apache Storm
Taewoo Kim
Clojure at BackType
Clojure at BackType
nathanmarz
Relational Databases are Evolving To Support New Data Capabilities
Relational Databases are Evolving To Support New Data Capabilities
EDB
Weitere ähnliche Inhalte
Was ist angesagt?
Apache Storm Internals
Apache Storm Internals
Humoyun Ahmedov
Storm Real Time Computation
Storm Real Time Computation
Sonal Raj
Storm
Storm
Pouyan Rezazadeh
Multi-tenant Apache Storm as a service
Multi-tenant Apache Storm as a service
Robert Evans
Apache Storm
Apache Storm
masifqadri
Apache Storm based Real Time Analytics for Recommending Trending Topics and S...
Apache Storm based Real Time Analytics for Recommending Trending Topics and S...
Humoyun Ahmedov
Spark vs storm
Spark vs storm
Trong Ton
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
DataWorks Summit/Hadoop Summit
Analysis big data by use php with storm
Analysis big data by use php with storm
毅 吕
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)
Brian Brazil
Introduction to Apache Storm
Introduction to Apache Storm
Tiziano De Matteis
Introduction to Apache Storm - Concept & Example
Introduction to Apache Storm - Concept & Example
Dung Ngua
Apache Storm and twitter Streaming API integration
Apache Storm and twitter Streaming API integration
Uday Vakalapudi
Storm and Cassandra
Storm and Cassandra
T Jake Luciani
Storm: The Real-Time Layer - GlueCon 2012
Storm: The Real-Time Layer - GlueCon 2012
Dan Lynn
Apache Spark
Apache Spark
masifqadri
Jan 2012 HUG: Storm
Jan 2012 HUG: Storm
Yahoo Developer Network
Using Hystrix to Build Resilient Distributed Systems
Using Hystrix to Build Resilient Distributed Systems
Matt Jacobs
Storm-on-YARN: Convergence of Low-Latency and Big-Data
Storm-on-YARN: Convergence of Low-Latency and Big-Data
DataWorks Summit
Real-Time Analytics with Apache Storm
Real-Time Analytics with Apache Storm
Taewoo Kim
Was ist angesagt?
(20)
Apache Storm Internals
Apache Storm Internals
Storm Real Time Computation
Storm Real Time Computation
Storm
Storm
Multi-tenant Apache Storm as a service
Multi-tenant Apache Storm as a service
Apache Storm
Apache Storm
Apache Storm based Real Time Analytics for Recommending Trending Topics and S...
Apache Storm based Real Time Analytics for Recommending Trending Topics and S...
Spark vs storm
Spark vs storm
Resource Aware Scheduling in Apache Storm
Resource Aware Scheduling in Apache Storm
Analysis big data by use php with storm
Analysis big data by use php with storm
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)
Counting with Prometheus (CloudNativeCon+Kubecon Europe 2017)
Introduction to Apache Storm
Introduction to Apache Storm
Introduction to Apache Storm - Concept & Example
Introduction to Apache Storm - Concept & Example
Apache Storm and twitter Streaming API integration
Apache Storm and twitter Streaming API integration
Storm and Cassandra
Storm and Cassandra
Storm: The Real-Time Layer - GlueCon 2012
Storm: The Real-Time Layer - GlueCon 2012
Apache Spark
Apache Spark
Jan 2012 HUG: Storm
Jan 2012 HUG: Storm
Using Hystrix to Build Resilient Distributed Systems
Using Hystrix to Build Resilient Distributed Systems
Storm-on-YARN: Convergence of Low-Latency and Big-Data
Storm-on-YARN: Convergence of Low-Latency and Big-Data
Real-Time Analytics with Apache Storm
Real-Time Analytics with Apache Storm
Andere mochten auch
Clojure at BackType
Clojure at BackType
nathanmarz
Relational Databases are Evolving To Support New Data Capabilities
Relational Databases are Evolving To Support New Data Capabilities
EDB
Scala Abide: A lint tool for Scala
Scala Abide: A lint tool for Scala
Iulian Dragos
Puppet at Google
Puppet at Google
Puppet
Why Spark?
Why Spark?
Álvaro Agea Herradón
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
In-Memory Computing Summit
The Need for Async @ ScalaWorld
The Need for Async @ ScalaWorld
Konrad Malawski
Delivering Meaning In Near-Real Time At High Velocity In Massive Scale with A...
Delivering Meaning In Near-Real Time At High Velocity In Massive Scale with A...
Helena Edelson
Purely Functional Data Structures in Scala
Purely Functional Data Structures in Scala
Vladimir Kostyukov
Monadic Java
Monadic Java
Mario Fusco
NewSQL overview, Feb 2015
NewSQL overview, Feb 2015
Ivan Glushkov
The Newest in Session Types
The Newest in Session Types
Roland Kuhn
Scala Days San Francisco
Scala Days San Francisco
Martin Odersky
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Amy W. Tang
Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)
Scott Wlaschin
Concurrency: The Good, The Bad and The Ugly
Concurrency: The Good, The Bad and The Ugly
legendofklang
Andere mochten auch
(16)
Clojure at BackType
Clojure at BackType
Relational Databases are Evolving To Support New Data Capabilities
Relational Databases are Evolving To Support New Data Capabilities
Scala Abide: A lint tool for Scala
Scala Abide: A lint tool for Scala
Puppet at Google
Puppet at Google
Why Spark?
Why Spark?
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
IMCSummit 2015 - Day 2 IT Business Track - 4 Myths about In-Memory Databases ...
The Need for Async @ ScalaWorld
The Need for Async @ ScalaWorld
Delivering Meaning In Near-Real Time At High Velocity In Massive Scale with A...
Delivering Meaning In Near-Real Time At High Velocity In Massive Scale with A...
Purely Functional Data Structures in Scala
Purely Functional Data Structures in Scala
Monadic Java
Monadic Java
NewSQL overview, Feb 2015
NewSQL overview, Feb 2015
The Newest in Session Types
The Newest in Session Types
Scala Days San Francisco
Scala Days San Francisco
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Espresso: LinkedIn's Distributed Data Serving Platform (Paper)
Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)
Concurrency: The Good, The Bad and The Ugly
Concurrency: The Good, The Bad and The Ugly
Ähnlich wie Your Code is Wrong
Security for AWS : Journey to Least Privilege (update)
Security for AWS : Journey to Least Privilege (update)
dhubbard858
Security for AWS: Journey to Least Privilege
Security for AWS: Journey to Least Privilege
Lacework
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Sylvain Kalache
Evolution of Monitoring and Prometheus (Dublin 2018)
Evolution of Monitoring and Prometheus (Dublin 2018)
Brian Brazil
Hacking android apps by srini0x00
Hacking android apps by srini0x00
srini0x00
Talos: Neutralizing Vulnerabilities with Security Workarounds for Rapid Respo...
Talos: Neutralizing Vulnerabilities with Security Workarounds for Rapid Respo...
Zhen Huang
An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)
Brian Brazil
Online java compiler with security editor
Online java compiler with security editor
IRJET Journal
How To Do Excel-Like Row Selection in jQuery DataTable?
How To Do Excel-Like Row Selection in jQuery DataTable?
Polyxer Systems
Internet census 2012
Internet census 2012
Giuliano Tavaroli
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
lior mazor
Procuring the Anomaly Packets and Accountability Detection in the Network
Procuring the Anomaly Packets and Accountability Detection in the Network
IOSR Journals
Deep Dive: AWS X-Ray London Summit 2017
Deep Dive: AWS X-Ray London Summit 2017
Randall Hunt
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Brian Brazil
Time Series Anomaly Detection with Azure and .NETT
Time Series Anomaly Detection with Azure and .NETT
Marco Parenzan
Node.js security tour
Node.js security tour
Giacomo De Liberali
Software Birthmark Based Theft/Similarity Comparisons of JavaScript Programs
Software Birthmark Based Theft/Similarity Comparisons of JavaScript Programs
Swati Patel
Interview with Dmitriy Vyukov - the author of Relacy Race Detector (RRD)
Interview with Dmitriy Vyukov - the author of Relacy Race Detector (RRD)
PVS-Studio
Secure programming with php
Secure programming with php
Mohmad Feroz
Information Management 2marks with answer
Information Management 2marks with answer
suchi2480
Ähnlich wie Your Code is Wrong
(20)
Security for AWS : Journey to Least Privilege (update)
Security for AWS : Journey to Least Privilege (update)
Security for AWS: Journey to Least Privilege
Security for AWS: Journey to Least Privilege
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Skynet project: Monitor, analyze, scale, and maintain a system in the Cloud
Evolution of Monitoring and Prometheus (Dublin 2018)
Evolution of Monitoring and Prometheus (Dublin 2018)
Hacking android apps by srini0x00
Hacking android apps by srini0x00
Talos: Neutralizing Vulnerabilities with Security Workarounds for Rapid Respo...
Talos: Neutralizing Vulnerabilities with Security Workarounds for Rapid Respo...
An Introduction to Prometheus (GrafanaCon 2016)
An Introduction to Prometheus (GrafanaCon 2016)
Online java compiler with security editor
Online java compiler with security editor
How To Do Excel-Like Row Selection in jQuery DataTable?
How To Do Excel-Like Row Selection in jQuery DataTable?
Internet census 2012
Internet census 2012
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
The Hacking Games - Operation System Vulnerabilities Meetup 29112022
Procuring the Anomaly Packets and Accountability Detection in the Network
Procuring the Anomaly Packets and Accountability Detection in the Network
Deep Dive: AWS X-Ray London Summit 2017
Deep Dive: AWS X-Ray London Summit 2017
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Monitoring What Matters: The Prometheus Approach to Whitebox Monitoring (Berl...
Time Series Anomaly Detection with Azure and .NETT
Time Series Anomaly Detection with Azure and .NETT
Node.js security tour
Node.js security tour
Software Birthmark Based Theft/Similarity Comparisons of JavaScript Programs
Software Birthmark Based Theft/Similarity Comparisons of JavaScript Programs
Interview with Dmitriy Vyukov - the author of Relacy Race Detector (RRD)
Interview with Dmitriy Vyukov - the author of Relacy Race Detector (RRD)
Secure programming with php
Secure programming with php
Information Management 2marks with answer
Information Management 2marks with answer
Mehr von nathanmarz
Demystifying Data Engineering
Demystifying Data Engineering
nathanmarz
Runaway complexity in Big Data... and a plan to stop it
Runaway complexity in Big Data... and a plan to stop it
nathanmarz
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computation
nathanmarz
ElephantDB
ElephantDB
nathanmarz
Become Efficient or Die: The Story of BackType
Become Efficient or Die: The Story of BackType
nathanmarz
The Secrets of Building Realtime Big Data Systems
The Secrets of Building Realtime Big Data Systems
nathanmarz
Cascalog workshop
Cascalog workshop
nathanmarz
Cascalog at Strange Loop
Cascalog at Strange Loop
nathanmarz
Cascalog at Hadoop Day
Cascalog at Hadoop Day
nathanmarz
Cascalog at May Bay Area Hadoop User Group
Cascalog at May Bay Area Hadoop User Group
nathanmarz
Cascalog
Cascalog
nathanmarz
Cascading
Cascading
nathanmarz
Mehr von nathanmarz
(12)
Demystifying Data Engineering
Demystifying Data Engineering
Runaway complexity in Big Data... and a plan to stop it
Runaway complexity in Big Data... and a plan to stop it
Storm: distributed and fault-tolerant realtime computation
Storm: distributed and fault-tolerant realtime computation
ElephantDB
ElephantDB
Become Efficient or Die: The Story of BackType
Become Efficient or Die: The Story of BackType
The Secrets of Building Realtime Big Data Systems
The Secrets of Building Realtime Big Data Systems
Cascalog workshop
Cascalog workshop
Cascalog at Strange Loop
Cascalog at Strange Loop
Cascalog at Hadoop Day
Cascalog at Hadoop Day
Cascalog at May Bay Area Hadoop User Group
Cascalog at May Bay Area Hadoop User Group
Cascalog
Cascalog
Cascading
Cascading
Kürzlich hochgeladen
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
The Digital Insurer
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
Remote DBA Services
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
Dropbox
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
The Digital Insurer
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Product Anonymous
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
apidays
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
DianaGray10
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Anna Loughnan Colquhoun
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Juan lago vázquez
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
wesley chun
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
Nanddeep Nachan
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
sudhanshuwaghmare1
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
The Digital Insurer
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
Martijn de Jong
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
jfdjdjcjdnsjd
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
apidays
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
MIND CTI
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
Khushali Kathiriya
Kürzlich hochgeladen
(20)
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
presentation ICT roal in 21st century education
presentation ICT roal in 21st century education
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
Apidays Singapore 2024 - Scalable LLM APIs for AI and Generative AI Applicati...
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
Your Code is Wrong
1.
Your Code is
Wrong Nathan Marz @nathanmarz 1
2.
Let’s start with
an example
3.
Storm’s “reportError” method
4.
(Storm is a
realtime computation system, like Hadoop but for realtime)
5.
Storm architecture
6.
Storm architecture Master node
(similar to Hadoop JobTracker)
7.
Storm architecture Used for
cluster coordination
8.
Storm architecture Run worker
processes
9.
Storm’s “reportError” method
10.
Used to show
errors in the Storm UI
11.
Error info is
stored in Zookeeper
12.
What happens when
a user deploys code like this?
13.
Denial-of-service on Zookeeper and
cluster goes down
14.
Robust! Designed input space
Actual input space
15.
Your code is
wrong
16.
Your code is
literally wrong
17.
Your code is
wrong
18.
19.
Why do you
believe your code is correct?
20.
Your code Dependency 1 Dependency
2 Dependency 3
21.
Dependency 1 Dependency 4 Dependency
5
22.
Dependency 4 Dependency 6 Dependency
9 Dependency 7 Dependency 8
23.
Dependency 3,000,000 Hardware
24.
Electronics
25.
Chemistry
26.
Atomic physics
27.
Quantum mechanics
28.
I think I
can safely say that nobody understands quantum mechanics. Richard Feynman
29.
Your code is
wrong
30.
Your code ...
31.
All the software
you’ve used has had bugs in it
32.
Including the software you’ve
written
33.
Your code is sometimes
correct
34.
That’s good enough!
35.
36.
Treat code as
nondeterministic
37.
Embrace “your code
is wrong” to design better software
38.
Robust! Designed input space
Actual input space
39.
Robust! Designed input space
Actual input space
40.
An example
41.
Learning from Hadoop Jobtracker Job Job Job
42.
Learning from Hadoop Jobtracker Job Job Job
43.
Learning from Hadoop Jobtracker Job Job Job
44.
Your code is
wrong
45.
So your processes
will crash
46.
Storm’s daemons are process
fault-tolerant
47.
Storm Nimbus Topology Topology Topology
48.
Storm Nimbus Topology Topology Topology
49.
Storm Nimbus Topology Topology Topology
50.
Storm Nimbus Topology Topology Topology
51.
Storm Nimbus Topology Topology Topology
52.
Robust! Designed input space
Actual input space
53.
Robust! Designed input space
Actual input space
54.
The impact of
code being wrong
55.
Robust! Designed input space
Actual input space Failures! Bad performance! Security holes! Irrelevant!
56.
Design principle #1 Measuring
and monitoring are the foundation of solid engineering
57.
Measuring: Under what range
of inputs does my software function well?
58.
Monitoring: What’s the actual
input space of my software?
59.
Measure & Monitor Latency Throughput Stack
traces Buffer sizes Memory usage CPU usage #threads spawned ...
60.
How you monitor
your software is as important as its functionality
61.
Design principle #2 Embrace
immutability
62.
Read/write database Application
63.
MySQLApplication
64.
MongoDBApplication
65.
RiakApplication
66.
CassandraApplication
67.
HBaseApplication
68.
Your code is
wrong
69.
So data will
be corrupted
70.
And you may
not know why
71.
Views Immutable, ever-growing data Application Architecture based on
immutability
72.
Views Immutable, ever-growing data Application Lambda architecture
73.
Design principle #3 Minimize
dependencies
74.
The less that
can go wrong, the less that will go wrong
75.
Example: Storm’s usage of
Zookeeper
76.
Worker locations stored
in Zookeeper
77.
All workers must
know locations of other workers to send messages
78.
Two ways to
get location updates
79.
1. Poll Zookeeper Worker
Zookeeper
80.
2. Use Zookeeper
“watch” feature to get push notifications Worker Zookeeper
81.
Method 2 is
faster but relies on another feature
82.
Storm uses both
methods Worker Zookeeper
83.
If watch feature
fails, locations still propagate via polling
84.
Eliminating dependence justified by
small amount of code required
85.
Design principle #4 Explicitly
respect functional input ranges
86.
Storm’s “reportError” method
87.
Implement self-throttling to avoid
overloading other systems
88.
Design principle #5 Embrace
recomputation
89.
“Your code is
wrong” meanings 1. Design input space differs from actual input space 2. The logic of your code is wrong 3. Requirements are constantly changing
90.
You must be
able to change your code to match shifting requirements
91.
Example: blogging software
92.
New requirement: search
93.
Have to build
a search index
94.
95.
Recomputation gives you so
much more
96.
Views Immutable, ever-growing data Application
97.
Building software no
different than any other engineering
98.
The underlying challenges are
the same
99.
100.
101.
What will break
it?
102.
What are limits
of my dependencies?
103.
How can I
add redundancy to increase robustness?
104.
Can I isolate
failures?
105.
Our raw materials
are ideas instead of matter
106.
Thank you
Jetzt herunterladen