SlideShare ist ein Scribd-Unternehmen logo
1 von 38
@michaelneale
www.CloudBees.com
@michaelneale
github.com/michaelneale
currently: trying to make less reliable things more
reliable.
Some things we use to get it done:
- scala (back end apps)
- ruby (chef, rails, proxymachine, lots)
- java (jenkins and plugins)
- erlang (“agents” and lots more..)
Exceptions. Errors.
What The Hell Is That Process Doing?
and how it relates to..
On Call PTSD
Exception handling is wrong...
But it seems harmless...
Erlang says no no no...
Question: a process on server not responding
Answer:
1) panic
2) take time thinking about underlying cause
3) bounce the bastard and pray **
FACT: Answer#3 will often work so well, you
often don’t really spend time to think about
doing #2**



** FibreError?
Imagine a programming environment:
 - unlimited process spawning
 - concurrency someone else’s problem
 - don’t try to handle errors - let it fail
Erlang
Built for stability, and not bothering me.
Anything else is gravy


I wasn’t an erlang programmer
Process isolation
- it works so well
- nothing shared
- has worked so well for so long with OSes
- in erlang - each ‘process’ is like a process !
nginx    Example: My Host OS

     haproxy               Erlang process

                         supervisor
                            ...
                       p1         p999



Each: own heap, garbage collector
Erlang is:
- A vm based runtime (light !)
- language with prolog heritage (!)
- compiler
- interpreter
- debug tools - batteries included
- OTP - an application (server?) platform
Erlang is:
- Not OO
- no shared state **
- state kept in “term stores” - mini DBs
- processes easy to hot swap
- Quasi functional programming
    (immutable variables, higher order)
- distributed (if you want)
OTP
- Open Telecommunications Platform
- not about telco !
- network stacks/apis, supervisor trees
- toolkit for reliable servers (middleware)
- behaviours
State in erlang




files
         Host + real kernel
                        Erlang process

DETS                supervisor
             ...
ETS                p1                    p999



              Send                       Receive
Erlang processes, like OS processes


- Can only send messages to each other
- Can fail independently
- Can be restarted, safely, independently
- Easy to write apps for - Old School !
Better


- Tiny overhead (can have 100K+ of them !)
- perfect for supervisory agents
- distribute: run on every box if you like !
- lots of processes == lots of concurrency
- if you need to block, block !
Have used erlang for


- Supervisory agents - controllers
- Autoscaling services (realtime data analytics)
- misc backend systems
- fun command line tools !
- is “lite” VM (startup, footprint)
Supervisor configuration


- Declare rules for restarting, failure
- So you don’t have to intervene !
- can hot reload changes
- can have dependent processes
Restart just busted process
Program 1 (module with behaviour)
Program 2 (module with behaviour)
Anyway, that is why we use Erlang, also...
- RabbitMQ
- Riak (nosql DB)
- CouchDB (nosql DB)
- Probably your telephone exchange
Alternative approaches:
- Use “native” with daemontools/monit etc
- Akka (Scala) has similar supervisor
- library for your favourite language
- ephemeral “state less” apps
- Write software without bugs
- Or just restart everything 400 times a day
What is not awesome:
- OTP boiler plate
- VM not as fast as some
- No one seems to like the language
- tools? IDEs?
Thank you

Weitere ähnliche Inhalte

Was ist angesagt?

Cassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQLCassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQL
Eric Evans
 
Spark intro by Adform Research
Spark intro by Adform ResearchSpark intro by Adform Research
Spark intro by Adform Research
Vasil Remeniuk
 

Was ist angesagt? (19)

Application Profiling for Memory and Performance
Application Profiling for Memory and PerformanceApplication Profiling for Memory and Performance
Application Profiling for Memory and Performance
 
Async and Await on the Server
Async and Await on the ServerAsync and Await on the Server
Async and Await on the Server
 
Task parallel library presentation
Task parallel library presentationTask parallel library presentation
Task parallel library presentation
 
Debugging ansible modules
Debugging ansible modulesDebugging ansible modules
Debugging ansible modules
 
Cassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQLCassandra: Not Just NoSQL, It's MoSQL
Cassandra: Not Just NoSQL, It's MoSQL
 
Celery introduction
Celery introductionCelery introduction
Celery introduction
 
Testing in Scala. Adform Research
Testing in Scala. Adform ResearchTesting in Scala. Adform Research
Testing in Scala. Adform Research
 
Ansible: What, Why & How
Ansible: What, Why & HowAnsible: What, Why & How
Ansible: What, Why & How
 
Spark intro by Adform Research
Spark intro by Adform ResearchSpark intro by Adform Research
Spark intro by Adform Research
 
Play Framework
Play FrameworkPlay Framework
Play Framework
 
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
Gabriele Santomaggio - Inside Elixir/Erlang - Codemotion Milan 2018
 
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
Continuous testing and deployment in Perl (London.pm Technical Meeting Octobe...
 
Automating Perl deployments with Hudson
Automating Perl deployments with HudsonAutomating Perl deployments with Hudson
Automating Perl deployments with Hudson
 
Async/Await
Async/AwaitAsync/Await
Async/Await
 
Open stack and_vagrant-os-meetup-2015
Open stack and_vagrant-os-meetup-2015Open stack and_vagrant-os-meetup-2015
Open stack and_vagrant-os-meetup-2015
 
Coroutines in Kotlin
Coroutines in KotlinCoroutines in Kotlin
Coroutines in Kotlin
 
The art of concurrent programming
The art of concurrent programmingThe art of concurrent programming
The art of concurrent programming
 
Gatling @ Scala.Io 2013
Gatling @ Scala.Io 2013Gatling @ Scala.Io 2013
Gatling @ Scala.Io 2013
 
Async await
Async awaitAsync await
Async await
 

Andere mochten auch (7)

裝飾設計
裝飾設計裝飾設計
裝飾設計
 
Errors and handling them. YOW nights Sydney 2011
Errors and handling them. YOW nights Sydney 2011Errors and handling them. YOW nights Sydney 2011
Errors and handling them. YOW nights Sydney 2011
 
Devoxx 2014 michael_neale
Devoxx 2014 michael_nealeDevoxx 2014 michael_neale
Devoxx 2014 michael_neale
 
Scala sydoct2011
Scala sydoct2011Scala sydoct2011
Scala sydoct2011
 
Cors michael
Cors michaelCors michael
Cors michael
 
Legal counsel it_cv
Legal counsel it_cvLegal counsel it_cv
Legal counsel it_cv
 
Cross site calls with javascript - the right way with CORS
Cross site calls with javascript - the right way with CORSCross site calls with javascript - the right way with CORS
Cross site calls with javascript - the right way with CORS
 

Ähnlich wie Osdc 2011 michael_neale

0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
Maxim Kharchenko
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
Sentifi
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
grim_radical
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
Holden Karau
 
Chelberg ptcuser 2010
Chelberg ptcuser 2010Chelberg ptcuser 2010
Chelberg ptcuser 2010
Clay Helberg
 
DevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 SlidesDevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 Slides
Alex Cruise
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB
 

Ähnlich wie Osdc 2011 michael_neale (20)

Repeating History...On Purpose...with Elixir
Repeating History...On Purpose...with ElixirRepeating History...On Purpose...with Elixir
Repeating History...On Purpose...with Elixir
 
Erlang
ErlangErlang
Erlang
 
0.5mln packets per second with Erlang
0.5mln packets per second with Erlang0.5mln packets per second with Erlang
0.5mln packets per second with Erlang
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlang
 
Joe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystemsJoe armstrong erlanga_languageforprogrammingreliablesystems
Joe armstrong erlanga_languageforprogrammingreliablesystems
 
Puppet for Sys Admins
Puppet for Sys AdminsPuppet for Sys Admins
Puppet for Sys Admins
 
REX about installing Kubernetes on Premise
REX about installing Kubernetes on PremiseREX about installing Kubernetes on Premise
REX about installing Kubernetes on Premise
 
PuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into OperationsPuppetDB: Sneaking Clojure into Operations
PuppetDB: Sneaking Clojure into Operations
 
You shall not get excited
You shall not get excitedYou shall not get excited
You shall not get excited
 
Understanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New YorkUnderstanding Spark Tuning: Strata New York
Understanding Spark Tuning: Strata New York
 
Spark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New YorkSpark Autotuning Talk - Strata New York
Spark Autotuning Talk - Strata New York
 
Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018Spark Autotuning - Strata EU 2018
Spark Autotuning - Strata EU 2018
 
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
Fault Tolerance in Spark: Lessons Learned from Production: Spark Summit East ...
 
Chelberg ptcuser 2010
Chelberg ptcuser 2010Chelberg ptcuser 2010
Chelberg ptcuser 2010
 
DevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 SlidesDevOps Days Vancouver 2014 Slides
DevOps Days Vancouver 2014 Slides
 
Introduction to Erlang/(Elixir) at a Webilea Hands-On Session
Introduction to Erlang/(Elixir) at a Webilea Hands-On SessionIntroduction to Erlang/(Elixir) at a Webilea Hands-On Session
Introduction to Erlang/(Elixir) at a Webilea Hands-On Session
 
Concurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple SpacesConcurrent Programming with Ruby and Tuple Spaces
Concurrent Programming with Ruby and Tuple Spaces
 
MongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, AnalyticsMongoDB Use Cases: Healthcare, CMS, Analytics
MongoDB Use Cases: Healthcare, CMS, Analytics
 
Elixir
ElixirElixir
Elixir
 
Beam me up, Scotty
Beam me up, ScottyBeam me up, Scotty
Beam me up, Scotty
 

Mehr von Michael Neale

Mehr von Michael Neale (11)

Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)Jenkins X intro (from google app dev conference)
Jenkins X intro (from google app dev conference)
 
Cd syd
Cd sydCd syd
Cd syd
 
Microservices and functional programming
Microservices and functional programmingMicroservices and functional programming
Microservices and functional programming
 
Java one 2011_michaelneale
Java one 2011_michaelnealeJava one 2011_michaelneale
Java one 2011_michaelneale
 
Sjug aug 2010_cloud
Sjug aug 2010_cloudSjug aug 2010_cloud
Sjug aug 2010_cloud
 
SJUG March 2010 Restful design
SJUG March 2010 Restful designSJUG March 2010 Restful design
SJUG March 2010 Restful design
 
On Scala Slides - OSDC 2009
On Scala Slides - OSDC 2009On Scala Slides - OSDC 2009
On Scala Slides - OSDC 2009
 
Osdc Complex Event Processing
Osdc Complex Event ProcessingOsdc Complex Event Processing
Osdc Complex Event Processing
 
Scala Sjug 09
Scala Sjug 09Scala Sjug 09
Scala Sjug 09
 
Jaoo Michael Neale 09
Jaoo Michael Neale 09Jaoo Michael Neale 09
Jaoo Michael Neale 09
 
Osdc Michael Neale 2008
Osdc Michael Neale 2008Osdc Michael Neale 2008
Osdc Michael Neale 2008
 

Kürzlich hochgeladen

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Kürzlich hochgeladen (20)

ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 

Osdc 2011 michael_neale

  • 2. @michaelneale github.com/michaelneale currently: trying to make less reliable things more reliable.
  • 3. Some things we use to get it done: - scala (back end apps) - ruby (chef, rails, proxymachine, lots) - java (jenkins and plugins) - erlang (“agents” and lots more..)
  • 4. Exceptions. Errors. What The Hell Is That Process Doing? and how it relates to..
  • 7. But it seems harmless...
  • 8.
  • 9.
  • 10. Erlang says no no no...
  • 11. Question: a process on server not responding Answer: 1) panic 2) take time thinking about underlying cause 3) bounce the bastard and pray **
  • 12. FACT: Answer#3 will often work so well, you often don’t really spend time to think about doing #2** ** FibreError?
  • 13.
  • 14. Imagine a programming environment: - unlimited process spawning - concurrency someone else’s problem - don’t try to handle errors - let it fail
  • 15. Erlang Built for stability, and not bothering me. Anything else is gravy I wasn’t an erlang programmer
  • 16. Process isolation - it works so well - nothing shared - has worked so well for so long with OSes - in erlang - each ‘process’ is like a process !
  • 17. nginx Example: My Host OS haproxy Erlang process supervisor ... p1 p999 Each: own heap, garbage collector
  • 18. Erlang is: - A vm based runtime (light !) - language with prolog heritage (!) - compiler - interpreter - debug tools - batteries included - OTP - an application (server?) platform
  • 19. Erlang is: - Not OO - no shared state ** - state kept in “term stores” - mini DBs - processes easy to hot swap - Quasi functional programming (immutable variables, higher order) - distributed (if you want)
  • 20.
  • 21.
  • 22.
  • 23. OTP - Open Telecommunications Platform - not about telco ! - network stacks/apis, supervisor trees - toolkit for reliable servers (middleware) - behaviours
  • 24.
  • 25.
  • 26.
  • 27. State in erlang files Host + real kernel Erlang process DETS supervisor ... ETS p1 p999 Send Receive
  • 28. Erlang processes, like OS processes - Can only send messages to each other - Can fail independently - Can be restarted, safely, independently - Easy to write apps for - Old School !
  • 29. Better - Tiny overhead (can have 100K+ of them !) - perfect for supervisory agents - distribute: run on every box if you like ! - lots of processes == lots of concurrency - if you need to block, block !
  • 30. Have used erlang for - Supervisory agents - controllers - Autoscaling services (realtime data analytics) - misc backend systems - fun command line tools ! - is “lite” VM (startup, footprint)
  • 31. Supervisor configuration - Declare rules for restarting, failure - So you don’t have to intervene ! - can hot reload changes - can have dependent processes
  • 33. Program 1 (module with behaviour)
  • 34. Program 2 (module with behaviour)
  • 35. Anyway, that is why we use Erlang, also... - RabbitMQ - Riak (nosql DB) - CouchDB (nosql DB) - Probably your telephone exchange
  • 36. Alternative approaches: - Use “native” with daemontools/monit etc - Akka (Scala) has similar supervisor - library for your favourite language - ephemeral “state less” apps - Write software without bugs - Or just restart everything 400 times a day
  • 37. What is not awesome: - OTP boiler plate - VM not as fast as some - No one seems to like the language - tools? IDEs?

Hinweis der Redaktion

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n