SlideShare ist ein Scribd-Unternehmen logo
1 von 23
Downloaden Sie, um offline zu lesen
© Copyright 2018 Pivotal Software, Inc. All rights Reserved.
Jerry Kuch - jkuch@pivotal.io
Wayne Lund - wlund@pivotal.io
12 Dec 2018
10 Things Every
Developer Using RabbitMQ Should Know
Introduction
● RabbitMQ is a complex, multi-
faceted system
● RabbitMQ has many features
● A few basic ideas can go a
long way in:
○ Understanding
RabbitMQ
○ Using RabbitMQ
effectively
Broker
Producer
Message
Exchange
Bindings
Queues
Consumer
Agenda
●Ten Topics
●From Three Major Themes:
○ Guiding Principles
○ Messages and Their Handling
○ Resource Management
●Where to Go Next
Broker
Producer
Message
Exchange
Bindings
Queues
Consumer
Guiding Principles
Ten Best Practices
● RabbitMQ is not a database
● Aim to keep queue lengths short in practice
○ Balancing ingress and egress rates is the
first, best idea
○ Know what a “normal” length for a given
queue is
● Problems with long queues:
○ Memory alarms, paging to disk, TCP back
pressure
● Mind message sizes
● Monitor!
Guiding Principles
Item 1: “A Happy Rabbit
is an Empty Rabbit"
● RabbitMQ is written in Erlang
● Erlang concurrency is based on lightweight processes
○ Rabbit is built of a collection of these communicating
processes
○ Each queue you create has its own queue process
○ Erlang processes are scheduled on a pool of OS
threads
○ Queues are thus natural concurrency bottlenecks
● Mechanical sympathy for your system helps you better
exploit it
● By contrast exchanges are not backed by their own
processes
Guiding Principles
Item 2: “The Queue is
Rabbit’s
Basic Unit of
Concurrency"
Messages and
Their Handling
Ten Best Practices
● What makes us think anything happened in a
distributed system?
● Transfer of Responsibility:
○ At any moment, know who is responsible for
the existence and integrity of a message
○ Maintain the responsibility and hand it off in
a disciplined way
● Consumer side: AMQP ACKs.
● Producer side: Publisher Confirms (also AMQP
transactions)
Messages and
Their Handling
Item 3: The “Transfer of
Responsibility” Notion
● Ways to consume: basic.get and basic.consume
● Most clients should favor basic.consume
● Prefetch (prefetch_count in basic.qos method):
○ How many messages a client is willing to
take at a time
○ Broker will deliver up to that many and wait
for ACK
● Small prefetches can inhibit throughput
● Large prefetch can result in one of many
consumers on a queue monopolizing queue’s
output
● Choose based on how fast consumer gets work
done before ACKing
Messages and
Their Handling
Item 4: Consumers
Should Eat Not Nibble
● Messages in many applications break into
natural tiers.
● Many RabbitMQ features impose non-trivial
costs:
○ Persistence
○ HA Mirroring of Queues
○ Clustering
○ Publisher Confirms and AMQP Transactions
○ Consumer ACKs vs noACK/autoACK
● Think about your traffic, the benefits of a feature
and spend accordingly!
Messages and
Their Handling
Item 5: Know Costs of
Guarantees and Don’t
Overpay Unnecessarily
Resource
Management
Ten Best Practices
● Some objects are more heavyweight than others
● CPU and RAM are limitations more often than
networking
● Management can impose overhead:
● Beware in large clusters of ‘detailed’ metrics
● Don’t hit the wall (and understand the padding in
front of it)!
● Padding: Memory and disk alarms, TCP back
pressure
● The Wall: Fatal resource exhaustion is fatal
Resource
Management
Item 6: Don’t Overtax
Your
System’s Resources
● Understand AMQP Connections vs. Channels
○ Connections are expensive
○ Channels try to amortize the expense via
multiplexing
● Don’t churn connections and mind your OS limits
● Don’t share channels between threads
● Consider separate connections for publishers
and consumers
Resource
Management
Item 7: Manage
Connections and
Channels Thoughtfully
● Auto-delete queues: delete when last subscriber
unsubscribes.
● Exclusive queues: exclusive to one connection,
deleted when connection closes.
● Queue length limit: as number of messages or
count of bytes;
● Per-Queue Message TTL
● Per-Message TTL
● Queue TTL: whole queue vanishes after TTL if no
consumers
Resource
Management
Item 8: Tips for Cleaning
Up Queues
● Consider virtual hosts—remember they’re only
namespaces.
● Beware noisy neighbors!
● Sometimes separate clusters can be best:
○ Keep noisy neighbors apart
○ Easier to debug, determine culpability, etc.
Resource
Management
Item 9: Think About
Isolation
● Monitoring! Web UI, management HTTP API, logs
○ Workloads:
https://github.com/rabbitmq/workloadsBuilt-
in liveness check API
● Can wire API to Splunk, Nagios, etc.
● Monitor not just node health but also your
message fabric:
○ Its shape and typical parameters
● Management statistics collection: collection rate
configurable, beware heavy load
● Read, Learn, Experiment, Understand, Repeat.
Resource
Management
Item 10: Know What’s
Happening
Guiding Principles
● A Happy Rabbit is an Empty Rabbit
● The Queue is Rabbit’s Unit of Concurrency
Messages and Their Handling
● The “Transfer of Responsibility” Notion
● Consumers Should Eat, Not Nibble
● Know Costs of Guarantees and Don’t Overpay
Unnecessarily
SUMMARY
Resource Management
● Don’t Overtax Your System’s Resources
● Manage Connections and Channels Thoughtfully
● Tips for Cleaning Up Queues
● Think About Isolation
● Know What’s Happening
SUMMARY
(Continued)
● RabbitMQ Website: http://www.rabbitmq.com
● RabbitMQ in Action, Manning Publications
● RabbitMQ in Depth, Manning Publications
● Workloads: https://github.com/rabbitmq/workloads
● Marcial Rosales developer guide:
https://github.com/MarcialRosales/rabbitmq-developer-guide
● Jack Vanlightly Blogs: https://jack-vanlightly.com/
● CloudAMQP Best Practices:
https://www.cloudamqp.com/blog/index.html
Where to Learn More
Transforming How The World Builds Software
© Copyright 2018 Pivotal Software, Inc. All rights Reserved.

Weitere ähnliche Inhalte

Was ist angesagt?

ASP.NET과 C#으로 개발하는 대규모 소셜 게임
ASP.NET과 C#으로 개발하는 대규모 소셜 게임ASP.NET과 C#으로 개발하는 대규모 소셜 게임
ASP.NET과 C#으로 개발하는 대규모 소셜 게임
흥배 최
 
Scalable and Available, Patterns for Success
Scalable and Available, Patterns for SuccessScalable and Available, Patterns for Success
Scalable and Available, Patterns for Success
Derek Collison
 

Was ist angesagt? (20)

Rabbitmq basics
Rabbitmq basicsRabbitmq basics
Rabbitmq basics
 
Building microservices with grpc
Building microservices with grpcBuilding microservices with grpc
Building microservices with grpc
 
Messaging with RabbitMQ and AMQP
Messaging with RabbitMQ and AMQPMessaging with RabbitMQ and AMQP
Messaging with RabbitMQ and AMQP
 
ASP.NET과 C#으로 개발하는 대규모 소셜 게임
ASP.NET과 C#으로 개발하는 대규모 소셜 게임ASP.NET과 C#으로 개발하는 대규모 소셜 게임
ASP.NET과 C#으로 개발하는 대규모 소셜 게임
 
KAFKA 3.1.0.pdf
KAFKA 3.1.0.pdfKAFKA 3.1.0.pdf
KAFKA 3.1.0.pdf
 
Introduction to AMQP Messaging with RabbitMQ
Introduction to AMQP Messaging with RabbitMQIntroduction to AMQP Messaging with RabbitMQ
Introduction to AMQP Messaging with RabbitMQ
 
What we've learned from running thousands of production RabbitMQ clusters - L...
What we've learned from running thousands of production RabbitMQ clusters - L...What we've learned from running thousands of production RabbitMQ clusters - L...
What we've learned from running thousands of production RabbitMQ clusters - L...
 
gRPC and Microservices
gRPC and MicroservicesgRPC and Microservices
gRPC and Microservices
 
IBM MQ - better application performance
IBM MQ - better application performanceIBM MQ - better application performance
IBM MQ - better application performance
 
LMAX Architecture
LMAX ArchitectureLMAX Architecture
LMAX Architecture
 
RabbitMq
RabbitMqRabbitMq
RabbitMq
 
Easy Microservices with JHipster - Devoxx BE 2017
Easy Microservices with JHipster - Devoxx BE 2017Easy Microservices with JHipster - Devoxx BE 2017
Easy Microservices with JHipster - Devoxx BE 2017
 
Introduction to gRPC
Introduction to gRPCIntroduction to gRPC
Introduction to gRPC
 
gRPC Overview
gRPC OverviewgRPC Overview
gRPC Overview
 
Deploying and managing IBM MQ in the Cloud
Deploying and managing IBM MQ in the CloudDeploying and managing IBM MQ in the Cloud
Deploying and managing IBM MQ in the Cloud
 
Rabbitmq & Kafka Presentation
Rabbitmq & Kafka PresentationRabbitmq & Kafka Presentation
Rabbitmq & Kafka Presentation
 
CQRS and Event Sourcing for Java Developers
CQRS and Event Sourcing for Java DevelopersCQRS and Event Sourcing for Java Developers
CQRS and Event Sourcing for Java Developers
 
Scalable and Available, Patterns for Success
Scalable and Available, Patterns for SuccessScalable and Available, Patterns for Success
Scalable and Available, Patterns for Success
 
RabbitMQ.pptx
RabbitMQ.pptxRabbitMQ.pptx
RabbitMQ.pptx
 
Rabbit MQ introduction
Rabbit MQ introductionRabbit MQ introduction
Rabbit MQ introduction
 

Ähnlich wie 10 Things Every Developer Using RabbitMQ Should Know

Ähnlich wie 10 Things Every Developer Using RabbitMQ Should Know (20)

WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...
WebCamp 2016: Python. Вячеслав Каковский: Real-time мессенджер на Python. Осо...
 
WebCamp Ukraine 2016: Instant messenger with Python. Back-end development
WebCamp Ukraine 2016: Instant messenger with Python. Back-end developmentWebCamp Ukraine 2016: Instant messenger with Python. Back-end development
WebCamp Ukraine 2016: Instant messenger with Python. Back-end development
 
Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafka
Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache KafkaStrata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafka
Strata+Hadoop 2017 San Jose: Lessons from a year of supporting Apache Kafka
 
GLC webinar: limiting bandwidth using mikrotik
GLC webinar: limiting bandwidth using mikrotikGLC webinar: limiting bandwidth using mikrotik
GLC webinar: limiting bandwidth using mikrotik
 
Event driven architectures with Kinesis
Event driven architectures with KinesisEvent driven architectures with Kinesis
Event driven architectures with Kinesis
 
Working with Asynchronous Events
Working with Asynchronous EventsWorking with Asynchronous Events
Working with Asynchronous Events
 
MuleSoft Surat Virtual Meetup#33 - Unleash the power of Anypoint MQ and DLQ
MuleSoft Surat Virtual Meetup#33 - Unleash the power of Anypoint MQ and DLQ MuleSoft Surat Virtual Meetup#33 - Unleash the power of Anypoint MQ and DLQ
MuleSoft Surat Virtual Meetup#33 - Unleash the power of Anypoint MQ and DLQ
 
Think beyond http
Think beyond httpThink beyond http
Think beyond http
 
Age-based Cooperative Caching in Information-Centric Networks
Age-based Cooperative Caching in Information-Centric NetworksAge-based Cooperative Caching in Information-Centric Networks
Age-based Cooperative Caching in Information-Centric Networks
 
Bench, a Framework for Benchmarking Kafka Using K8s and OpenMessaging Benchma...
Bench, a Framework for Benchmarking Kafka Using K8s and OpenMessaging Benchma...Bench, a Framework for Benchmarking Kafka Using K8s and OpenMessaging Benchma...
Bench, a Framework for Benchmarking Kafka Using K8s and OpenMessaging Benchma...
 
Web performance optimization - MercadoLibre
Web performance optimization - MercadoLibreWeb performance optimization - MercadoLibre
Web performance optimization - MercadoLibre
 
SPDY and What to Consider for HTTP/2.0
SPDY and What to Consider for HTTP/2.0SPDY and What to Consider for HTTP/2.0
SPDY and What to Consider for HTTP/2.0
 
AMQP with RabbitMQ
AMQP with RabbitMQAMQP with RabbitMQ
AMQP with RabbitMQ
 
2 years into drinking the Microservice kool-aid (Fact and Fiction)
2 years into drinking the Microservice kool-aid (Fact and Fiction)2 years into drinking the Microservice kool-aid (Fact and Fiction)
2 years into drinking the Microservice kool-aid (Fact and Fiction)
 
Reducing load with RabbitMQ
Reducing load with RabbitMQReducing load with RabbitMQ
Reducing load with RabbitMQ
 
Rate limits and Performance
Rate limits and PerformanceRate limits and Performance
Rate limits and Performance
 
Do More With Message Queue
Do More With Message QueueDo More With Message Queue
Do More With Message Queue
 
Web performance mercadolibre - ECI 2013
Web performance   mercadolibre - ECI 2013Web performance   mercadolibre - ECI 2013
Web performance mercadolibre - ECI 2013
 
Streaming millions of Contact Center interactions in (near) real-time with Pu...
Streaming millions of Contact Center interactions in (near) real-time with Pu...Streaming millions of Contact Center interactions in (near) real-time with Pu...
Streaming millions of Contact Center interactions in (near) real-time with Pu...
 
Streaming Millions of Contact Center Interactions in (Near) Real-Time with Pu...
Streaming Millions of Contact Center Interactions in (Near) Real-Time with Pu...Streaming Millions of Contact Center Interactions in (Near) Real-Time with Pu...
Streaming Millions of Contact Center Interactions in (Near) Real-Time with Pu...
 

Mehr von VMware Tanzu

Mehr von VMware Tanzu (20)

What AI Means For Your Product Strategy And What To Do About It
What AI Means For Your Product Strategy And What To Do About ItWhat AI Means For Your Product Strategy And What To Do About It
What AI Means For Your Product Strategy And What To Do About It
 
Make the Right Thing the Obvious Thing at Cardinal Health 2023
Make the Right Thing the Obvious Thing at Cardinal Health 2023Make the Right Thing the Obvious Thing at Cardinal Health 2023
Make the Right Thing the Obvious Thing at Cardinal Health 2023
 
Enhancing DevEx and Simplifying Operations at Scale
Enhancing DevEx and Simplifying Operations at ScaleEnhancing DevEx and Simplifying Operations at Scale
Enhancing DevEx and Simplifying Operations at Scale
 
Spring Update | July 2023
Spring Update | July 2023Spring Update | July 2023
Spring Update | July 2023
 
Platforms, Platform Engineering, & Platform as a Product
Platforms, Platform Engineering, & Platform as a ProductPlatforms, Platform Engineering, & Platform as a Product
Platforms, Platform Engineering, & Platform as a Product
 
Building Cloud Ready Apps
Building Cloud Ready AppsBuilding Cloud Ready Apps
Building Cloud Ready Apps
 
Spring Boot 3 And Beyond
Spring Boot 3 And BeyondSpring Boot 3 And Beyond
Spring Boot 3 And Beyond
 
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdfSpring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
Spring Cloud Gateway - SpringOne Tour 2023 Charles Schwab.pdf
 
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
Simplify and Scale Enterprise Apps in the Cloud | Boston 2023
 
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
Simplify and Scale Enterprise Apps in the Cloud | Seattle 2023
 
tanzu_developer_connect.pptx
tanzu_developer_connect.pptxtanzu_developer_connect.pptx
tanzu_developer_connect.pptx
 
Tanzu Virtual Developer Connect Workshop - French
Tanzu Virtual Developer Connect Workshop - FrenchTanzu Virtual Developer Connect Workshop - French
Tanzu Virtual Developer Connect Workshop - French
 
Tanzu Developer Connect Workshop - English
Tanzu Developer Connect Workshop - EnglishTanzu Developer Connect Workshop - English
Tanzu Developer Connect Workshop - English
 
Virtual Developer Connect Workshop - English
Virtual Developer Connect Workshop - EnglishVirtual Developer Connect Workshop - English
Virtual Developer Connect Workshop - English
 
Tanzu Developer Connect - French
Tanzu Developer Connect - FrenchTanzu Developer Connect - French
Tanzu Developer Connect - French
 
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
Simplify and Scale Enterprise Apps in the Cloud | Dallas 2023
 
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring BootSpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
SpringOne Tour: Deliver 15-Factor Applications on Kubernetes with Spring Boot
 
SpringOne Tour: The Influential Software Engineer
SpringOne Tour: The Influential Software EngineerSpringOne Tour: The Influential Software Engineer
SpringOne Tour: The Influential Software Engineer
 
SpringOne Tour: Domain-Driven Design: Theory vs Practice
SpringOne Tour: Domain-Driven Design: Theory vs PracticeSpringOne Tour: Domain-Driven Design: Theory vs Practice
SpringOne Tour: Domain-Driven Design: Theory vs Practice
 
SpringOne Tour: Spring Recipes: A Collection of Common-Sense Solutions
SpringOne Tour: Spring Recipes: A Collection of Common-Sense SolutionsSpringOne Tour: Spring Recipes: A Collection of Common-Sense Solutions
SpringOne Tour: Spring Recipes: A Collection of Common-Sense Solutions
 

Kürzlich hochgeladen

Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Victor Rentea
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
?#DUbAI#??##{{(☎️+971_581248768%)**%*]'#abortion pills for sale in dubai@
 

Kürzlich hochgeladen (20)

DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Exploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with MilvusExploring Multimodal Embeddings with Milvus
Exploring Multimodal Embeddings with Milvus
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdfRising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
Rising Above_ Dubai Floods and the Fortitude of Dubai International Airport.pdf
 
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
Apidays New York 2024 - Passkeys: Developing APIs to enable passwordless auth...
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUKSpring Boot vs Quarkus the ultimate battle - DevoxxUK
Spring Boot vs Quarkus the ultimate battle - DevoxxUK
 

10 Things Every Developer Using RabbitMQ Should Know

  • 1. © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Jerry Kuch - jkuch@pivotal.io Wayne Lund - wlund@pivotal.io 12 Dec 2018 10 Things Every Developer Using RabbitMQ Should Know
  • 2. Introduction ● RabbitMQ is a complex, multi- faceted system ● RabbitMQ has many features ● A few basic ideas can go a long way in: ○ Understanding RabbitMQ ○ Using RabbitMQ effectively Broker Producer Message Exchange Bindings Queues Consumer
  • 3. Agenda ●Ten Topics ●From Three Major Themes: ○ Guiding Principles ○ Messages and Their Handling ○ Resource Management ●Where to Go Next Broker Producer Message Exchange Bindings Queues Consumer
  • 5. ● RabbitMQ is not a database ● Aim to keep queue lengths short in practice ○ Balancing ingress and egress rates is the first, best idea ○ Know what a “normal” length for a given queue is ● Problems with long queues: ○ Memory alarms, paging to disk, TCP back pressure ● Mind message sizes ● Monitor! Guiding Principles Item 1: “A Happy Rabbit is an Empty Rabbit"
  • 6. ● RabbitMQ is written in Erlang ● Erlang concurrency is based on lightweight processes ○ Rabbit is built of a collection of these communicating processes ○ Each queue you create has its own queue process ○ Erlang processes are scheduled on a pool of OS threads ○ Queues are thus natural concurrency bottlenecks ● Mechanical sympathy for your system helps you better exploit it ● By contrast exchanges are not backed by their own processes Guiding Principles Item 2: “The Queue is Rabbit’s Basic Unit of Concurrency"
  • 8. ● What makes us think anything happened in a distributed system? ● Transfer of Responsibility: ○ At any moment, know who is responsible for the existence and integrity of a message ○ Maintain the responsibility and hand it off in a disciplined way ● Consumer side: AMQP ACKs. ● Producer side: Publisher Confirms (also AMQP transactions) Messages and Their Handling Item 3: The “Transfer of Responsibility” Notion
  • 9. ● Ways to consume: basic.get and basic.consume ● Most clients should favor basic.consume ● Prefetch (prefetch_count in basic.qos method): ○ How many messages a client is willing to take at a time ○ Broker will deliver up to that many and wait for ACK ● Small prefetches can inhibit throughput ● Large prefetch can result in one of many consumers on a queue monopolizing queue’s output ● Choose based on how fast consumer gets work done before ACKing Messages and Their Handling Item 4: Consumers Should Eat Not Nibble
  • 10. ● Messages in many applications break into natural tiers. ● Many RabbitMQ features impose non-trivial costs: ○ Persistence ○ HA Mirroring of Queues ○ Clustering ○ Publisher Confirms and AMQP Transactions ○ Consumer ACKs vs noACK/autoACK ● Think about your traffic, the benefits of a feature and spend accordingly! Messages and Their Handling Item 5: Know Costs of Guarantees and Don’t Overpay Unnecessarily
  • 12. ● Some objects are more heavyweight than others ● CPU and RAM are limitations more often than networking ● Management can impose overhead: ● Beware in large clusters of ‘detailed’ metrics ● Don’t hit the wall (and understand the padding in front of it)! ● Padding: Memory and disk alarms, TCP back pressure ● The Wall: Fatal resource exhaustion is fatal Resource Management Item 6: Don’t Overtax Your System’s Resources
  • 13. ● Understand AMQP Connections vs. Channels ○ Connections are expensive ○ Channels try to amortize the expense via multiplexing ● Don’t churn connections and mind your OS limits ● Don’t share channels between threads ● Consider separate connections for publishers and consumers Resource Management Item 7: Manage Connections and Channels Thoughtfully
  • 14. ● Auto-delete queues: delete when last subscriber unsubscribes. ● Exclusive queues: exclusive to one connection, deleted when connection closes. ● Queue length limit: as number of messages or count of bytes; ● Per-Queue Message TTL ● Per-Message TTL ● Queue TTL: whole queue vanishes after TTL if no consumers Resource Management Item 8: Tips for Cleaning Up Queues
  • 15. ● Consider virtual hosts—remember they’re only namespaces. ● Beware noisy neighbors! ● Sometimes separate clusters can be best: ○ Keep noisy neighbors apart ○ Easier to debug, determine culpability, etc. Resource Management Item 9: Think About Isolation
  • 16. ● Monitoring! Web UI, management HTTP API, logs ○ Workloads: https://github.com/rabbitmq/workloadsBuilt- in liveness check API ● Can wire API to Splunk, Nagios, etc. ● Monitor not just node health but also your message fabric: ○ Its shape and typical parameters ● Management statistics collection: collection rate configurable, beware heavy load ● Read, Learn, Experiment, Understand, Repeat. Resource Management Item 10: Know What’s Happening
  • 17.
  • 18.
  • 19.
  • 20. Guiding Principles ● A Happy Rabbit is an Empty Rabbit ● The Queue is Rabbit’s Unit of Concurrency Messages and Their Handling ● The “Transfer of Responsibility” Notion ● Consumers Should Eat, Not Nibble ● Know Costs of Guarantees and Don’t Overpay Unnecessarily SUMMARY
  • 21. Resource Management ● Don’t Overtax Your System’s Resources ● Manage Connections and Channels Thoughtfully ● Tips for Cleaning Up Queues ● Think About Isolation ● Know What’s Happening SUMMARY (Continued)
  • 22. ● RabbitMQ Website: http://www.rabbitmq.com ● RabbitMQ in Action, Manning Publications ● RabbitMQ in Depth, Manning Publications ● Workloads: https://github.com/rabbitmq/workloads ● Marcial Rosales developer guide: https://github.com/MarcialRosales/rabbitmq-developer-guide ● Jack Vanlightly Blogs: https://jack-vanlightly.com/ ● CloudAMQP Best Practices: https://www.cloudamqp.com/blog/index.html Where to Learn More
  • 23. Transforming How The World Builds Software © Copyright 2018 Pivotal Software, Inc. All rights Reserved.