Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Advanced Messaging with
       Apache ActiveMQ


Bosanac Dejan
May 2011




                                              ...
About me

 Bosanac Dejan
 Senior Software Engineer at FUSESource - http://
  fusesource.com
 Apache ActiveMQ committer ...
What we are going to cover?




       What is ActiveMQ
       The Basics
       Flow control
       Scaling
       H...
What is ActiveMQ?




                                                                                                    ...
Apache ActiveMQ



 Apache ActiveMQ
     • Leading Open Source messaging platform
     • Supported Java Standards:
      ...
Background




       ActiveMQ started in 2005 at CodeHaus
       Moved to Apache Software Foundation in 2006
       1,...
The Basics




                                                                                                           ...
Messaging is




 Loosely coupled exchange of messages between
  applications
 Location transparency
 Can be persistent...
Topics




                                                                                                               ...
Topics




 One message goes to 0-to-many consumers
  based on the current subscribers
 Think like mailing lists or disc...
Queues




                                                                                                               ...
Queues




    Messages are load balanced across many consumers
    Each message goes to exactly one consumer
    Consu...
Challanges


 Create a general messaging platform
 Support variety of use-cases
     • Large number of clients
     • La...
Flow Control




                                                                                                         ...
Flow Control - Why?

 Dealing with deep queues
 Dealing with slow consumers
 We want to prevent broker from being floode...
Flow Control - How?




 Message Cursors
 Producer Flow Control




                                                    ...
Flow Control - Cursors




                                                                                               ...
Flow Control - Store-based Cursor




                                                                                    ...
Flow Control - Non-persistent cursor




                                                                                 ...
Flow Control - Limits

                        Per destination
     <destinationPolicy>
           <policyMap>
           ...
Flow Control - Producer Flow Control

 Throttling producer speed to the speed of consumers




                          ...
Flow Control - VM Cursor




     <policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
       <pendingQueu...
Scaling




                                                                                                              ...
Scaling - Types




 Vertical scaling
 Horizontal scaling
 Traffic partitioning




                                   ...
Vertical Scaling




     Increase load capacity using a single broker can handle.



     Problems:
     • Thread count
 ...
Vertical Scaling - Threads




     What are threads used for
     • For Connections - Thread per Connection (blocking tra...
Vertical Scaling - Number of Connections




     Use non-blocking transport

     <transportConnectors>
       <transport...
Vertical Scaling - Number of Destinations




     Don’t use dedicated task runner

      ACTIVEMQ_OPTS="-Dorg.apache.acti...
Vertical Scaling - Memory


 Give broker enough memory

     ACTIVEMQ_OPTS="-Xmx2048M -Dorg.apache.activemq.UseDedicatedT...
Vertical Scaling - CPU




 Disable tight encoding

 It uses more CPU to create smaller packets

 uri = "failover://(tcp:...
Vertical Scaling - Conclusion




     There is a limit to the scalability a single machine can
                          ...
Horizontal Scaling


     Increase load capacity using networked brokers




  Concepts:
 Network of Broker




         ...
Horizontal Scaling - Network of Brokers




                                                                              ...
Horizontal Scaling - NOB Usage

                                                       Configuration

                <netw...
Horizontal Scaling - Conclusion




 More latency in processing messages
 Beware of complex topologies




             ...
Hybrid Scaling

     Partition traffic to more non-connected brokers




                                                 ...
Hybrid Scaling - Conclusion




 Pros
     • You can use all the tuning techniques used in Vertical scaling
     • Have b...
High Availability




                                                                                                    ...
High Availability




 Pure Master/Slave
 JDBC Master/Slave
 Shared File System Master/Slave




                      ...
Pure Master-Slave




                                                                                                    ...
Pure Master-Slave




 Shared nothing
 Fully replicated
     • All messages
     • All acknowledgements
     • All trans...
JDBC Master-Slave




                                                                                                    ...
JDBC Master-Slave




    Extreme reliability – but not as fast
    Recommended if already using an enterprise database
...
Shared Storage Master-Slave




                                                                                          ...
Shared Storage Master-Slave




    Recommended if you have a SAN
    No restriction on number of slaves
    Simple con...
Future




                                                                                                               ...
Future - ActiveMQ Apollo


 http://activemq.apache.org/apollo

 ActiveMQ 5.x reached scalability and performance
  limit...
Future - ActiveMQ Apollo




    Reactor Based Thread Model
    Scala 2.8 Implementation
    Protocol Agnostic
    RES...
Future - ActiveMQ Apollo Performance


                            http://hiramchirino.net/blog




                      ...
Conclusions




    Dynamic community
    Leading in terms of messaging innovation
    Built for Enterprise
    Scalab...
Questions?




 ActiveMQ Web sites:
     • http://activemq.apache.org/
     • http://fusesource.com/products/enterprise-a...
Nächste SlideShare
Wird geladen in …5
×

von

Advanced messaging with Apache ActiveMQ Slide 1 Advanced messaging with Apache ActiveMQ Slide 2 Advanced messaging with Apache ActiveMQ Slide 3 Advanced messaging with Apache ActiveMQ Slide 4 Advanced messaging with Apache ActiveMQ Slide 5 Advanced messaging with Apache ActiveMQ Slide 6 Advanced messaging with Apache ActiveMQ Slide 7 Advanced messaging with Apache ActiveMQ Slide 8 Advanced messaging with Apache ActiveMQ Slide 9 Advanced messaging with Apache ActiveMQ Slide 10 Advanced messaging with Apache ActiveMQ Slide 11 Advanced messaging with Apache ActiveMQ Slide 12 Advanced messaging with Apache ActiveMQ Slide 13 Advanced messaging with Apache ActiveMQ Slide 14 Advanced messaging with Apache ActiveMQ Slide 15 Advanced messaging with Apache ActiveMQ Slide 16 Advanced messaging with Apache ActiveMQ Slide 17 Advanced messaging with Apache ActiveMQ Slide 18 Advanced messaging with Apache ActiveMQ Slide 19 Advanced messaging with Apache ActiveMQ Slide 20 Advanced messaging with Apache ActiveMQ Slide 21 Advanced messaging with Apache ActiveMQ Slide 22 Advanced messaging with Apache ActiveMQ Slide 23 Advanced messaging with Apache ActiveMQ Slide 24 Advanced messaging with Apache ActiveMQ Slide 25 Advanced messaging with Apache ActiveMQ Slide 26 Advanced messaging with Apache ActiveMQ Slide 27 Advanced messaging with Apache ActiveMQ Slide 28 Advanced messaging with Apache ActiveMQ Slide 29 Advanced messaging with Apache ActiveMQ Slide 30 Advanced messaging with Apache ActiveMQ Slide 31 Advanced messaging with Apache ActiveMQ Slide 32 Advanced messaging with Apache ActiveMQ Slide 33 Advanced messaging with Apache ActiveMQ Slide 34 Advanced messaging with Apache ActiveMQ Slide 35 Advanced messaging with Apache ActiveMQ Slide 36 Advanced messaging with Apache ActiveMQ Slide 37 Advanced messaging with Apache ActiveMQ Slide 38 Advanced messaging with Apache ActiveMQ Slide 39 Advanced messaging with Apache ActiveMQ Slide 40 Advanced messaging with Apache ActiveMQ Slide 41 Advanced messaging with Apache ActiveMQ Slide 42 Advanced messaging with Apache ActiveMQ Slide 43 Advanced messaging with Apache ActiveMQ Slide 44 Advanced messaging with Apache ActiveMQ Slide 45 Advanced messaging with Apache ActiveMQ Slide 46 Advanced messaging with Apache ActiveMQ Slide 47 Advanced messaging with Apache ActiveMQ Slide 48 Advanced messaging with Apache ActiveMQ Slide 49 Advanced messaging with Apache ActiveMQ Slide 50 Advanced messaging with Apache ActiveMQ Slide 51
Nächste SlideShare
Messaging With ActiveMQ
Weiter
Herunterladen, um offline zu lesen und im Vollbildmodus anzuzeigen.

68 Gefällt mir

Teilen

Herunterladen, um offline zu lesen

Advanced messaging with Apache ActiveMQ

Herunterladen, um offline zu lesen

Features of ActiveMQ message broker for dealing with high load, scaling and high-availability

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Advanced messaging with Apache ActiveMQ

  1. 1. Advanced Messaging with Apache ActiveMQ Bosanac Dejan May 2011 A Progress Software 1 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  2. 2. About me  Bosanac Dejan  Senior Software Engineer at FUSESource - http:// fusesource.com  Apache ActiveMQ committer and PMC member  Co-author of ActiveMQ in Action A Progress Software 2 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  3. 3. What we are going to cover?  What is ActiveMQ  The Basics  Flow control  Scaling  High Availability  Future  Conclusion A Progress Software 3 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  4. 4. What is ActiveMQ? A Progress Software 4 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  5. 5. Apache ActiveMQ  Apache ActiveMQ • Leading Open Source messaging platform • Supported Java Standards: o JMS 1.1, J2EE 1.4, JCA 1.5 and XA  Reliable, high performance messaging • Out-performs many legacy proprietary message queues • Configurable for many different deployments  Multi-Protocol/Multi-Language Support A Progress Software 5 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  6. 6. Background  ActiveMQ started in 2005 at CodeHaus  Moved to Apache Software Foundation in 2006  1,117,537 lines of code  24 committers  Now the most widely used open source messaging system on the planet A Progress Software 6 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  7. 7. The Basics A Progress Software 7 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  8. 8. Messaging is  Loosely coupled exchange of messages between applications  Location transparency  Can be persistent or non-persistent  Can be transactional A Progress Software 8 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  9. 9. Topics A Progress Software 9 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  10. 10. Topics  One message goes to 0-to-many consumers based on the current subscribers  Think like mailing lists or discussion forums  Ideal for publishing business events  Distributed observer pattern  Allows one part of your system to notify anyone else who may be interested in an event A Progress Software 10 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  11. 11. Queues A Progress Software 11 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  12. 12. Queues  Messages are load balanced across many consumers  Each message goes to exactly one consumer  Consumers compete for messages  Its easy to browse and monitor queues  Ideal for grid style applications A Progress Software 12 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  13. 13. Challanges  Create a general messaging platform  Support variety of use-cases • Large number of clients • Large number of destinations • Slow consumers  Provide enterprise feaures • Security • High availability • Management • etc A Progress Software 13 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  14. 14. Flow Control A Progress Software 14 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  15. 15. Flow Control - Why?  Dealing with deep queues  Dealing with slow consumers  We want to prevent broker from being flooded with messages  We want to prevent broker running out of memory and other resources A Progress Software 15 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  16. 16. Flow Control - How?  Message Cursors  Producer Flow Control A Progress Software 16 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  17. 17. Flow Control - Cursors A Progress Software 17 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  18. 18. Flow Control - Store-based Cursor A Progress Software 18 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  19. 19. Flow Control - Non-persistent cursor A Progress Software 19 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  20. 20. Flow Control - Limits Per destination <destinationPolicy> <policyMap> <policyEntries> <policyEntry queue=">" memoryLimit="10mb"/> </policyEntries> </policyMap> </destinationPolicy> System settings <systemUsage> <systemUsage> <memoryUsage> <memoryUsage limit="256 mb" /> </memoryUsage> <storeUsage> <storeUsage limit="100 gb" /> </storeUsage> <tempUsage> <tempUsage limit="10 gb" /> </tempUsage> </systemUsage> </systemUsage> A Progress Software 20 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  21. 21. Flow Control - Producer Flow Control  Throttling producer speed to the speed of consumers A Progress Software 21 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  22. 22. Flow Control - VM Cursor <policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb"> <pendingQueuePolicy> <vmQueueCursor/> </pendingQueuePolicy> </policyEntry> A Progress Software 22 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  23. 23. Scaling A Progress Software 23 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  24. 24. Scaling - Types  Vertical scaling  Horizontal scaling  Traffic partitioning A Progress Software 24 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  25. 25. Vertical Scaling Increase load capacity using a single broker can handle. Problems: • Thread count • Memory usage • CPU usage A Progress Software 25 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  26. 26. Vertical Scaling - Threads What are threads used for • For Connections - Thread per Connection (blocking transport) • For Dispatching - Thread per Destination A Progress Software 26 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  27. 27. Vertical Scaling - Number of Connections Use non-blocking transport <transportConnectors> <transportConnector name="nio" uri="nio://0.0.0.0:61616"/> </<transportConnectors> Enables handling large number of clients A Progress Software 27 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  28. 28. Vertical Scaling - Number of Destinations Don’t use dedicated task runner ACTIVEMQ_OPTS="-Dorg.apache.activemq.UseDedicatedTaskRunner=false" Use optimized dispatch for queues <destinationPolicy> <policyMap> <policyEntries> <policyEntry topic=">" optimizedDispatch="true"> ... </policyEntry> </policyEntries> </policyMap> </destinationPolicy> A Progress Software 28 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  29. 29. Vertical Scaling - Memory  Give broker enough memory ACTIVEMQ_OPTS="-Xmx2048M -Dorg.apache.activemq.UseDedicatedTaskRunner=false"  Configure big enough memory usage <systemUsage> <systemUsage> <memoryUsage> <memoryUsage limit="1024 mb" /> </memoryUsage> ... </systemUsage> </systemUsage> A Progress Software 29 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  30. 30. Vertical Scaling - CPU  Disable tight encoding It uses more CPU to create smaller packets uri = "failover://(tcp://localhost:61616wireFormat.tightEncodingEnabled=false)"; A Progress Software 30 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  31. 31. Vertical Scaling - Conclusion There is a limit to the scalability a single machine can give A Progress Software 31 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  32. 32. Horizontal Scaling Increase load capacity using networked brokers Concepts:  Network of Broker A Progress Software 32 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  33. 33. Horizontal Scaling - Network of Brokers A Progress Software 33 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  34. 34. Horizontal Scaling - NOB Usage Configuration <networkConnector name="broker1-broker2" uri="static:(tcp://broker2:61617)" dynamicOnly="true" prefetchSize="1000" conduitSubscriptions="true" decreaseNetworkConsumerPriority="true" suppressDuplicateTopicSubscriptions="true" networkTTL="3"> </networkConnector> Connecting failover://(tcp://broker1:61616,tcp://broker2:61616)?randomize=true A Progress Software 34 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  35. 35. Horizontal Scaling - Conclusion  More latency in processing messages  Beware of complex topologies A Progress Software 35 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  36. 36. Hybrid Scaling Partition traffic to more non-connected brokers A Progress Software 36 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  37. 37. Hybrid Scaling - Conclusion  Pros • You can use all the tuning techniques used in Vertical scaling • Have better Horizontal scaleability than using Network Of Brokers (Less broker cross talk)  Cons • Added complexity required on the end user Application A Progress Software 37 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  38. 38. High Availability A Progress Software 38 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  39. 39. High Availability  Pure Master/Slave  JDBC Master/Slave  Shared File System Master/Slave A Progress Software 39 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  40. 40. Pure Master-Slave A Progress Software 40 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  41. 41. Pure Master-Slave  Shared nothing  Fully replicated • All messages • All acknowledgements • All transactions  Slave does not start any transports or network connections A Progress Software 41 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  42. 42. JDBC Master-Slave A Progress Software 42 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  43. 43. JDBC Master-Slave  Extreme reliability – but not as fast  Recommended if already using an enterprise database  No restriction on number of slaves  Simple configuration A Progress Software 43 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  44. 44. Shared Storage Master-Slave A Progress Software 44 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  45. 45. Shared Storage Master-Slave  Recommended if you have a SAN  No restriction on number of slaves  Simple configuration  N.B. – ensure file locking works – and times out – NFSv4 good! A Progress Software 45 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  46. 46. Future A Progress Software 46 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. FuseSource Company A Progress Software Company
  47. 47. Future - ActiveMQ Apollo  http://activemq.apache.org/apollo  ActiveMQ 5.x reached scalability and performance limits with the current architecture  New broker core A Progress Software 47 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  48. 48. Future - ActiveMQ Apollo  Reactor Based Thread Model  Scala 2.8 Implementation  Protocol Agnostic  REST Based Management A Progress Software 48 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  49. 49. Future - ActiveMQ Apollo Performance http://hiramchirino.net/blog 10 producers/10 consumers single topic using Stomp 20 byte payload A Progress Software 49 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  50. 50. Conclusions  Dynamic community  Leading in terms of messaging innovation  Built for Enterprise  Scalable, Good Performance, Reliable A Progress Software 50 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  51. 51. Questions?  ActiveMQ Web sites: • http://activemq.apache.org/ • http://fusesource.com/products/enterprise-activemq/  Blog: • http://www.nighttale.net/  Twitter: • http://twitter.com/dejanb • http://twitter.com/fusenews A Progress Software 51 Copyright © 2011 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved. Company
  • jadeflon

    Oct. 24, 2017
  • AmarBabuKakumanu

    Oct. 12, 2017
  • EminBAHADIR

    Sep. 17, 2017
  • kunkunur

    Jun. 2, 2017
  • DariuszFurmaniak

    May. 6, 2017
  • hyh961

    Apr. 15, 2017
  • TatuMikkonen

    Mar. 11, 2017
  • sharmisthamona

    Feb. 16, 2017
  • rafaelcba

    Dec. 30, 2016
  • AlexMart001

    Nov. 1, 2016
  • MerkerXu

    Sep. 14, 2016
  • MuralimohanDintakurt

    Aug. 28, 2016
  • pvmsikrsna

    May. 12, 2016
  • PhaniTurlapati

    Apr. 14, 2016
  • jesusvegagonzalez

    Apr. 5, 2016
  • AlejandroFlores158

    Mar. 13, 2016
  • hyoungseunglee

    Mar. 2, 2016
  • ssuser52c883

    Feb. 22, 2016
  • moviesnmusic

    Dec. 16, 2015
  • AdeLucas1

    Nov. 18, 2015

Features of ActiveMQ message broker for dealing with high load, scaling and high-availability

Aufrufe

Aufrufe insgesamt

40.661

Auf Slideshare

0

Aus Einbettungen

0

Anzahl der Einbettungen

5.205

Befehle

Downloads

1.150

Geteilt

0

Kommentare

0

Likes

68

×