Weitere ähnliche Inhalte Mehr von Joshua Long (20) Kürzlich hochgeladen (20) Dynamic Routing at 1 Million Messages per Second with Spring Integration1. Dynamic Routing at 1 Million
Messages per Second
with Spring Integration
John Davies, CEO, Incept 5
@jtdavies
!
Josh Long, Spring Developer Advocate, Pivotal
@starbuxman
2. S P R I N G I N T E G R AT I O N 1 0 1
!2
3. what is “EAI”?
“
EAI encompasses approaches,
methodologies, standards, and
technologies allowing very
diverse but important systems
to share information, processes,
and behavior in support of the
core business.
-David Linthicum
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
”
4. what is “EAI”?
§ file transfer
§ shared database
§ remote procedure call
§ messaging
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
6. Spring Integration
n (n-1)
2
§ At the core, an embedded Message Bus
§ Also, an Application Integration
Framework
§ Support Enterprise Integration Patterns*
via the Spring programming model
§ Build on the Spring Platform
B
C
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
A
D
7. Spring Integration
§ Payload can be any object
§ Header values are stored in a read-only java.util.Map<K,V>
§ Similar to JMS messages, email mime envelopes, XMPP messages, etc.
public interface Message<T> {
MessageHeaders getHeaders();
T getPayload();
}
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
8. Spring Integration
§ Decouples producers from consumers
§ provide extension points for interceptors
§ messages may be queued and buffered
{
!
!
!
point to point
!
{
!
!
!
publish-subscribe
!
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
producer
send(Message)
message channel
receive(Message)
consumer
<channel id="async-p2p">
<dispatcher task-executor=”simpleAsyncThreadPool" />
<queue capacity=“10” />
</channel>
consumer
producer
send(Message)
message channel
consumer
receive(Message)
<publish-subscribe-channel
id="async-pubsub”
task-executor="someThreadPool" />
consumer
9. Spring Integration
transformers: convert payloads and modify headers
filter: discard messages based on test
router: determine next channel based on message and test
splitter: generate multiple messages from one
aggregator: assemble a single message from multiple messages
service activator: delegates processing to a backend service call
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
10. Spring Integration
<channel id = “customerChannel” />
!
<jms:outbound-channel-adapter
channel = “customerChannel” destination-name=“stocks”
connection-factory = “connectionFactory” />
!
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
jms:outbound-adapter
producer
send(Message)
jmsTemplate.send(javax.jms.Message)
ActiveMQ
11. Spring Integration
<channel id = “fileChannel” />
!
<file:inbound-channel-adapter channel = "fileChannel "
directory= "#{systemProperties['user.home']}/Desktop/in ">
<int:poller fixed-rate="1000"/>
</file:inbound-channel-adapter>
Document
Document
Document
new File(directory).list()
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
file:inbound-channel-adapter
!
Message<File> mf = fileChannel.receive()
consumer
12. Spring Integration
!
MessageChannel channel = ...
Message<Customer> customerMessage =
MessageBuilder.withPayload(customer)
.setHeader("customer-id", 10)
.build();
channel.send(customerMessage);
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
14. We need a good safety margin
• If we can process 200k/sec and we get a peek of 400k for an hour
(e.g. Christmas, Thanksgiving, breaking news etc.)
• We have a backlog of 200k/sec for 3600 seconds, if the volume drops to
150k/sec it will take 4 hours to catch up
• And we’ll need 360 GB of RAM to queue it! And that’s just 500 bytes/msg
!
• RabbitMQ was used at a large “fruit” vendor
• Queue sizes were critical in the planing of the deployment
!
• Even working at 500k/sec and a peak of 550k for 30 minutes
dropping to 400k
• 15 minute’s delay and 45 GB of RAM
!
• We need “REALTIME”
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
15. Where’s the problem?
• In order to do any work on a message we need to parse it
• For XML that’s a SAX parser, DOM or Java Binding (JAXB, JIBX etc.)
!
• If you’re going to store, search or deliver messages these are
the basic steps…
• Parse the message or part of the message you need
• Query and route the message or…
• Query to search the message
• - Possibly serialise and de-serialise the message
• - Possibly enrich or transform the message
• Write the message out (if it’s been bound to Java)
!
• All this takes time :-(
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
16. Java Binding
• Keeping the example simple a CSV generates something like
this…
public class Row extends ComplexDataObject {!
private static final long serialVersionUID = 1L;!
private String name;!
Name,Card Number,Expiry Date,Amount,Currency,Transaction Date,Commission,Vendor ID,Country!
private String cardNumber;!
Stephen Hawkins,4325-6486-3757-2674,10/06,100,GBP,26-09-2006,2,14988603,UK!
private String expiryDate;!
Tim Berners-Lee,4724-7345-4725-7833,11/07,258,USD,21-12-2006,5,15688632,UK!
Bill Clinton,4924-7264-1264-8532,04/09,1250.6,USD,13-09-2006,15,66846035,US!
private double amount;!
Angela Merkel,4457-4356-0087-0107,05/08,350,EUR,13-11-2006,7,93440252,DE!
Richard Branson,4724-7345-4725-7833,11/06,250,USD,22-02-2006,5,14988103,UK!
private boolean isamountSet;! Tim Cook,4924-7264-1264-8532,04/09,12250,USD,16-09-2006,1.3,67434435,US!
Alan Turing,4325-6486-3757-2674,10/06,50,GBP,23-02-2006,1,14119663,UK
private String currency;!
private Date transactionDate;!
private double commission;!
private boolean iscommissionSet;!
private long vendorID;!
private boolean isvendorIDSet;!
private String country;
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
17. Message Parsing
• Taking a message with about 50-60 attributes and parsing
them so that they can be available to Spring Integration/Batch
is not exactly complex
!
• But providing the ability to manage the changes and dozens
of standards starts to add complexity
• Things change over time, we need to manage that change too
• We’d also like a consistent API so Java-Binding is still ideal
!
• Two relatively simple examples from the Telco and Financial
Services industry are RADIUS (rfc-2865) and FIX (from FPL)
• Others include ASN.1 & ISO-8583
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
18. Radius (rfc-2865)
• Remote Authentication Dial In User Service (RADIUS)
• 77 pages of binary spec...
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
19. Modelling in C24 first
•The RADIUS standard...
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
20. What we can now do
• The spec may say that bit 13 of a 32 bit field represents the
presence of a field ABC (later in the message)
• Programatically we can test bit 5 with a mask 0x00002000
• Using the generated code we can simply call isAbcSet()
!
• It may say that bits 4-6 represent the version ID
• Programatically we can mask it and then shift it...
• mask 0x00000070
• shift >> 4
• Using the generated code we can simply call getVersionId()
!
• We now have a nice API that hides the complexity of the
binary implementation
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
21. Bits and Bytes to…
• We can now go from this…
!
jd-server:Radius TestData jdavies$
00000000 01 02 00 74 ea d5 7c 62
00000010 35 25 e5 8c 1a 17 00 00
00000020 35 37 30 36 32 37 38 38
00000030 31 35 39 30 36 32 35 38
00000040 33 34 32 31 30 32 30 39
00000050 34 37 30 30 34 31 38 38
00000060 af 08 07 32 33 34 33 35
00000070 4e 97 57 a8
00000074
hexdump -C radius.dat
1f d0 f6 fe a3 bf 36 4c
28 af 01 11 32 33 34 34
35 33 36 01 11 32 33 34
38 35 33 36 1f 11 33 35
34 35 35 36 38 5e 0e 34
36 37 33 1a 0d 00 00 28
06 06 00 00 00 02 37 06
!
• To being able to use it in Spring
• Or Mule, Fuse, Camel etc…
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
|...t..|b......6L|
|5%......(...2344|
|57062788536..234|
|159062588536..35|
|3421020945568^.4|
|47004188673....(|
|...23435......7.|
|N.W.|
22. To Spring Integration...
• Test bits and binary data in native Spring…
!
• <filter input-channel="filter-message-channel" outputchannel="process-message-channel" ref=”payload”
method=”isAbcSet"/>
!
• <filter input-channel="filter-message-channel" outputchannel="process-message-channel"
expression="payload.versionId == 5"/>
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
23. A good start but…
• We now have a pretty cool API and can use Spring but it’s
still pretty slow as it generates LOTS of objects
!
• With our telco standards binary to Java binding was “slow”
due to the number of objects, we got around 20k/sec/core
• Fine for most purposes and with an 8 core machine potentially good
for 100k/sec but we wanted better
!
• We needed a better solution, it wasn’t the parsing but the
complexity of the objects we were creating and parsing into
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
24. ByteBuffer
• Java 1.4 added java.nio.ByteBuffer
• Basically a wrapper for a byte[]
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
25. Lazy parsing
• Rather than parse all of the elements from the message
every time we retrieve the elements only when we need
them
• Similar to comparing a DOM with a SAX parser
• We assume that we will only need to filter/sort/query on a limited
number of fields so we save a lot of redundant parsing
!
• Performance goes from ~20k/sec (50µs per message) to
~1 million/sec (1µS per message), some 50 times faster
• Even parsing the entire message is significantly faster
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
26. Here’s where it gets interesting...
• So we now have a Java object with a ByteBuffer holding the
message data and dozens of get() methods to get the
content
• Message call = new Message(data);
• call.getDuration();
• This works pretty fast, plus the JIT compiler kicks in after
10,000 iterations and optimises the method
bottleneck moves to the SpEL queries
• Now theinput-channel="filter-message-channel" output-channel="process-message-channel"
<filter
•
expression="payload.duration lt 0.1"/>
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
27. Reflection
• What’s happening under the hood is reflection, very powerful
but sadly still rather “slow”
• expression="payload.duration lt 0.1"
• Turns into something like...
!
!
!
• This adds about 700nS to each message
• A few of these and we’ve more than halved the performance
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
28. The Java Compiler to the rescue!
• New from Java 1.6: ToolProvider.getSystemJavaCompiler();
• We can create a generic accessor for double values...
!
!
• Can now write a class on the fly that implements this
method for the getter we want e.g. getDuration()
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
29. Java on the fly...
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
31. Running it...
• Load up the compiled class (or byte-code) and run it
!
!
• Note however that the first two lines (above) are done only
once, outside of the loop
!
• The result is “native” performance as if it was code
• Which of course it is
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
32. Back to spring...
• SpEL expressions are currently interpreted
!
• There is some basic caching
• java.lang.reflect.Field
• java.lang.reflect.Method objects are cached once discovered and
reused on subsequent evaluations
!
• But overall they are “slow” (for what we need)
• We need to look at avoiding reflection by compiling the parsed
expression into a class
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
33. SpEL compiler
• Currently proposed by Andy Clement (Spring guru) is a SpEL
compiler
• With the proposed changes SpEL has a Mixed Mode Interpreter
(MMI) system in place
• Mixed means it mixes the current interpreter with a real expression
compiler
• When an expression is evaluated X number of times SpEL compiles it to byte-code and
uses that going forward
• User does nothing, evaluations just accelerate
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
34. Does it make a difference?
• Property access:foo.bar.boo
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
35. Does it make a difference?
• Method invocation:hello()
!
!
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
36. Advantages - Performance
• We can now parse, compare and filter in around 1 µSec (1
million/sec) per core
• Make decisions on what you want early on and drop unwanted
messages
• Provide “on-the-fly” aggregation
• Around 90-95% less CPU per message - $€£¥₹
!
• Disadvantages
• Less error checking
• Hardware vendors make less money
• Machine rooms become uncomfortably cold
• Power bills go down
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
37. Other Advantages
• Messages in memory are binary encoded in ByteBuffers and
so occupy a fraction of the size of a bound Java object
• Typically around 1/12 of the size, using only 8% of the memory
!
• We can queue around 10 times more messages per unit of
memory
!
• We get around 1000% capacity advantage for in-memory
grids - GigaSpaces, Hazelcast, GemFire, Coherence
!
• We can provide custom serialisation of the ByteBuffer for all
of the above - boosting node to node replication
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
38. So - In a nutshell
• We’ve gone from 20k per second to around 800k
• 40 times faster, sometimes better
!
• We’ve gone reduced memory footprint by around 12 times
• Meaning you can store 12 times more data in your grid
• Or use 1/12th of the hardware/memory
!
• You can route this with Spring at “native” JIT-compiled Java
code speeds
• So far no other framework can offer this level of efficiency, dynamic
flexibility and performance
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
39. The Future
• SpEL to native Java compilation to be included in Spring
release
• A few issue remain to be resolved in the short term but early versions
look good
!
• Full support for standards like ASN.1
• This is a standard wire-level implementation option for ISO-20022
• ASN.1 supports multiple encoding rules, Basic, XML, Canonical, Packed
etc.
!
• XML messages encoded as packed ASN.1 (PER) would be
almost as performant as raw binary
• Complex XML as packed binary in a ByteBuffer
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
40. More Spring
• Read more on the tools we used...
!
• Spring XD - http://projects.spring.io/spring-xd/
!
• Spring Integration - http://projects.spring.io/spring-integration/
!
• Spring Batch - http://projects.spring.io/spring-batch/
!
• Spring AMQP - http://projects.spring.io/spring-amqp/
Copyright © 2013 Incept5 Ltd. http://www.incept5.com
41. Thank you!
+
Josh Long
John Davies
Twitter: @starbuxman
Twitter: @jtdavies
GitHub: http:// github.com/joshlong
LinkedIn: http://linkedin.com/in/jdavies/
E-Mail: jlong@gopivotal.com
E-Mail: John.Davies@C24.biz
Spring & Spring Integration:
http://spring.io
C24: http://www.c24.biz
Copyright © 2013 Incept5 Ltd. http://www.incept5.com