From ChicagoCoders conference 2014

1. Using Apache Camel a bit like AKKA
And debunking some SOA stuff
Johan Edstrom
SOA Executive and Apache developer.
Apache Camel PMC
Apache ServiceMix committer
Original CXF Blueprint Author
Cassandra client library developer
Author
jedstrom@savoirtech.com
joed@apache.org

2. Savoir Technologies - This is where we started

3. Where are we now?

4. Well, what happened there?
We kinda started focusing exclusively on scalability
• We went from the majority of our customers wanting “ESB”
• Or tune our “Messaging”
To
• Very modular software
• Infrastructure design that would last a while
• Key concept structures that were re-usable
• We’ve gotten quite a bit of experience here over the years

5. So what is that scalability we talk of?
The right resources in the right place
• Tomcat doesn’t work better with 1024 threads
Messaging
 Writing QueueBrowsers to see where messages go isn’t awesome
State driven - Not mentioned in EIP Patterns (Yet)
Conversational - Not mentioned in EIP Patterns (Yet)
Effectively aggregating - Don’t use a RDBMS as a mutex
• select for update 2000 times a second doesn’t scale
Transactionally safe - And that doesn’t have to mean XA
Correctly utilizing EE
• Connectionpools aren’t for cowards

6. This is what we are trying to avoid....

7. What is the AKKA statement?

8. So where does AKKA come in?
It could be part of your deployment in Apache Karaf
It could be stand-alone
What is it that we really think is cool?
• Asynchronous systems!
• Actors
• Supervisors
• Scaling
• Immutability
Is that the goal for every Java shop out there?

9. So what does that AKKA look like?

10. Or maybe a bit more complicated?
So what is going on here?
• ActorSystem - umbrella naming
• System bound error handling
• Logging
• Scaling
• Then we introduce some EIP Patterns like “stuff”
 We RoundRobin and allow it to scale
 And we finally mount it

11. Are you trying to write Camel stuff?
Sure
• And yes I know that Camel-AKKA exists or is it AKKA-Camel now?
• I know there is a Scala DSL for Camel - fixed a few things there…
• What about the Scalaz stuff?

12. So…. That was really cool. But….
All projects aren’t greenfield
• Can we refactor?
We have existing staff and investments
• Is it easy to train
We want to do this in pieces
• We really want to try, test, promote
Could we do this with the existing EE Infra we have?

13. Let’s pretend you have Apache Karaf
This is not far from an EE container
• Add OpenEJB if you want and you’ll have a full stack
It contains a web-services layer
Dependency injection services
OSGi integration and legacy (JBI) support (Don’t use that. - Seriously…)
Camel for routing, transformation and mediation
Aries and Spring for EE support, JNDI, TX, JPA
Web-app deployment support

14. And we’ll add Apache ActiveMQ
This is another presentation :)
• Jeff did that one earlier today, I hope you did attend!

15. And I presume ya’ll know Apache Camel
Camel is a Domain Specific Language (DSL) focused on
implementing Enterprise Integration Patterns (EIPs)
• Examples: multicast, splitter, content-based router, routing slip, “dynamic
routers”, aggregator
EIPs are implemented by Camel “Processors”
• Users can develop their own processors
• A processor is a fundamental building block
• Bean language and bindings exists so that not a single piece of Apache Camel
Imports will be necessary when integrating your existing code
Camel can connect to a wide variety of integration technologies
• Examples: JMS, HTTP, FTP, SOAP, File - There are ~ 180 components
• Integrations are implemented by Camel “Components”
• Users can develop their own components

16. Apache Camel

17. Do I need all of that?
Nope, many solutions will need just a few things
• jaxb, camel, jms, soap, rest and perhaps jdbc
• Cut the container down to fit your needs
• We don’t need to load all of the 100+ Apache Camel components
• Pick and choose!
Should I run that messaging solution inside the “ESB”
• Entirely up to you, let us look a little deeper at that in a sec.
Can I test these solutions or am I stuck with
System.out.println and a remote debugger?

18. I’ve heard that OSGi sucks?
Yes, It is a bit more complex to manage a full lifecycle
• On the other hand you get reliable hot-deployment and things like
configuration, service management and service subscriptions out of the box.
Yes, package level import export can be confusing
• So make sure you use packages in a nice modular way.
Yes, it takes a while to learn the tooling
• Then did you wake up one day just knowing that WAR or EAR layout?
Yes, you need to learn the classpath and a bit about
classloading
• Which’ll make you a better developer in the end of the day.
• Many problems in this area actually are created by applying EE classloaders
to an OSGi environment.

19. Decoupling with OSGi
Enforces knowledge of imports and exports
Allows us to version
Programming model with micro-services allows
for re-use on an Api level vs code level
Promotes contracts

20. Use case #1
We want to accept incoming SOAP requests
We need to inspect those HTTP headers and then use them
to route
We want this to be dynamic and flexible
 We might want to add a more complex routing engine later
We want this to be modular and possible to run across
multiple systems

21. Verify your use-cases!

22. UseCase #1 outline
Let us use some EIP pattern symbols to outline this
• We haven’t really made any technology statements here
 We are going to use Apache ActiveMQ because I said so.
 It could easily have been any other JMS provider.
• We know we’ll have a synchronous part
 Web-services by nature are.
• We’ll try and make all the other parts asynchronous
• We’ll also make sure that the participants don’t need strong integration or
visibility of each other, thus we’ll (hopefully) end up with some pretty re-usable
modules
• Another good approach is to also use sequence diagrams

23. What else do we want here?
A “Dynamic” Router!
• We’ll craft a simple header based Dynamic router from eip patterns.
 Technically it is in between a routing slip with termination and a real
control channel pattern, we utilize JMS as our control channel for this.
 We use this example to play with destinations from SoapUI
Remember that ActorSystem in AKKA?
• This dynamic router is a big part of “pretending” that we are doing that

24. The “Dynamic Router”
Yep, it is just a Java class.
• Imagine how easily you could integrate this with.... Anything!

25. Why build that type of a route?
We are able to turn “endpoints”, “routing”, “systems” or
whatever you want to call it into simple destination names.
We only need one route…..
We can break all these modules apart

26. So let’s sum up what we have at this point
We defined a CXF (JAX-WS) Endpoint, generated stubs and
tied it into Camel
• The MEP (Message Exchange Pattern) is IN/OUT since we use CXF
We then dropped in a router “Our Dynamic Router”
• It’ll listen to a JMS queue called recognizer
• Then it’ll start inspecting headers for a location

27. What about that testing?
We are going to be dealing with an OSGi environment.
• Like an EE container it makes testing a tad less fun....
Full blown container testing with something like SoapUI
• Can be scripted and automated in Maven
• Or you could BDD / Cucumber script it
Full OSGi tests with Pax-Exam
• Allows you to wire up a full container
Simple OSGi registry tests with PojoSr
• Let’s you simulate an environment so you can test BluePrint!
Functional Junit/TestNG tests with Camel-Test.
• Should be combined with at least one of the above!

28. Testing the router
First of, let us just do a Camel Test, we want to verify that
the Dynamic routing works as expected.
• As you can see, none of these routes know about each other.
 And if you can’t see that, they still don't.
Keep this
in mind!!

29. How we’ll be using this dynamic data
We basically can use this to write completely dynamic steps
that will restart on a configuration change from
• File changes
• Code changes
• JMX changes

30. Proof of concept outline
We now have the following modules
• customer-service : A model library
• customer-service-ws : The webservices endpoint
• dynamic-routing : A generic module for routing
• response-builder : A simple module that returns a payload
• That means that all of the needed patterns are fulfilled.
• Competing consumer is a natural thing in JMS - KACHIIIIING!!!!
• Eventing is a natural part of Apache Camel - KACHIIIIING!!!!
• Request Reply is “Handled for us” - KACHIIIIING!!!!
• Dynamic config - KACHIIIIING!!!!
•

31. So why didn’t we just slap a camel route in
Tomcat?
That’s why.

32. Otay - so what the heck did that have to do with
AKKA
I’m glad you asked!
• We have, thanks to putting it on ActiveMQ
 Immutability
 Scalability
 Asynchronous behavior
• We have a separation of Concerns - Kinda like the Actor
 Are we really there tough?
• Not yet!

33. Do we think we can make that “Actor”
Lets make it very generic
Lets make it really dynamic
Lets use the power of OSGi to “do stuff”
The basic premise now is -
• JMS is our “Actor System”
• Camel handles Messaging and in/out for us as well as a lot of nice to have
stuff like errorhandling, graceful shutdown, startup, validation and so on.
• We want a “generic” way of handling say… Objects?

34. Second use-case!!!
We are gonna take what we learned and build parts of a
dynamic ETL system - Basically CSV files

35. What is a mutator in this ETL system?

36. And how is it working?
Yey! We are manipulating strings!
• OMG, Such data….
• Kidding aside - we now have a tool where all the transformations can be
described, this system contains OGNL, pattern matchers, regular expression
engines and as we can see the MutationConfig contains factories for invoking
these guys.

37. Lets stick these in Camel (MutationManager)

38. See that Supervisor destination?
The “brian” of the whole system.
• Checks if this is an initial record
• Checks if there is pre-cleansing needed
• Then goes onto in some sub-processors talking to rules and persistence to
build a chain of MutationSets

39. Then we add an OSGi - Manager
(ServiceFactory)

40. This is purely config driven in OSGi

41. And the Master system initiation

42. And showing all the moving parts

43. Running the system

44. How did it get to Mock?
System wide “Role” for that particular incoming source file
was established, it was added last in line.
The roles are just written to a Cassandra layer, the
supervisor and the MutationManager executes them.

45. And the operations?
Runtime loaded
• Part of a config chain - Applied as roles in a processing chain
• Processing chain determined from config and built by the ServiceManager(s)
• Can be hot deployed and loaded
• Can be changed on running processing

46. End result?
100% Event driven
100% Asynchronous
No camel knowledge really needed
Competing consumer - Horizontally scalable
Hot-Deployable
• And most of the cool stuff I can’t even show you…..
• This pattern is quite well documented in OSGi, not that often combined with
Camel but you end up with all of the parts that really are interesting in
something like AKKA

47. And what else is Savoir doing?
https://github.com/savoirtech/aetos
• Cooler than Karaf :)
https://github.com/savoirtech/hecate
• Hector based Pojo Mapper
 Object Graph
 Annotations
• CQL3 based Pojo Mapper
 Object Graph
 Annotations
 True collection handling
 column adapters
https://github.com/seijoed/hazelcast-discovery
• HazelCast based ActiveMq discovery system

48. We might be releasing
Polymorphic CRUD
• Based on CXF
• Fully Cassandra (or whatever) backed
• Event driven
• Single bean wiring with entity Class

49. And we might be looking more at AKKA :)

50. Questions?