Apache Camel Design Patterns
Learned Through Blood, Sweat, and Tears
June 2016
Bilgin Ibryam
@bibryam

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears2
Bilgin Ibryam
● Senior Middleware Architect at Red Hat UK
● Apache Camel Committer and PMC member
● Apache OFBiz Committer and PMC member
● Author of Camel Design Patterns (new)
● Author of Apache Camel Message Routing
● Twitter: @bibryam
● Email: bibryam@gmail.com
● Blog: http://ofbizian.com
● LinkedIn: http://www.linkedin.com/in/bibryam

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears3
Apache Camel Project Status
It has all necessary ingredients for a successful open source project.
● Community: 52 committers, 903 users
● Support by large vendors (Red Hat)
● Connectors (256), DataFormats (40)
● Enterprise Integration Patterns++
● Domain Specific Language
● Ecosystem: Karaf, ActiveMQ, CXF, Fabric,
Hawtio, Spring and others
● Monolith, SOA, Microservices, Serverless
Source https://www.openhub.net/p/camel/
Stats Date: 06/06/2016

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears4
Application Integration with Camel
What do you need to know to create great Camel applications?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears5
Happy Path Scenarios
How Pipes and Filters Pattern looks like in Camel?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears6
VETRO
What is a typical processing flow for a Camel route?
● Validate: validation, schematron, MSV, Jing, bean validation components
● Enrich: enrich and pollEnrich EIPs, custom beans
● Transform: Data formats, auto type conversion, templating components
● Route: Message routing EIPs
● Operate: this is the essence of the processing flow

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears7
Edge Component
Let's start with a simple Camel route that consumes files

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears8
Edge Component
How to expose the same business functionality to multiple consumers?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears9
Edge Component
Encapsulate endpoint-specific details and prevent them from leaking into
the business logic of an integration flow.

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears10
Read vs Write Operations
How to evolve Read and Write operations independently?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears11
CQRS
This decouples read from write operations to allow them to evolve
independently.

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears12
Unhappy Path Scenarios
Happy paths are the easy ones. More work is required for designing and
implementing the unhappy paths.
● Data Integrity Pattern
● Saga Pattern
● Retry Pattern
● Idempotent Filter Pattern
● Circuit Breaker Pattern
● Error Channels Pattern
● Throttling Pattern

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears13
Data Integrity
How hard can it be to copy files from one location to another?
Download Data Integrity Chapter: http://bit.ly/came-design-patterns-sample

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears14
Data Integrity
Transactional systems
Local transaction manager
Global transaction manager

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears15
Saga
How to avoid distributed transactions and ensure data consistency?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears16
Saga
Ensures that each step of the business process has a compensating action
to undo the work completed in the case of partial failures.

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears17
Retry
To enable applications handle anticipated transient failures by transparently
retrying a failed operation with expectation it to be successful.
● Which failures to retry?
● How often to retry?
● Idempotency
● Monitoring
● Timeouts and SLAs

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears18
Retry
Camel RedeliveryPolicy
● The most well known retry mechanism in Camel
● Retries only the failing endpoint
● Fully in-memory
● Thread blocking behavior by default
● Can be asynchronous
● Good for small number of quick retries (in milliseconds)

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears19
Retry
ActiveMQ consumer RedeliveryPolicy
● Retries the message from the beginning of the Camel route
● Not used very often, but enabled by default
● Fully in-memory
● Thread blocking by default
● Good for small number of quick retries (in milliseconds)

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears20
Retry
ActiveMQ Broker Redelivery
● ActiveMQ specific and requires custom logic
● It will consume the message again from a queue
● Persisted at the broker rather than application memory
● Can be clustered and use fail over, load balancing, etc
● Good for long persisted retries (in minutes or hours)

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears21
Circuit Breaker
How to guard a system by cascading failures and slow responses from
other systems?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears22
Circuit Breaker
Improves the stability and the resilience of a system by guarding integration
points from cascading failures and slow responses.
Closed state
Open state

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears23
Circuit Breaker
Improves the stability and the resilience of a system by guarding integration
points from cascading failures and slow responses.

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears24
Circuit Breaker
Two Circuit Breaker Implementations in Camel 2.18

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears25
Bulkhead
How to enforce resource partitioning and damage containment in order to
preserve partial functionality in the case of a failure?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears26
Bulkhead
Enforces resource partitioning and damage containment in order to
preserve partial functionality in the case of a failure.
● Multi-threaded EIPs: Delayer, Multicast, Recipient List, Splitter, Threads,
Throttler, Wire Tap, Polling Consumer, ProducerTemplate, and OnCompletion.
● Async Error Handler
● Circuit Breaker EIP
Possible Camel bulkhead points:

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears27
Scalability Scenarios
Vertical scaling (performance tuning)
● Endpoints: messaging client buffers, DB client batching, template caching choices
● Concurrent consumers option: Seda, VM, JMS, RabbitMQ, Disruptor, AWS-SQS
● Data types choice: affects content based router, splitter, filter, aggregator
● Multithreading: parallel processing EIPs, threads DSL construct, Seda component,
asynchronous redelivery/retry
● Micro optimizations: log tuning, camel sampler EIP, disable JMX, disable message
history, disable original message record
● Startup/Shutdown: Use lazyLoadTypeConverters for a faster application startup, or
configure the shutdownStrategy for a faster shutdown
● Tune: JVM options, networking and operating system
Camel performance tuning blog post:
http://bit.ly/camel-tuning
Camel performance tuning blog post:
http://bit.ly/camel-tuning

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears28
Horizontal Scaling
Service Instance Pattern for accommodating increasing workloads.

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears29
Service Instance
Areas to consider before horizontally scaling a Camel application.
● Service state: load balancer, circuit breaker, resequencer, sampler,
throttler, idempotent consumer and aggregator are stateful EIPs!
● Request dispatcher: Messaging, HTTP, file (what about locking?)
● Message ordering: exclusive consumer, message groups, consumer
priority, message priority, virtual topics
● Singleton service requirements: for batch jobs, and concurrent polling
● Other resource contention and coupling considerations

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears30
What did we cover so far?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears31
How patterns are changing?
What is happening in the IT industry today?
● Canonical Data Model
● Edge Component
● Reusable Route
● Runtime Reconfiguration
● Singleton Service
● Batch jobs in JVM
● Bounded Context
● Standalone services
● Favor code duplication
● Less configuration, more redeployment
● Container managed singleton
● Container scheduling
● Circuit Breaker, Bulkhead, Health checks..

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears32
Win a print copy of Camel Design Patterns
When was the first commit to Apache Camel project done?

Apache Camel Design Patterns Learned Through Blood, Sweat, and Tears33
Win a print copy of Camel Design Patterns
When was the first commit to Apache Camel project done?

More Information
Learn more about Apache Camel: http://camel.apache.org
Check out Camel Design Patterns: https://leanpub.com/camel-design-patterns
Develop Apache Camel based integrations using Red Hat JBoss Fuse: red.ht/FuseDev