1. High Volume Web API Management
with WSO2 ESB
Paul Fremantle
CTO and Co-Founder
Hiranya Jayathilaka
Associate Technical Lead

2. WSO2 Offerings
• WSO2 Carbon
• On premise product platform
• Consistent products sharing the same OSGi core
• WSO2 Stratos
• A “Cloud-Enabled Application Platform” (CEAP)
• Multi-tenant, Elastic, Metered/Billed deployment
• An extension of the Carbon platform
• Same SVN code tree
• WSO2 StratosLive
• http://stratoslive.wso2.com
• A running Platform-as-a-Service (PaaS)
• An instance of Stratos
© WSO2 2011

3. ForresterWave™ ESB Q2 2011 - Leader

4. WSO2 ESB 4.0
© WSO2 2011

5. WSO2 ESB Features
• The WSO2 Enterprise Service Bus offers:
• Easy configuration through an intuitive graphical interface
• Lean, high-performance design conserves resources through
maximum efficiency.
• Built on the WSO2 Carbon OSGi platform: adding new capabilities
as you need them is a snap.
• Integrates with your existing systems, with support for
EDI, CSV, Files, FIX, HL7, SAP, JMS, etc
• Asynchronous architecture supports very high loads with
fixed resources
• Routing, mediation and transformation are
modular, extensible but provide most use-cases simply by
sequencing built-in mediators
• Full support for SOAP/WS-* and REST
© WSO2 2011

6. API Management Concerns
• Slow clients, IP denial of service, traffic shaping
• Security, authentication, authorization
• Monitoring, analytics and metrics
• Mediation, format translation, micro-orchestration
• Load-balancing, failover, routing
• Unification of multiple stacks
© WSO2 2011

7. API Management Concerns
• Slow clients, IP denial of service, traffic shaping
• Non-blocking transport, priority
execution, throttle/cache, Message Relay Transport
• Security, authentication, authorization
• Authn/Authz, SAML2, OAuth, XACML PEP
• Monitoring, analytics and metrics
• Built-in metrics, JMX, WSO2 BAM Event Publisher
• Mediation, format translation, micro-orchestration
• JSON support, SOAP<->JSON, Service chaining
• Load-balancing, failover, routing
• Built in elastic load balancing, session-aware, failover
• Unification of multiple stacks
• Lightweight proxy layer, CSG support
© WSO2 2011

8. eBay Case Study – High Volume
• Requirements:
• “The solutions we were using no longer met our needs, so we considered whether
to build a new internal system or to adopt third-party technology”, Abhinav
Kumar, eBay
• Needed enhanced service mediation and orchestration capabilities
• Scalability and performance to sustain increasing loads
• Evaluation:
• eBay evaluated several industry leading hardware and software ESB
solutions, looking at open source, commercially licensed ESBs and hardware
• Test production during 2009 holiday season proved that our system was rock solid
• 1m messages/day with zero downtime, zero memory growth
• Results
• Shared-nothing architecture on RedHat Linux on x86 hardware
• Currently running at more than 1 billion messages / day
• Low resource utilization and flat memory usage
• The benefits
• “Using the WSO2 ESBs, we've been able to provide customers and partners with
the quality experience they expect on eBay, even as our global customer base
has grown,” Abhinav Kumar
© WSO2 2011

9. Memory Usage at eBay
© WSO2 2011

10. Deployment
© WSO2 2011

11. Configuring for High Volume
• Operating System
• JVM
• ESB
• Management
© WSO2 2011

12. Tuning the OS
• Optimize core network settings for high
throughput network activity
• /etc/sysctl.conf
• Increase the number of open file descriptors
allowed by the OS
• /etc/security/limits.conf

13. Tuning the JVM
• Allocate sufficient memory for the heap
• -Xms256m -Xmx2048m -XX:MaxPermSize=256m
• Consider reducing the new ratio
• -XX:NewRatio=n
• Consider using the concurrent mark and sweep
collector
• -XX:+UseConcMarkSweepGC
• Read more on JVM and GC tuning
• http://wso2.org/library/articles/2010/11/taming-
java-garbage-collector

14. Tuning the ESB
• Configure transport thread pools
• Configured through nhttp.properties file
• IO dispatcher threads carry out network IO at the
wire level
• Recommended to have one IO dispatcher per CPU
core
• Server workers and client workers mediate the
messages
• More the merrier (But keep memory usage and load
in mind)

15. Message Relay Mode
• WSO2 ESB uses Apache AXIOM and the StAX
API for processing XML
• XML payloads are streamed through the ESB
• Pull parsing model
• But for pure routing, load-balancing and header-
only mediation we can avoid even this step
• Message relay mode works around this glitch and
enables 100% pure streaming of messages
• Works regardless of size and format of messages

16. Enabling Message Relay
• Enable the binary relay builder (axis2.xml)
• Instructs the ESB to stream the incoming messages through
without touching their payloads
• Enable the expanding message formatter (axis2.xml)
• Allows the ESB to send messages that were received through the
binary relay builder
<messageBuilder contentType="application/xml"
class="org.wso2.carbon.relay.BinaryRelayBuilder"/>
<messageFormatter contentType="application/xml"
class="org.wso2.carbon.relay.ExpandingMessageFormatter"/>
• Above configuration tells the ESB to process all application/xml
messages in message relay mode.

17. HTTP Relay Transport
• Brand new in WSO2 ESB 4.0 release
• Non blocking HTTP transport implementation
specially designed for streaming messages
• Doesn’t require the binary relay builder and
expanding formatter
• To enable, simply uncomment the relevant entries
in the axis2.xml file

18. Performance
© WSO2 2011

19. Error Handling
• In a large scale, high throughput
deployment, errors are not unusual – Expect the
unexpected
• Configure endpoints to gracefully handle the
common errors
• Connection timeouts
• Connection close/reset
• Connection refused
• HTTP protocol violations!!!!

20. More on Error Handling
• Configure sequences to clearly log errors and if
needed notify system administrators
• In case the back end server fails to respond, send
detailed fault responses to clients – Makes your
application more appealing to the customers
• Pay attention to HTTP error codes

21. Monitoring
• In general keep an eye on:
• CPU usage
• Memory usage
• Thread counts
• Fault counts
• Latency/Response time
• Active connections

22. Monitoring Tools
• Utilities Provided by OS
• top
• netstat
• Mediation statistics
• JMX Clients
• Jconsole

23. Connections
© WSO2 2011

24. Endpoints
© WSO2 2011

25. Latency
© WSO2 2011

26. Listeners
© WSO2 2011

27. Memory and Garbage Collection
© WSO2 2011

28. S2S Latency
© WSO2 2011

29. Threads
© WSO2 2011

30. Questions?
http://www.flickr.com/photos/oberazzi/
30

31. Resources
• http://wso2.com/landing/ebay-the-esb-casestudy/
• WSO2 ESB 4.0 Download and Library
• http://wso2.org/library/esb
• Message Relay Mode
• http://wso2.org/project/esb/java/4.0.0/docs/messag
e_relay.html
• Twitter: @pzfreo
• Blogs:
• http://pzf.fremantle.org
• http://techfeast-hiranya.blogspot.com/
• Email: paul@wso2.com hiranya@wso2.com
© WSO2 2011