This document provides notes on Netty Part 2 focusing on transports and buffers. It discusses the different Netty transport options including NIO, epoll, and OIO. It explains that Netty provides a common interface for different implementations. The document also covers Netty buffers including ByteBuf, direct vs array-backed buffers, composite buffers, and buffer pooling. It emphasizes that performance gains come from reducing byte copies and buffer allocation.

1. NOTES ON
NETTY PART 2
RICK HIGHTOWER’S
TRANSPORTS AND BUFFERS

2. ABOUT RICK HIGHTOWER
ABOUT RICK
• Implemented Microservices, Vert.x/Netty at massive scale
• Author of QBit, microservices lib and Boon, Json parser and utility lib
• Founder of Mammatus Technology
• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare

3. Great book on Netty!
https://www.manning.com/books/netty-in-
action

4. Great talk about Netty Best Practices given at Facebook,
then Twitter University
https://goo.gl/LbXheq

5. http://www.infoq.com/presentations/apple-netty
Great talk about Netty at scale

6. PREVIOUS SLIDE DECK
PREVIOUS SLIDE DECK
• Notes on Netty Basics Part 1
• Slideshare
• http://www.slideshare.net/ric
hardhightower/notes-on-
netty-baics
• Notes on Netty Basics Part 1
• Google Slides
• https://goo.gl/aUGm2N

7. CHANNELS AND
TRANSPORTS

8. NETTY TRANSPORTS
TRANSPORTS
• OIO - old school Java networking IO lib pre NIO
• (IMO Deprecated)
• NIO - Uses Java Async API
• Epoll - Uses Linux Network libs via Java Native Interface (JNI)
• Local Transport - Async communication same JVM
• Embedded Transport - for unit testing your channel handlers
• UDS - Uses Unix Domain Sockets for same Server Interprocess
communication (IPC)

9. IN THE END IT IS ALL STREAMS OF OCTETS
STREAMS OF BYTES
• Old school blocking API for Java is easy to use but nothing like NIO
• Hard to switch from one to the other
• Netty provides common interface of
• blocking API (Why?),
• Java NIO (works on OS X, Windows, BSD),
• faster Linux epoll NIO,
• UDS, internal JVM, embedded for unit tests
• Same interface easy to switch between implementations

10. FASTER WHEN YOU DON’T HAVE TO COPY BYTES
ZERO COPY
• Only available with NIO and epoll
• Moves data from file system to network without copying buffers from
user space (your program) and kernel space (Linux, OS X, etc.)
• Works great for HTTP, FTP, anytime you can move files

11. HIGHLY SCALABLE IO EVENT NOTIFICATION
EPOLL
• Part of Linux Kernel since 2002 (other OS have similar event systems)
• Faster than select/poll system calls
• Java NIO uses epoll and equiv.
• Netty now has highly tuned Linux epoll implementation
• uses corking,
• send file,
• edge limit,
• much less synchronization than Java NIO

12. WHEN SHOULD YOU USE OIO?
OIO
• Probably never
• If doing blocking IO operations (like JDBC) maybe, but…
• You should just use your own thread pools (JDK concurrent) instead or
something like Vert.x or QBit
• Vert.x allows pools of workers called worker verticles
• QBit allows pools of service queues
• Both use Netty

13. TRANSPORTS
TRANSPORTS
• NIO
• TCP, UDP, SCTP, UDT (Interprocess communication)
• Epoll Linux only
• TCP, UDP
• OIO (IMO deprecated)
• TCP, UDP, SCTP, UDT
• SCTP is stream message oriented
• When to use SCTP over TCP? Who uses SCTP? 4G LTE, telephony

14. NETTY BUFFERS

15. BIG PERFORMANCE GAINS TO BE HAD
BYTEBUF
• It is all about the bytes
• Performance is reducing byte copies
• Performance is not zero-ing out buffers when not needed
• Performance is reducing the amount of direct buffer creation by
pooling buffers

16. BYTEBUF, BYTEBUFHOLDER
BYTE BUF API
• Easier to use API than JDK ByteBuffer
• no flip (index for read, index for write)
• Capacity is expanded on demand like ArrayList, StringBuilder
• Fluent API (method-chaining)
• Performance improvements
• Zero copy by using built-in composite buffers
• Pooling is supported

17. BYTEBUF, BYTEBUFHOLDER
BYTE BUF API
• read…(), skip…() operation increase read index
• write…() operation increase write index
• get…() operations to not update the indexes
• markReaderIndex(), markWriterIndex(), resetReaderIndex(), resetWriterIndex(),
writerIndex(index), readerIndex(index) to move indexes around, reset to 0, etc.
• forEachByte(predicate) to find a location in the buffer.
• If you try to read beyond what is written you get an IndexOutOfBoundException
• You can specify max capacity, defaults to MAX_VALUE on Integer

18. THERE ARE TWO MAIN TYPES OF BYTE BUFFERS
TWO TYPES OF BYTE BUFS
• Array backed - backed by a byte[] (byte array)
• Direct buffer
• memory gets allocated off JVM heap
• GOOD: Allows native IO calls that do not have to copy buffers to IO lib
• BAD: More expensive to allocate and release
• You need to copy data into heap to use it, can’t just ask for the array
(hasArray() == false, byteBuf.array() throws exception)

19. AGGREGATED VIEW OF MULTIPLE BYTEBUFS
COMPOSITE BUFFERS
• ByteBuf, CompositeByteBuf
• includes N number of on-heap (array backed) and direct byte buffers
• hasArray() only true if only backed by one on heap instance
• Use addComponent(byteBuf) to combine buffers and then no need to copy buffer
contents
• Two areas responsible for different parts of the message but the message sent back
to client
• e.g., one area creates header response, one area creates response body.
• Instead of copying buffers use CompositeByteBuf (add headers and then add
response body buf then send composite)

20. VIEW OR SLICE OF AN EXISTING BUFFER
DERIVED BUFFERS
• duplicate(), slice(), slice(start, stop), unmodifiableBuffer(…),
order(ByteOrder), readSlice(index)
• Return new buffer view with their own indexes and marker indexes
• Changes also change underlying buffer (use copy(), copy(start, stop) if
this is not the behavior that you want)

21. HOLDS A BYTE BUFFER, USED FOR BUFFER REF COUNTING
BYTE BUF HOLDER
• holds a ByteBuf
• content() returns held ByteBuf
• copy() gives a copy of the held ByteBuf
• duplicate() gives a view with unique read/write/mark indexes to the
same ByteBuf buffer data as held ByteBuf

22. ALLOCATING BUFFERS FROM POOLS
BYTE BUF ALLOCATORS
• ByteBufAllocator (you get it from the channel.alloc())
• Used to allocate a buffer, buffer(…), heapBuffer(…), directBuffer(…), compositeBuffer(…),
ioBuffer(…)
• May create a new buffer or use a pooled buffer
• Netty ByteBuf Allocators use Java version of jemalloc
• see talk cited in into area of this slide deck to learn more about jemalloc
• efficient pools of buffers by allowing threads to have their own cache of buffers and then fallback
to larger pool if buffer not available in thread local cache
• reduces overhead, fast, Fast, FAST
• First rule of using pools, if you allocated it, you must free it
• buffer.release()

23. THANKS AND BUY
NORMAN’S BOOK

24. NETTY IN ACTION
IDEAS FOR SLIDES
• Many ideas for slides are directly derived from Netty
in Action book by Norman et al and/or his talks on
Netty
• BUY THE BOOK Netty in Action!
• The slides are a study aid for myself so I can better
learn Netty
• I’ve worked with ByteBuffer, NIO, Vert.x, and have
often wanted to use parts of Netty on projects but
lacked the knowledge of Netty internals so used NIO
or ByteBuffer or OIO when I really wanted to use
Netty instead

25. PREVIOUS SLIDE DECK
PREVIOUS SLIDE DECK
• Notes on Netty Basics Part 1
• Slideshare
• http://www.slideshare.net/ric
hardhightower/notes-on-
netty-baics
• Notes on Netty Basics Part 1
• Google Slides
• https://goo.gl/aUGm2N

26. ABOUT RICK HIGHTOWER
ABOUT RICK
• Implemented Microservices, Vert.x/Netty at massive scale
• Author of QBit, microservices lib and Boon, Json parser and utility lib
• Founder of Mammatus Technology
• Rick’s Twitter, Rick’s LinkedIn, Rick’s Blog, Rick’s Slideshare
The end… sleepless dev.