Zero-Copy Event-Driven   Servers with Netty               Daniel BimschasInstitute of Telematics, University of Lübeck    ...
Content•  Basics   –  Zero-Copy Techniques   –  Event-Driven Architectures•  Introduction to Netty   –  Buffers   –  Codecs...
Zero-Copy Techniques                       3
Traditional Approach      •  copying and context switches between kernel         and user space à poor performance!      ...
Zero-Copy Approach      •  Kernel handles the copy process via Direct Memory Access (DMA)              –  No CPU load     ...
Simple Benchmark: Copy vs. Zero-Copy              Duration [ms]                                                           ...
Zero-Copy Between Communication Layers      •  Often copying is not necessary              –  If data is not modi ed a sli...
Zero-Copy Between Communication Layers  •  Sometimes slices of multiple packages can be     combined to extract e.g., a pa...
Event-Driven Architecture in Networking                                          9
Request Processing in Multi-Thread Serverst1: Thread        S1: ServerSocket                                       Waits m...
Request Processing in Multi-Thread Servers•  Usually one thread per request  –  Thread idle most of the time (e.g. waiting...
Request Processing in Event-Driven Servers    s1: Socket          s2: Socket         io1: NioWorker                  e1: E...
Request Processing in Event-Driven Servers•  Calls to I/O functions of OS are non-blocking•  Heavy usage of zero-copy stra...
Introduction to Netty                        14
Introduction to Netty•  „The Netty project is an effort to provide an asynchronous   event-driven network application fram...
Netty – Feature Overview                           16
Introduction to Netty - Buffers•  Netty uses a zero-copy strategy for efficiency•  Primitive byte[] are wrapped in a ChannelB...
Introduction to Netty - Buffers  •  Combine & slice ChannelBuffers without copying     any payload data by e.g.,          – ...
Netty – Feature Overview                           19
Introduction to Netty - Codes•  Many protocol encoders/decoders included  –  Base64  –  Zlib  –  Framing/Deframing  –  HTT...
Introduction to Netty - Codecs•  Abstract base classes for easy implementation  –  OneToOneEncoder  –  OneToOneDecoder    ...
Netty – Putting it all together                                  22
Introduction to Netty – Pipelines & Handlers                      •  Every socket is attached                         to a...
Introduction to Netty – Pipelines & Handlers                      •  One ChannelPipeline per                         Conne...
Netty – ChannelPipeline Example: HTTP(S) Client                        Client Application                                 ...
Netty ExamplesDISCARD Server                 26
Netty Examples – DISCARD Serverhttp://tools.ietf.org/html/rfc863Source: Netty project source code @ http://github.com/nett...
Netty Examples – DISCARD Server BootstrapSource: Netty project source code @ http://github.com/netty/netty                ...
Netty Examples – DISCARD ServerSource: Netty project source code @ http://github.com/netty/netty                          ...
Netty Examples – Echo ServerSource: Netty project source code @ http://github.com/netty/netty                             ...
Netty Examples – Static HTTP File ServerSource: Netty project source code @ http://github.com/netty/netty                 ...
Netty Examples – Static HTTP File ServerSource: Netty project source code @ http://github.com/netty/netty                 ...
Netty Examples – Static HTTP File Server                                                 ...Source: Netty project source c...
References•  Netty  –  Project: http://netty.io  –  Tutorial: http://netty.io/docs/  –  JavaDoc: http://netty.io/docs/3.2....
Nächste SlideShare
Wird geladen in …5
×

Zero-Copy Event-Driven Servers with Netty

22.941 Aufrufe

Veröffentlicht am

Veröffentlicht in: Technologie, Business
1 Kommentar
109 Gefällt mir
Statistik
Notizen
Keine Downloads
Aufrufe
Aufrufe insgesamt
22.941
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
74
Aktionen
Geteilt
0
Downloads
517
Kommentare
1
Gefällt mir
109
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Zero-Copy Event-Driven Servers with Netty

  1. 1. Zero-Copy Event-Driven Servers with Netty Daniel BimschasInstitute of Telematics, University of Lübeck http://www.itm.uni-luebeck.de 1
  2. 2. Content•  Basics –  Zero-Copy Techniques –  Event-Driven Architectures•  Introduction to Netty –  Buffers –  Codecs –  Pipelines & Handlers•  Netty Examples –  Discard –  HTTP Server 2
  3. 3. Zero-Copy Techniques 3
  4. 4. Traditional Approach •  copying and context switches between kernel and user space à poor performance! Socket Read Buffer NIC Buffer Fast Buffer Slow Kernel Context Application Context Application App BufferSlide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 4
  5. 5. Zero-Copy Approach •  Kernel handles the copy process via Direct Memory Access (DMA) –  No CPU load –  Lower load on bus system –  No copying between kernelspace and userspace Socket Read Buffer NIC Buffer Fast Buffer Kernel Context Perfect! Task Application Context Application App BufferSlide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 5
  6. 6. Simple Benchmark: Copy vs. Zero-Copy Duration [ms] Data [Mbyte]Slide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 6
  7. 7. Zero-Copy Between Communication Layers •  Often copying is not necessary –  If data is not modi ed a slice can be passed forward without copying to a different buffer Ethernet IP TCP HTTP XML Application Ethernet IP TCP HTTP XML Transport Ethernet IP TCP HTTP XML Internet Ethernet IP TCP HTTP XML Link Layer Ethernet IP TCP HTTP XMLSlide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 7
  8. 8. Zero-Copy Between Communication Layers •  Sometimes slices of multiple packages can be combined to extract e.g., a payload that is split over multiple packages •  Newly “created” buffer points to original buffers à No copying necessaryVirtual HTTP (Part 1) HTTP (Part 2)BufferReceived TCP HTTP (Part 1) TCP HTTP (Part 2)Buffers 8
  9. 9. Event-Driven Architecture in Networking 9
  10. 10. Request Processing in Multi-Thread Serverst1: Thread S1: ServerSocket Waits most of time db1: DataBase socket = accept() s2: Socket without doing <<create>>(socket) t2: Thread actual work! run() socket = accept() waitForData() bytes = read() <<create>> d1: Decoder decode(bytes) waitForData() bytes = read() req = decode(bytes) <<create>> s1: Servlet processRequest(req) query(...) response results write(response) = thread idle 10
  11. 11. Request Processing in Multi-Thread Servers•  Usually one thread per request –  Thread idle most of the time (e.g. waiting for I/O) –  Thread even more idle when network slow –  Number of simultaneous clients mostly limited by maximum number of threads•  Thread construction is expensive –  Higher latency when constructing on request –  Can be improved using pools of Threads (see Java‘s ExecutorService & Executors classes) 11
  12. 12. Request Processing in Event-Driven Servers s1: Socket s2: Socket io1: NioWorker e1: ExecutorThread = request 1 dataAvailable() = request 2 bytes = read() handleEvent(s1, bytes) <<create>> dataAvailable() d1: Decoder decode(bytes) bytes = read() handleEvent(s2, bytes) <<create>> dataAvailable() d2: Decoder decode(bytes) bytes = read() handleEvent(s1, bytes) req = decode(bytes) resp = processRequest(bytes) write(resp) dataAvailable() bytes = read() handleEvent(s2, bytes) req = decode(bytes) resp = processRequest(bytes) write(resp)Disclaimer: this slide may contain errors and is far away from real implementation code but should do good for illustrative purposes 12
  13. 13. Request Processing in Event-Driven Servers•  Calls to I/O functions of OS are non-blocking•  Heavy usage of zero-copy strategies•  Everything is an event –  Data available for reading –  Writing data –  Connection established / shut down•  Different requests share threads•  Work is split into small tasks –  Tasks are solved by worker threads from a pool –  Thread number and control decoupled from individual connections / simultaneous requests•  Number of simultaneous clients can be very high –  Netty: 50.000 on commodity hardware! 13
  14. 14. Introduction to Netty 14
  15. 15. Introduction to Netty•  „The Netty project is an effort to provide an asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers & clients.“ Source: http://netty.io•  Good reasons to use Netty: •  Simpli es development •  Amazing performance •  Amazing documentation (Tutorials, JavaDocs), clean concepts •  Lots of documenting examples •  Unit testability for protocols •  Heavily used at e.g., twitter for performance critical applications 15
  16. 16. Netty – Feature Overview 16
  17. 17. Introduction to Netty - Buffers•  Netty uses a zero-copy strategy for efficiency•  Primitive byte[] are wrapped in a ChannelBuffer•  Simple read/write operations, e.g.: –  writeByte() –  writeLong() –  readByte() –  readLong() –  …•  Hides complexities such as byte order•  Uses low overhead index pointers for realization: 17
  18. 18. Introduction to Netty - Buffers •  Combine & slice ChannelBuffers without copying any payload data by e.g., –  ChannelBuffer.slice(int index, int length) –  ChannelBuffers.wrappedBuffer(ChannelBuffer... Buffers) •  Pseudo-Code Example: requestPart1 = buffer1.slice(OFFSET_PAYLOAD, buffer1.readableBytes() – OFFSET_PAYLOAD); requestPart2 = buffer2.slice(OFFSET_PAYLOAD, buffer2.readableBytes() – OFFSET_PAYLOAD); request = ChannelBuffers.wrappedBuffer(requestPart1, requestPart2);Virtual HTTP (Part 1) HTTP (Part 2)BufferReceived TCP HTTP (Part 1) TCP HTTP (Part 2)Buffers 18
  19. 19. Netty – Feature Overview 19
  20. 20. Introduction to Netty - Codes•  Many protocol encoders/decoders included –  Base64 –  Zlib –  Framing/Deframing –  HTTP –  WebSockets –  Google Protocol Buffers –  Real-Time Streaming Protocol (RTSP) –  Java Object Serialization –  String –  (SSL/TLS) 20
  21. 21. Introduction to Netty - Codecs•  Abstract base classes for easy implementation –  OneToOneEncoder –  OneToOneDecoder •  Converts one Object (e.g. a ChannelBuffer) into another (e.g. a HttpServletRequest) –  ReplayingDecoder •  The bytes necessary to decode an Object (e.g. a HttpServletRequest) may be split over multiple data events •  Manual implementation forces to check and accumulate data all the time •  ReplayingDecoder allows you to implement decoding methods just like all required bytes were already received 21
  22. 22. Netty – Putting it all together 22
  23. 23. Introduction to Netty – Pipelines & Handlers •  Every socket is attached to a ChannelPipeline •  It contains a stack of handlers –  Protocol Encoders / Decoders –  Security Layers (SSL/TLS, Authentication) –  Application Logic –  ... 23
  24. 24. Introduction to Netty – Pipelines & Handlers •  One ChannelPipeline per Connection •  Handlers can handle –  Downstream events –  Upstream events –  Or both •  Everything is an event –  Data available for reading –  Writing data –  Connection established / shut down –  … 24
  25. 25. Netty – ChannelPipeline Example: HTTP(S) Client Client Application •  Applications based read(httpResponse) write(httpRequest) on Netty are built as Channel a stack httpResponse httpRequest •  Application Logic ChannelPipeline HttpRequestDecoder HttpRequestEncoder String String sites on top of the StringDecoder StringEncoder channel ChannelBuffer ChannelBuffer •  Everything else SSLDecoder SSLEncoder (decoding, ChannelBuffer ChannelBuffer securing, ...) is done OS Socket object inside the pipelineDisclaimer: this slide is imprecise, may contain errors and there’s no one-to-one implementation. It shows a logic conceptual view of the Netty pipeline. 25
  26. 26. Netty ExamplesDISCARD Server 26
  27. 27. Netty Examples – DISCARD Serverhttp://tools.ietf.org/html/rfc863Source: Netty project source code @ http://github.com/netty/netty 27
  28. 28. Netty Examples – DISCARD Server BootstrapSource: Netty project source code @ http://github.com/netty/netty 28
  29. 29. Netty Examples – DISCARD ServerSource: Netty project source code @ http://github.com/netty/netty 29
  30. 30. Netty Examples – Echo ServerSource: Netty project source code @ http://github.com/netty/netty 30
  31. 31. Netty Examples – Static HTTP File ServerSource: Netty project source code @ http://github.com/netty/netty 31
  32. 32. Netty Examples – Static HTTP File ServerSource: Netty project source code @ http://github.com/netty/netty 32
  33. 33. Netty Examples – Static HTTP File Server ...Source: Netty project source code @ http://github.com/netty/netty 33
  34. 34. References•  Netty –  Project: http://netty.io –  Tutorial: http://netty.io/docs/ –  JavaDoc: http://netty.io/docs/3.2.6.Final/api/ –  Sources: http://netty.io/docs/3.2.6.Final/xref/ –  Development: https://github.com/netty/netty 34

×