Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Zero-Copy Event-Driven Servers with Netty

25.611 Aufrufe

Veröffentlicht am

Veröffentlicht in: Technologie, Business
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/y8nn3gmc } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier
  • good
       Antworten 
    Sind Sie sicher, dass Sie …  Ja  Nein
    Ihre Nachricht erscheint hier

Zero-Copy Event-Driven Servers with Netty

  1. 1. Zero-Copy Event-Driven Servers with Netty Daniel BimschasInstitute of Telematics, University of Lübeck http://www.itm.uni-luebeck.de 1
  2. 2. Content•  Basics –  Zero-Copy Techniques –  Event-Driven Architectures•  Introduction to Netty –  Buffers –  Codecs –  Pipelines & Handlers•  Netty Examples –  Discard –  HTTP Server 2
  3. 3. Zero-Copy Techniques 3
  4. 4. Traditional Approach •  copying and context switches between kernel and user space à poor performance! Socket Read Buffer NIC Buffer Fast Buffer Slow Kernel Context Application Context Application App BufferSlide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 4
  5. 5. Zero-Copy Approach •  Kernel handles the copy process via Direct Memory Access (DMA) –  No CPU load –  Lower load on bus system –  No copying between kernelspace and userspace Socket Read Buffer NIC Buffer Fast Buffer Kernel Context Perfect! Task Application Context Application App BufferSlide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 5
  6. 6. Simple Benchmark: Copy vs. Zero-Copy Duration [ms] Data [Mbyte]Slide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 6
  7. 7. Zero-Copy Between Communication Layers •  Often copying is not necessary –  If data is not modi ed a slice can be passed forward without copying to a different buffer Ethernet IP TCP HTTP XML Application Ethernet IP TCP HTTP XML Transport Ethernet IP TCP HTTP XML Internet Ethernet IP TCP HTTP XML Link Layer Ethernet IP TCP HTTP XMLSlide Source: Distributed Systems Course 2011/2012 by Dennis Pfisterer, Institute of Telematics, University of Lübeck, Germany 7
  8. 8. Zero-Copy Between Communication Layers •  Sometimes slices of multiple packages can be combined to extract e.g., a payload that is split over multiple packages •  Newly “created” buffer points to original buffers à No copying necessaryVirtual HTTP (Part 1) HTTP (Part 2)BufferReceived TCP HTTP (Part 1) TCP HTTP (Part 2)Buffers 8
  9. 9. Event-Driven Architecture in Networking 9
  10. 10. Request Processing in Multi-Thread Serverst1: Thread S1: ServerSocket Waits most of time db1: DataBase socket = accept() s2: Socket without doing <<create>>(socket) t2: Thread actual work! run() socket = accept() waitForData() bytes = read() <<create>> d1: Decoder decode(bytes) waitForData() bytes = read() req = decode(bytes) <<create>> s1: Servlet processRequest(req) query(...) response results write(response) = thread idle 10
  11. 11. Request Processing in Multi-Thread Servers•  Usually one thread per request –  Thread idle most of the time (e.g. waiting for I/O) –  Thread even more idle when network slow –  Number of simultaneous clients mostly limited by maximum number of threads•  Thread construction is expensive –  Higher latency when constructing on request –  Can be improved using pools of Threads (see Java‘s ExecutorService & Executors classes) 11
  12. 12. Request Processing in Event-Driven Servers s1: Socket s2: Socket io1: NioWorker e1: ExecutorThread = request 1 dataAvailable() = request 2 bytes = read() handleEvent(s1, bytes) <<create>> dataAvailable() d1: Decoder decode(bytes) bytes = read() handleEvent(s2, bytes) <<create>> dataAvailable() d2: Decoder decode(bytes) bytes = read() handleEvent(s1, bytes) req = decode(bytes) resp = processRequest(bytes) write(resp) dataAvailable() bytes = read() handleEvent(s2, bytes) req = decode(bytes) resp = processRequest(bytes) write(resp)Disclaimer: this slide may contain errors and is far away from real implementation code but should do good for illustrative purposes 12
  13. 13. Request Processing in Event-Driven Servers•  Calls to I/O functions of OS are non-blocking•  Heavy usage of zero-copy strategies•  Everything is an event –  Data available for reading –  Writing data –  Connection established / shut down•  Different requests share threads•  Work is split into small tasks –  Tasks are solved by worker threads from a pool –  Thread number and control decoupled from individual connections / simultaneous requests•  Number of simultaneous clients can be very high –  Netty: 50.000 on commodity hardware! 13
  14. 14. Introduction to Netty 14
  15. 15. Introduction to Netty•  „The Netty project is an effort to provide an asynchronous event-driven network application framework for rapid development of maintainable high-performance protocol servers & clients.“ Source: http://netty.io•  Good reasons to use Netty: •  Simpli es development •  Amazing performance •  Amazing documentation (Tutorials, JavaDocs), clean concepts •  Lots of documenting examples •  Unit testability for protocols •  Heavily used at e.g., twitter for performance critical applications 15
  16. 16. Netty – Feature Overview 16
  17. 17. Introduction to Netty - Buffers•  Netty uses a zero-copy strategy for efficiency•  Primitive byte[] are wrapped in a ChannelBuffer•  Simple read/write operations, e.g.: –  writeByte() –  writeLong() –  readByte() –  readLong() –  …•  Hides complexities such as byte order•  Uses low overhead index pointers for realization: 17
  18. 18. Introduction to Netty - Buffers •  Combine & slice ChannelBuffers without copying any payload data by e.g., –  ChannelBuffer.slice(int index, int length) –  ChannelBuffers.wrappedBuffer(ChannelBuffer... Buffers) •  Pseudo-Code Example: requestPart1 = buffer1.slice(OFFSET_PAYLOAD, buffer1.readableBytes() – OFFSET_PAYLOAD); requestPart2 = buffer2.slice(OFFSET_PAYLOAD, buffer2.readableBytes() – OFFSET_PAYLOAD); request = ChannelBuffers.wrappedBuffer(requestPart1, requestPart2);Virtual HTTP (Part 1) HTTP (Part 2)BufferReceived TCP HTTP (Part 1) TCP HTTP (Part 2)Buffers 18
  19. 19. Netty – Feature Overview 19
  20. 20. Introduction to Netty - Codes•  Many protocol encoders/decoders included –  Base64 –  Zlib –  Framing/Deframing –  HTTP –  WebSockets –  Google Protocol Buffers –  Real-Time Streaming Protocol (RTSP) –  Java Object Serialization –  String –  (SSL/TLS) 20
  21. 21. Introduction to Netty - Codecs•  Abstract base classes for easy implementation –  OneToOneEncoder –  OneToOneDecoder •  Converts one Object (e.g. a ChannelBuffer) into another (e.g. a HttpServletRequest) –  ReplayingDecoder •  The bytes necessary to decode an Object (e.g. a HttpServletRequest) may be split over multiple data events •  Manual implementation forces to check and accumulate data all the time •  ReplayingDecoder allows you to implement decoding methods just like all required bytes were already received 21
  22. 22. Netty – Putting it all together 22
  23. 23. Introduction to Netty – Pipelines & Handlers •  Every socket is attached to a ChannelPipeline •  It contains a stack of handlers –  Protocol Encoders / Decoders –  Security Layers (SSL/TLS, Authentication) –  Application Logic –  ... 23
  24. 24. Introduction to Netty – Pipelines & Handlers •  One ChannelPipeline per Connection •  Handlers can handle –  Downstream events –  Upstream events –  Or both •  Everything is an event –  Data available for reading –  Writing data –  Connection established / shut down –  … 24
  25. 25. Netty – ChannelPipeline Example: HTTP(S) Client Client Application •  Applications based read(httpResponse) write(httpRequest) on Netty are built as Channel a stack httpResponse httpRequest •  Application Logic ChannelPipeline HttpRequestDecoder HttpRequestEncoder String String sites on top of the StringDecoder StringEncoder channel ChannelBuffer ChannelBuffer •  Everything else SSLDecoder SSLEncoder (decoding, ChannelBuffer ChannelBuffer securing, ...) is done OS Socket object inside the pipelineDisclaimer: this slide is imprecise, may contain errors and there’s no one-to-one implementation. It shows a logic conceptual view of the Netty pipeline. 25
  26. 26. Netty ExamplesDISCARD Server 26
  27. 27. Netty Examples – DISCARD Serverhttp://tools.ietf.org/html/rfc863Source: Netty project source code @ http://github.com/netty/netty 27
  28. 28. Netty Examples – DISCARD Server BootstrapSource: Netty project source code @ http://github.com/netty/netty 28
  29. 29. Netty Examples – DISCARD ServerSource: Netty project source code @ http://github.com/netty/netty 29
  30. 30. Netty Examples – Echo ServerSource: Netty project source code @ http://github.com/netty/netty 30
  31. 31. Netty Examples – Static HTTP File ServerSource: Netty project source code @ http://github.com/netty/netty 31
  32. 32. Netty Examples – Static HTTP File ServerSource: Netty project source code @ http://github.com/netty/netty 32
  33. 33. Netty Examples – Static HTTP File Server ...Source: Netty project source code @ http://github.com/netty/netty 33
  34. 34. References•  Netty –  Project: http://netty.io –  Tutorial: http://netty.io/docs/ –  JavaDoc: http://netty.io/docs/3.2.6.Final/api/ –  Sources: http://netty.io/docs/3.2.6.Final/xref/ –  Development: https://github.com/netty/netty 34

×