SlideShare ist ein Scribd-Unternehmen logo
1 von 15
Multimedia flow classification at 10
Gbps using acceleration techniques on
commodity hardware
Rafael Leira1, Pedro Gómez1, Iván González1,2
Jorge E. López de Vergara1,2
<jorge@naudit.es>
1Universidad Autónoma de Madrid, Spain
2Naudit High Performance Computing and Networking, Spain
First International Workshop on Quality Monitoring,
SaCoNeT, 17th June 2013, Paris, France
Contents
 Introduction
 Related Work
 Architecture
 Performance and validation tests
 Conclusions and future work
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 2
Introduction
 The Internet size is growing and changing day by
day, with more and more servers, protocols and end
users.
 Many businesses require classify such traffic.
For instance:
» To identify network intrusions
» To provide QoS (Quality of Service).
» To filter out protocols within a subnet
 For this purpose, a probe is required that can
support a sustained high throughput.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 3
Related Work
 Nowadays there are several methods for traffic
classification, commercial probes and different
mechanisms that implement traffic classification.
 The most commonly methods are:
» Classification by port.
» DPI (Deep Packet Inspection) classification.
» Statistical classification.
 Commercial probes can afford more than 20
Gbps, giving many optional features. However, those
probe are extremely expensive.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 4
Architecture
 The implemented probe has the following modules:
» Network module : Intel DPDK is used to capture the traffic
at 10 Gbps. It also provides a flexible and scalable
architecture, useful for parallel programming.
» FlowBuilder module : The module is designed to group
packets in different flows. It also provides statistics about
every flow inside the network, in a similar way as Cisco
NetFlow.
» GPU classification module : This module performs the
flow classification within a GPU. It maximizes
performance, pipelining the analysis of different flows.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 5
Architecture – Network and
FlowBuilder modules
Nic
0
Nic
1
I/O RX
0
Worker
0
I/O TX
0
Nic
1
Nic
0
FlowBuilder (Worker 0)
process_packet export_flow
A flow
expires
If there is no expired flow,
the function ends.
GPU
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 6
Architecture – Flow Builder
export_flow
The flow payload
is copied to
the buffer
return
There is
empty space
GPU Classification
The flow buffer
is copied into
GPU memory
The buffer
is full of flows
The results
are returned
Get the
current
active
buffer
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 7
Architecture – GPU classification
1) A flow block enters inside the
module.
2) The flow block is copied inside the
GPU memory.
3) The GPU transposes the block.
4) The flows are classified in parallel.
5) The results are transposed again.
6) The results are also reduced in order to
minimize data transfers.
7) The flow block is returned to the Host
memory.
8) Finally, the results can be stored or relayed.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 8
Performance Tests
 Network module
» Using a hardware traffic generator we have achieved to capture
14.9 Million packets per second without packet loss.
 Network and FlowBuilder integration
» Performance tests show a
high dependency on the
number of concurrent
flows inside the link.
» However, the flow builder
can support a typical
number of concurrent flows
in a 10 Gbps link.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 9
Signatures Host
↔
GPU
Traspose Matching Traspose Reduction GPU
↔
Host
Global
Gbps /
Mflowps
8 35.92 90.84 51.4 98.27 64.83 260.42 24 / 11.9
16 32.42 84.46 47.35 53.69 34.49 169.84 18 / 8.7
24 33.17 72.67 43.28 36.51 23.39 124.01 13 / 6.1
32 31.38 64.57 39.56 27.80 17.72 100.81 10 / 4.8
40 29.76 58.74 27.08 22.45 14.33 85.38 8 / 3.9
Performance Tests
 GPU classification
» Reached GPU performance in Gigabits per second.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 10
Performance Tests
 GPU classification
11
Validation Tests
 Flows matching ratings
» TCP syn 145 / 576
(25,17%) (without
content)
» ICMP 116 / 576 (20,46%)
» NTP 5 / 576 (0.87%)
» RTCP 118 / 576 (20.49%)
» RTP 143 / 576 (24.83%)
» Unknown 49 / 576
(8.51%)
 Classification accuracy highly
depends on each protocol
signature.
 Binary protocols represent a major
problem to define a signature for
them.
 According to the protocol, it can be
impossible to define a signature
without a loss of accuracy.
 The used trace was captured in
Alcatel-Lucent premises, in the
scope of the IPNQSIS project.
 The unknown flows are minor
protocols that are not important in
multimedia classification. In
fact, they are network-booting
protocols.
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 12
Conclusions
 Network Module: it allows a flexible and parallel architecture, and
it takes advantage of the hardware capabilities in a simple way.
 FlowBuilder Module: it is very dependent on the number of flows
inside a link. It can support up to 10 Gbps without packet losses.
 GPU classification Module: it reduces CPU consumption. It highly
depends on the quality of the signatures: problem with binary
protocols such as RTP.
 In summary, the system can process and classify a 10 Gbps data
rate in normal traffic conditions.
 It has been difficult to validate the system, feeding it at this speed.
 Future work
» GPU performance has to be improved in order to minimize the impact of
multiple signatures.
» The FlowBuilder module has to be improved too in order to support
higher performance ( > 20 Gbps )
Multimedia flow classification at 10 Gbps using
acceleration techniques on commodity hardware 13
Multimedia flow classification at 10
Gbps using acceleration techniques on
commodity hardware
Rafael Leira1, Pedro Gómez1, Iván González1,2
Jorge E. López de Vergara1,2
<jorge@naudit.es>
1Universidad Autónoma de Madrid, Spain
2Naudit High Performance Computing and Networking, Spain
First International Workshop on Quality Monitoring,
SaCoNeT, 17th June 2013, Paris, France
Architecture - summary
15

Weitere ähnliche Inhalte

Was ist angesagt?

Quality impact of scalable video coding tunneling for media aware content del...
Quality impact of scalable video coding tunneling for media aware content del...Quality impact of scalable video coding tunneling for media aware content del...
Quality impact of scalable video coding tunneling for media aware content del...
Alpen-Adria-Universität
 
LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...
LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...
LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...
LF_DPDK
 
LF_DPDK17_Integrating and using DPDK with Open vSwitch
LF_DPDK17_Integrating and using DPDK with Open vSwitchLF_DPDK17_Integrating and using DPDK with Open vSwitch
LF_DPDK17_Integrating and using DPDK with Open vSwitch
LF_DPDK
 
LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...
LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...
LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...
LF_DPDK
 
LF_DPDK17_Accelerating P4-based Dataplane with DPDK
LF_DPDK17_Accelerating P4-based Dataplane with DPDKLF_DPDK17_Accelerating P4-based Dataplane with DPDK
LF_DPDK17_Accelerating P4-based Dataplane with DPDK
LF_DPDK
 

Was ist angesagt? (20)

gRPC on .NET Core - NDC Sydney 2019
gRPC on .NET Core - NDC Sydney 2019gRPC on .NET Core - NDC Sydney 2019
gRPC on .NET Core - NDC Sydney 2019
 
Quality impact of scalable video coding tunneling for media aware content del...
Quality impact of scalable video coding tunneling for media aware content del...Quality impact of scalable video coding tunneling for media aware content del...
Quality impact of scalable video coding tunneling for media aware content del...
 
gRPC stack supporting Intel Resource Director technology (RDT)
gRPC stack supporting Intel Resource Director technology (RDT)gRPC stack supporting Intel Resource Director technology (RDT)
gRPC stack supporting Intel Resource Director technology (RDT)
 
SDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN componentsSDN interfaces and performance analysis of SDN components
SDN interfaces and performance analysis of SDN components
 
PlatCon-19 ICMPv6SD
PlatCon-19 ICMPv6SDPlatCon-19 ICMPv6SD
PlatCon-19 ICMPv6SD
 
LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...
LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...
LF_DPDK17_GRO/GSO Libraries: Bring Significant Performance Gains to DPDK-base...
 
Error and Flow Control Protocol (EFCP) Design and Implementation: A Data Tran...
Error and Flow Control Protocol (EFCP) Design and Implementation: A Data Tran...Error and Flow Control Protocol (EFCP) Design and Implementation: A Data Tran...
Error and Flow Control Protocol (EFCP) Design and Implementation: A Data Tran...
 
Analysis of video quality and end-to-end latency in WebRTC
Analysis of video quality and end-to-end latency in WebRTCAnalysis of video quality and end-to-end latency in WebRTC
Analysis of video quality and end-to-end latency in WebRTC
 
Investigating the Impact of Network Topology on the Processing Times of SDN C...
Investigating the Impact of Network Topology on the Processing Times of SDN C...Investigating the Impact of Network Topology on the Processing Times of SDN C...
Investigating the Impact of Network Topology on the Processing Times of SDN C...
 
FIWARE Global Summit - Real-time Media Stream Processing Using Kurento
FIWARE Global Summit - Real-time Media Stream Processing Using KurentoFIWARE Global Summit - Real-time Media Stream Processing Using Kurento
FIWARE Global Summit - Real-time Media Stream Processing Using Kurento
 
LF_DPDK17_Integrating and using DPDK with Open vSwitch
LF_DPDK17_Integrating and using DPDK with Open vSwitchLF_DPDK17_Integrating and using DPDK with Open vSwitch
LF_DPDK17_Integrating and using DPDK with Open vSwitch
 
Rina2020 michal
Rina2020 michalRina2020 michal
Rina2020 michal
 
LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...
LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...
LF_DPDK17_OpenNetVM: A high-performance NFV platforms to meet future communic...
 
LF_DPDK17_Accelerating P4-based Dataplane with DPDK
LF_DPDK17_Accelerating P4-based Dataplane with DPDKLF_DPDK17_Accelerating P4-based Dataplane with DPDK
LF_DPDK17_Accelerating P4-based Dataplane with DPDK
 
Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...
Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...
Using IO Visor to Secure Microservices Running on CloudFoundry [OpenStack Sum...
 
JavaZone 2016 : MQTT and CoAP for the Java Developer
JavaZone 2016 : MQTT and CoAP for the Java DeveloperJavaZone 2016 : MQTT and CoAP for the Java Developer
JavaZone 2016 : MQTT and CoAP for the Java Developer
 
Accelerating Networked Applications with Flexible Packet Processing
Accelerating Networked Applications with Flexible Packet ProcessingAccelerating Networked Applications with Flexible Packet Processing
Accelerating Networked Applications with Flexible Packet Processing
 
Networking revolution
Networking revolutionNetworking revolution
Networking revolution
 
OpenContrail, Real Speed: Offloading vRouter
OpenContrail, Real Speed: Offloading vRouterOpenContrail, Real Speed: Offloading vRouter
OpenContrail, Real Speed: Offloading vRouter
 
Reactive micro services using RSocket
Reactive micro services using RSocketReactive micro services using RSocket
Reactive micro services using RSocket
 

Andere mochten auch

Addy Pope: Go-Geo! and GeoDoc
Addy Pope: Go-Geo! and GeoDocAddy Pope: Go-Geo! and GeoDoc
Addy Pope: Go-Geo! and GeoDoc
AGI Geocommunity
 
Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1
Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1
Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1
nuraishah91
 
Dyer_Polly_916_FoundExamCertificate2015NEW
Dyer_Polly_916_FoundExamCertificate2015NEWDyer_Polly_916_FoundExamCertificate2015NEW
Dyer_Polly_916_FoundExamCertificate2015NEW
Polly Boyd
 
Симетрія. Презентація групи істориків
Симетрія. Презентація групи істориківСиметрія. Презентація групи істориків
Симетрія. Презентація групи істориків
LLNegoy
 
Charadas rebecca e pérolam
Charadas   rebecca  e pérolamCharadas   rebecca  e pérolam
Charadas rebecca e pérolam
turma401
 

Andere mochten auch (15)

15 lessons of lord buddha
15 lessons of lord buddha15 lessons of lord buddha
15 lessons of lord buddha
 
Addy Pope: Go-Geo! and GeoDoc
Addy Pope: Go-Geo! and GeoDocAddy Pope: Go-Geo! and GeoDoc
Addy Pope: Go-Geo! and GeoDoc
 
Photos powerpoint
Photos powerpointPhotos powerpoint
Photos powerpoint
 
LAW 531 WEEK 5
LAW 531 WEEK 5LAW 531 WEEK 5
LAW 531 WEEK 5
 
Social Media: What? Why? How?
Social Media: What? Why? How?Social Media: What? Why? How?
Social Media: What? Why? How?
 
Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1
Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1
Pembentangan kemahiran pengurusan maklumat dan pembelajaran sepanjang hayat1
 
Dyer_Polly_916_FoundExamCertificate2015NEW
Dyer_Polly_916_FoundExamCertificate2015NEWDyer_Polly_916_FoundExamCertificate2015NEW
Dyer_Polly_916_FoundExamCertificate2015NEW
 
Place de la filiere touristique dans la region de Marrakech
Place de la filiere touristique dans la region de Marrakech
Place de la filiere touristique dans la region de Marrakech
Place de la filiere touristique dans la region de Marrakech
 
Qualité de la formation référentiel bureau veritas
Qualité de la formation   référentiel bureau veritasQualité de la formation   référentiel bureau veritas
Qualité de la formation référentiel bureau veritas
 
Математика і поезія
Математика і поезіяМатематика і поезія
Математика і поезія
 
Симетрія. Презентація групи істориків
Симетрія. Презентація групи істориківСиметрія. Презентація групи істориків
Симетрія. Презентація групи істориків
 
Симетрія в архітектурі
Симетрія в архітектуріСиметрія в архітектурі
Симетрія в архітектурі
 
Charadas rebecca e pérolam
Charadas   rebecca  e pérolamCharadas   rebecca  e pérolam
Charadas rebecca e pérolam
 
Ampa summer camp y campus tecnologico
Ampa summer camp y campus tecnologicoAmpa summer camp y campus tecnologico
Ampa summer camp y campus tecnologico
 
Option Strategies
Option StrategiesOption Strategies
Option Strategies
 

Ähnlich wie Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware

Profiling And Optimization Of Software Base Network Analysis Applications
Profiling And Optimization Of Software Base Network Analysis ApplicationsProfiling And Optimization Of Software Base Network Analysis Applications
Profiling And Optimization Of Software Base Network Analysis Applications
Hargyo T. Nugroho
 

Ähnlich wie Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware (20)

NUVX Technologies general solutions
NUVX Technologies general solutionsNUVX Technologies general solutions
NUVX Technologies general solutions
 
Chapter04
Chapter04Chapter04
Chapter04
 
Continuum pcap-oem
Continuum pcap-oemContinuum pcap-oem
Continuum pcap-oem
 
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
Dataplane networking acceleration with OpenDataplane / Максим Уваров (Linaro)
 
APIs at the Edge
APIs at the EdgeAPIs at the Edge
APIs at the Edge
 
Monitoring federation open stack infrastructure
Monitoring federation open stack infrastructureMonitoring federation open stack infrastructure
Monitoring federation open stack infrastructure
 
Cubro subprocessor appliance in nic format
Cubro subprocessor appliance in nic formatCubro subprocessor appliance in nic format
Cubro subprocessor appliance in nic format
 
Resume2015
Resume2015Resume2015
Resume2015
 
Converged IO for HP ProLiant Gen8
Converged IO for HP ProLiant Gen8Converged IO for HP ProLiant Gen8
Converged IO for HP ProLiant Gen8
 
Graphical packet generator
Graphical packet generatorGraphical packet generator
Graphical packet generator
 
Automatically partitioning packet processing applications for pipelined archi...
Automatically partitioning packet processing applications for pipelined archi...Automatically partitioning packet processing applications for pipelined archi...
Automatically partitioning packet processing applications for pipelined archi...
 
Interconnect your future
Interconnect your futureInterconnect your future
Interconnect your future
 
Swisscom Network Analytics
Swisscom Network AnalyticsSwisscom Network Analytics
Swisscom Network Analytics
 
The new (is it really ) api stack
The new (is it really ) api stackThe new (is it really ) api stack
The new (is it really ) api stack
 
Accelerated SDN in Azure
Accelerated SDN in AzureAccelerated SDN in Azure
Accelerated SDN in Azure
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentInternational Journal of Engineering Research and Development
International Journal of Engineering Research and Development
 
guna_2015.DOC
guna_2015.DOCguna_2015.DOC
guna_2015.DOC
 
Swisscom Network Analytics Data Mesh Architecture - ETH Viscon - 10-2022.pdf
Swisscom Network Analytics Data Mesh Architecture - ETH Viscon - 10-2022.pdfSwisscom Network Analytics Data Mesh Architecture - ETH Viscon - 10-2022.pdf
Swisscom Network Analytics Data Mesh Architecture - ETH Viscon - 10-2022.pdf
 
5 maximazing networkcapacity_v4-jorge_alvarado
5 maximazing networkcapacity_v4-jorge_alvarado5 maximazing networkcapacity_v4-jorge_alvarado
5 maximazing networkcapacity_v4-jorge_alvarado
 
Profiling And Optimization Of Software Base Network Analysis Applications
Profiling And Optimization Of Software Base Network Analysis ApplicationsProfiling And Optimization Of Software Base Network Analysis Applications
Profiling And Optimization Of Software Base Network Analysis Applications
 

Mehr von Jorge E. López de Vergara Méndez

On the feasibility of 40 Gbps network data capture and retention with general...
On the feasibility of 40 Gbps network data capture and retention with general...On the feasibility of 40 Gbps network data capture and retention with general...
On the feasibility of 40 Gbps network data capture and retention with general...
Jorge E. López de Vergara Méndez
 

Mehr von Jorge E. López de Vergara Méndez (9)

On the feasibility of 40 Gbps network data capture and retention with general...
On the feasibility of 40 Gbps network data capture and retention with general...On the feasibility of 40 Gbps network data capture and retention with general...
On the feasibility of 40 Gbps network data capture and retention with general...
 
Evaluación de equipamiento de bajo coste para realizar medidas de red en ento...
Evaluación de equipamiento de bajo coste para realizar medidas de red en ento...Evaluación de equipamiento de bajo coste para realizar medidas de red en ento...
Evaluación de equipamiento de bajo coste para realizar medidas de red en ento...
 
Dictyogram: a Statistical Approach for the Definition and Visualization of Ne...
Dictyogram: a Statistical Approach for the Definition and Visualization of Ne...Dictyogram: a Statistical Approach for the Definition and Visualization of Ne...
Dictyogram: a Statistical Approach for the Definition and Visualization of Ne...
 
Análisis de Datos Funcionales para Gestión de Red: Téecnicas, Retos y Oportun...
Análisis de Datos Funcionales para Gestión de Red: Téecnicas, Retos y Oportun...Análisis de Datos Funcionales para Gestión de Red: Téecnicas, Retos y Oportun...
Análisis de Datos Funcionales para Gestión de Red: Téecnicas, Retos y Oportun...
 
MONITORIZACIÓN Y ANÁLISIS DE TRÁFICO DE RED CON APACHE HADOOP
MONITORIZACIÓN Y ANÁLISIS DE TRÁFICO DE RED CON APACHE HADOOPMONITORIZACIÓN Y ANÁLISIS DE TRÁFICO DE RED CON APACHE HADOOP
MONITORIZACIÓN Y ANÁLISIS DE TRÁFICO DE RED CON APACHE HADOOP
 
Merging heterogeneous network measurement data
Merging heterogeneous network measurement dataMerging heterogeneous network measurement data
Merging heterogeneous network measurement data
 
Evaluating Quality of Experience in IPTV Services Using MPEG Frame Loss Rate
Evaluating Quality of Experience in IPTV Services Using MPEG Frame Loss RateEvaluating Quality of Experience in IPTV Services Using MPEG Frame Loss Rate
Evaluating Quality of Experience in IPTV Services Using MPEG Frame Loss Rate
 
Defining ontologies for IP traffic measurements at MOI ISG
Defining ontologies for IP traffic measurements at MOI ISGDefining ontologies for IP traffic measurements at MOI ISG
Defining ontologies for IP traffic measurements at MOI ISG
 
Integración semántica de información de distintos repositorios de medidas de red
Integración semántica de información de distintos repositorios de medidas de redIntegración semántica de información de distintos repositorios de medidas de red
Integración semántica de información de distintos repositorios de medidas de red
 

Kürzlich hochgeladen

Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 

Kürzlich hochgeladen (20)

Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf[BuildWithAI] Introduction to Gemini.pdf
[BuildWithAI] Introduction to Gemini.pdf
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
Apidays New York 2024 - APIs in 2030: The Risk of Technological Sleepwalk by ...
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin WoodPolkadot JAM Slides - Token2049 - By Dr. Gavin Wood
Polkadot JAM Slides - Token2049 - By Dr. Gavin Wood
 

Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware

  • 1. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware Rafael Leira1, Pedro Gómez1, Iván González1,2 Jorge E. López de Vergara1,2 <jorge@naudit.es> 1Universidad Autónoma de Madrid, Spain 2Naudit High Performance Computing and Networking, Spain First International Workshop on Quality Monitoring, SaCoNeT, 17th June 2013, Paris, France
  • 2. Contents  Introduction  Related Work  Architecture  Performance and validation tests  Conclusions and future work Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 2
  • 3. Introduction  The Internet size is growing and changing day by day, with more and more servers, protocols and end users.  Many businesses require classify such traffic. For instance: » To identify network intrusions » To provide QoS (Quality of Service). » To filter out protocols within a subnet  For this purpose, a probe is required that can support a sustained high throughput. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 3
  • 4. Related Work  Nowadays there are several methods for traffic classification, commercial probes and different mechanisms that implement traffic classification.  The most commonly methods are: » Classification by port. » DPI (Deep Packet Inspection) classification. » Statistical classification.  Commercial probes can afford more than 20 Gbps, giving many optional features. However, those probe are extremely expensive. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 4
  • 5. Architecture  The implemented probe has the following modules: » Network module : Intel DPDK is used to capture the traffic at 10 Gbps. It also provides a flexible and scalable architecture, useful for parallel programming. » FlowBuilder module : The module is designed to group packets in different flows. It also provides statistics about every flow inside the network, in a similar way as Cisco NetFlow. » GPU classification module : This module performs the flow classification within a GPU. It maximizes performance, pipelining the analysis of different flows. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 5
  • 6. Architecture – Network and FlowBuilder modules Nic 0 Nic 1 I/O RX 0 Worker 0 I/O TX 0 Nic 1 Nic 0 FlowBuilder (Worker 0) process_packet export_flow A flow expires If there is no expired flow, the function ends. GPU Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 6
  • 7. Architecture – Flow Builder export_flow The flow payload is copied to the buffer return There is empty space GPU Classification The flow buffer is copied into GPU memory The buffer is full of flows The results are returned Get the current active buffer Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 7
  • 8. Architecture – GPU classification 1) A flow block enters inside the module. 2) The flow block is copied inside the GPU memory. 3) The GPU transposes the block. 4) The flows are classified in parallel. 5) The results are transposed again. 6) The results are also reduced in order to minimize data transfers. 7) The flow block is returned to the Host memory. 8) Finally, the results can be stored or relayed. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 8
  • 9. Performance Tests  Network module » Using a hardware traffic generator we have achieved to capture 14.9 Million packets per second without packet loss.  Network and FlowBuilder integration » Performance tests show a high dependency on the number of concurrent flows inside the link. » However, the flow builder can support a typical number of concurrent flows in a 10 Gbps link. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 9
  • 10. Signatures Host ↔ GPU Traspose Matching Traspose Reduction GPU ↔ Host Global Gbps / Mflowps 8 35.92 90.84 51.4 98.27 64.83 260.42 24 / 11.9 16 32.42 84.46 47.35 53.69 34.49 169.84 18 / 8.7 24 33.17 72.67 43.28 36.51 23.39 124.01 13 / 6.1 32 31.38 64.57 39.56 27.80 17.72 100.81 10 / 4.8 40 29.76 58.74 27.08 22.45 14.33 85.38 8 / 3.9 Performance Tests  GPU classification » Reached GPU performance in Gigabits per second. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 10
  • 11. Performance Tests  GPU classification 11
  • 12. Validation Tests  Flows matching ratings » TCP syn 145 / 576 (25,17%) (without content) » ICMP 116 / 576 (20,46%) » NTP 5 / 576 (0.87%) » RTCP 118 / 576 (20.49%) » RTP 143 / 576 (24.83%) » Unknown 49 / 576 (8.51%)  Classification accuracy highly depends on each protocol signature.  Binary protocols represent a major problem to define a signature for them.  According to the protocol, it can be impossible to define a signature without a loss of accuracy.  The used trace was captured in Alcatel-Lucent premises, in the scope of the IPNQSIS project.  The unknown flows are minor protocols that are not important in multimedia classification. In fact, they are network-booting protocols. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 12
  • 13. Conclusions  Network Module: it allows a flexible and parallel architecture, and it takes advantage of the hardware capabilities in a simple way.  FlowBuilder Module: it is very dependent on the number of flows inside a link. It can support up to 10 Gbps without packet losses.  GPU classification Module: it reduces CPU consumption. It highly depends on the quality of the signatures: problem with binary protocols such as RTP.  In summary, the system can process and classify a 10 Gbps data rate in normal traffic conditions.  It has been difficult to validate the system, feeding it at this speed.  Future work » GPU performance has to be improved in order to minimize the impact of multiple signatures. » The FlowBuilder module has to be improved too in order to support higher performance ( > 20 Gbps ) Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware 13
  • 14. Multimedia flow classification at 10 Gbps using acceleration techniques on commodity hardware Rafael Leira1, Pedro Gómez1, Iván González1,2 Jorge E. López de Vergara1,2 <jorge@naudit.es> 1Universidad Autónoma de Madrid, Spain 2Naudit High Performance Computing and Networking, Spain First International Workshop on Quality Monitoring, SaCoNeT, 17th June 2013, Paris, France