IT Brand Pulse Industry Brief describing the market dynamics leading to a new generation of 25, 50 and 100GbE, the Dell products, applications, and guidelines for when to use 25G.
2. Document # Industry2015005 v9, April, 2016 Page 2 of 10
It’s On
QLogic and Dell Announce Industry's First 25GbE Servers
On March 29, QLogic announced a collaboration with Dell resulting in the introduction of the industry’s first servers with
25GbE server connectivity. According to Ravi Pendekanti, vice president of server solutions marketing for Dell, the
company is rolling out 25GbE server adapter options and IT pros can order the products starting April 19.
He also pointed out the product launch starts an exciting journey towards a new generation of 25, 50 and 100Gb
Ethernet technology which was driven by the needs of hyperscale cloud service providers, but will equally benefit the
rest of the non-hyperscale IT world.
For IT organizations, the new generation of networks deliver more cost-effective performance, as well as new features
unrelated to performance which are key to unlocking the potential of software defined data centers.
I believe the announcement establishes early incumbency for QLogic and Dell at a major technology inflection point
which promises to reward first movers with incremental revenue and market share.
Unlike 10GbE Adoption, This Race is a Sprint
10GbE products hit the market in 2003, but the huge disparity in cost versus 1GbE slowed adoption of 10GbE for the
better part of a decade. Adoption of the new generation of 25, 50 and 100GbE products is expected to occur much
faster, led by 25GbE which can plug-and-play using the same cabling technology as 10GbE.
As soon as 25G server connectivity is available, Mr. Pendekanti and Dell expect tier-1 hyperscale cloud service providers,
tier-2/3 cloud service providers, carriers, high-performance computing centers, and large enterprises all to
simultaneously begin deployment in I/O-intensive computing environments.
Because of the expected broad demand for this new Ethernet technology, Crehan Research predicts shipments of 25GbE
server ports (adapter and LOM shipments) will reach 2 million ports in just 3 years.
It took 6 years for 10GbE server connectivity to take off, and another two years before shipments passed 2 million ports per
year. As the need for more network bandwidth increases, Dell is responding with 25G server connectivity offering 2.5x the
performance of 10GbE, but at only 1.4x the cost—highlighting why 25GbE server connectivity will accelerate to 2 million ports
per year in just a few years.
3. Document # Industry2015005 v9, April, 2016 Page 3 of 10
Why IT Needs 25, 50 and 100GbE
Ethernet: Predictable Since 1973
Since 1973 when Ethernet was first invented, the speed of the
network increased by a factor of ten. When adoption of 10Gb and
40Gb Ethernet technology stalled, the industry stubbornly clung
to this road map for over a decade—until hyperscale cloud service
providers broke the log jam.
The Top 5 Reasons the Road Map Changed
1. Powerful new 2-socket and 4-socket servers with dozens of
cores, can drive more than 10Gb of traffic per link.
2. Hundreds of business, SDN/NFV, storage, and HPC apps use
newly available processing power to drive network I/O beyond
10Gb.
3. Cloud service providers use the power of the new servers to
increase VM density, which is critical to their profitability, and
is driving network I/O beyond 10Gb.
4. Upgrading from 10GbE to 40GbE is prohibitively expensive
because of the cost of optics, and because of the costs related
to deploying bulkier cabling.
5. Hyperscale cloud service providers in the million server club
have the purchasing power to motivate vendors like Dell and
QLogic to deliver the technology NOW.
From Concept to GA in Less Than 2 Years
The 25G Ethernet Consortium was formed in July, 2014 to create
specifications for 25 and 50Gb Ethernet. A draft of the 25G
specification was completed in September, 2015, and Dell is
shipping servers with 25GbE ports in April, 2016. In a rapid
development cycle never seen, the Ethernet industry moved from concept to general availability in less than two years.
Looking forward, the Ethernet Road Map features speeds based on multiples of 25 and 50Gb lanes.
Then suddenly…like Obi-wan…our industry felt a great
disturbance in the Force, as if millions of voices of Ethernet
engineers suddenly cried out in terror and were suddenly
silenced. We feared something terrible had happened. That
disturbance to the Ethernet road map was the emergence of
25 gig and 50 gig Ethernet.
Cloud Infrastructure-as-a-Service providers in the million
server club influenced the Ethernet industry to change course
and develop a new Ethernet road map.
4. Document # Industry2015005 v9, April, 2016 Page 4 of 10
Dell Bringing More Than Speed
Because Server Connectivity is More Important Than Ever
The data center world is undergoing a transformation to a software defined data center architecture where business and
infrastructure applications all run on industry standard servers. Therefore a software defined data center is a “server
defined data center,” making server connectivity to networks hugely important, and the requirements have changed
dramatically.
When Google was mostly a search company, the requirement for server connectivity to Ethernet networks was “dumb
and cheap”. But today cloud Infrastructure–as-a-Service providers like Google, AWS and Microsoft have joined
enterprises in hosting hundreds of different workloads, and are asking Dell for server connectivity that is:
Incredibly fast with up to 100G of high bandwidth, and with RDMA for ultra-low latency.
Intelligent with support for a wide variety of protocols.
Efficient with hardware offload of the protocol processing which preserves server CPU for virtual machines and their
workloads.
Anatomy of Intelligent Server Connectivity
The new generation of 25, 50 and 100G Ethernet technology brings with it a high level of intelligence in the form of the
ability to support many protocols needed for LANs, SANs, inter-processor communications (IPC) used in clusters,
integration with cloud platforms, packet processing, and network virtualization. In addition, the new generation of
Ethernet chips increase data center efficiency by offloading server Ethernet driver and protocol processing to preserve
server CPU resources for workloads.
Intelligent
LAN/SAN/IPC/Network Virtualization/Cloud/Packet Processing Protocol and API Support
(TCP/IP, iSCSI, VXLAN. NVGRE, GENEVE, SRIOV, DPDK)
Fast
10, 25, 50 and 100Gb bandwidth
RDMA for low-latency
Efficient
HW offload of driver and vSwitch processing
(TCP/IP. iSCSI, VXLAN, NVGRE, GENEVE, SRIOV)
5. Document # Industry2015005 v9, April, 2016 Page 5 of 10
Dell Products
Servers, Server Adapters and Switches
Dell server solutions marketing vice president Ravi Pendekanti says Dell will be marketing their PowerEdge servers with
QLogic 25GbE adapters and Dell Z9100-ON top-of-rack switches. The combination of products form a powerful and
incredibly flexible platform for computing and I/O in software defined data centers.
Mr. Pendakanti expects early adopters to include cloud Infrastructure-as-a-Service providers, telecoms deploying NFV
server appliances, enterprise customers driving dense virtualization and supporting high transaction based workloads,
and customers interested on standardizing on 25GE as a path to 100GE in the future.
DescriptionProduct
Dell PowerEdge Servers
Servers like the powerful Dell PowerEdge R930, with Intel
Xeon E7-8800 processors, demand 25GbE connectivity
because they’re capable of saturating 10GbE links.
QLogic QL45212 Adapters
Dual port 25GbE Intelligent Ethernet Adapters provide 25G
connectivity for new servers, and support a long list of
protocols and APIs needed for LANs, SANs, NFV appliances,
storage appliances, virtualizing networks to virtual
machines, and cloud integration.
Dell Z9100-ON Switches
1U 10/25/40/50/100GbE fixed port switch which supports
up to 128 ports of 25GbE (QSFP+) using a breakout cable.
The Z9100 is the key to what is expected to be a popular
new rack architecture: 25GbE links from server to ToR
switch, and 100GbE links to aggregation or core switches.
6. Document # Industry2015005 v9, April, 2016 Page 6 of 10
Cloud Service Provider Solution
Server Farms with Super High VM-Density
While enterprise IT organizations are measured by their ability to support continuity for the core business, for Cloud
Infrastructure-as-a-Service providers, their data centers are their core business, and the difference between profit and
loss is how efficiently they operate their infrastructure.
That’s why cloud service providers operate server farms with a super high number of virtual machines per server.
According to a CSP survey by Infonetics, the average number of VMs per server was 42 in 2015, growing to 98 in 2017.
That’s a lot. Along the way they discovered high VM-density leads to an aggregation of I/O, I/O bottlenecks, and that
when you have so many VMs, vSwitches start to consume a large percentage of server CPU.
Best Practice for Networking VMs with 25, 50 and 100GbE Server Ports
Dell servers with QLogic 25GbE adapters offer 2.5x more bandwidth than 10GbE adapters, the previous network adapter
of choice for CSPs. With all that bandwidth, and more coming with 50GbE and 100GbE, it will be essential to create
virtual networks in order to provide quality-of-service (QoS) to dozens of VMs and workloads on a per-VM basis.
Source: Infonetics
Data Center
Strategies Global
Service Provider
Survey, 2015
CSP Virtual Machines per Virtualized Server
Layer SW-based Services on HW-based Virtual Networking Services
7. Document # Industry2015005 v9, April, 2016 Page 7 of 10
Carrier Solution
Software Defined Infrastructure
Around the world, carriers like China Mobile, Telefonica and Verizon, deliver telephone, mobile phone, movie videos,
video surveillance, email, and many more services to you and your home.
In the past carriers used an array of purpose-built content delivery, storage and networking systems in their data
centers. Today, carriers are aggressively transforming their data centers into “software defined” data centers by
replacing purpose-built systems with server-based appliances.
Dell Servers with QL45212 Adapters are an Excellent Platform for NFV/SDN Appliances
Dell servers with QLogic QL45212 Ethernet adapters, support DPDK (data plane development kit) which allows industry
standard servers to do packet processing at line speed. Dell servers can then be used by carriers as hosts for a variety of
different NFV/SDN software appliances.
Where SDN/NFV Helps Carriers Automate & Lower Costs
8. Document # Industry2015005 v9, April, 2016 Page 8 of 10
Enterprise Solution
Scale-Out File Storage
The growth of data storage capacity is not driven by high-performance
databases, it’s driven by unstructured file data such as your photographs,
videos, songs and text messages. Scale-out file storage allows multiple domains
within a data center, or geographically dispersed data centers, to share a single
namespace for more efficient file storage management. But maintaining crisp
user response times across domains is not possible without multiple network
connections and the use of RDMA.
Dell Servers with QLE45212 Adapters are an Excellent
Platform for SMB 3.0
A killer app for RDMA over Converged Ethernet (RoCE) is SMB 3.0 file servers
where users accessing shared storage can experience the response time of
local storage. File servers turbo-charged with RoCE are commercially available via two Windows Server 2012 features
called SMB Multi-Channel and SMB Direct. With SMB Multichannel, SMB 3.0 automatically detects the RDMA capability
and creates multiple RDMA connections for a single session. This allows SMB to use the high throughput, low latency and
low CPU utilization offered by SMB Direct.
High Performance SMB 3.0 Storage Environment
Enterprise Storage System
Capacity Shipped (PB)
In this diagram Dell servers are at the core of a high performance environment for 3 app clusters and 1 file server cluster. Hyper-V
automatically senses the presence of RDMA NICs, then use multi-channel communications to evacuate VMs in seconds, and uses direct
memory access for higher I/O to shared storage inside the blade server.
Dell Servers
9. Document # Industry2015005 v9, April, 2016 Page 9 of 10
The 2550100GbE Zone
For IT professionals looking for some rough guidelines about when and where to use 25, 50 or 100GbE, here’s our top 6
reasons to enter the 2550100 Zone:
1. Your server has 2 sockets or more
A single socket server with one Xeon E7 8890 v3 processor has 18 cores. If your
server has 2 sockets or more, it’s easy to configure 50 cores. We think so much
processing power makes 25GbE server adapters an “must-have” in this case.
2. Your server is hosting more than 20 virtual machines
We think a simple guideline is to configure 1Gb of server adapter bandwidth for
every virtual machine. So our rule-of-thumb is if a server is hosting more than 20
VMs, it’s in the 2550100 Zone.
3. Primary storage for your server is Ethernet-connected all flash storage
All-flash iSCSI and NAS arrays are not yet available with 25, 50 and 100GbE interfaces, but they’re coming. If primary
storage for your server is going to be an Ethernet-connected all flash array, you might want to consider future-proofing
your server with a 25GbE server adapter.
4. Your application supports low-latency networking via RDMA
The Remote Direct Memory Access (RDMA) protocol has been used almost exclusively in the HPC arena for ultra low-
latency cluster interconnects. The technology is going mainstream in the form of RDMA over Ethernet (RoCE) support in
25, 50 and 100GbE server adapters. Because now it’s “just there,” a variety of applications are making use of the
protocol. For example, Microsoft SMB Direct is a popular scale-out file storage application which delivers best
performance when the storage server and client servers have Ethernet adapters communicating with RDMA.
5. You’re building a NFV software appliance
DPDK (Data Plane Development Kit) is a set of software libraries and Ethernet drivers
which allow industry standard servers t to do packet-processing at line-rate. Dell servers
with Intel processors and QLogic 45000 Series 25GbE adapters supporting DPDK, form an
excellent platform for router, firewall and content delivery NFV software appliances.
6. You want a future-proof server rack architecture
For many IT organizations with large populations of server racks pushing the limits of
10GbE links to ToR switches, and 40GbE links to aggregation or core switches, it makes
sense to consider a broad migration to a server rack architecture with 25GbE links to the
ToR switches, and 100GbE links to aggregation or core switches. You’ll end up with 2.5x
more performance headroom, and lot’s of new bells for a variety of different applications.
25GbE
to ToR
100GbE to
Core
10. Document # Industry2015005 v9, April, 2016 Page 10 of 10
The Bottom Line
Hyperscale Technology for the Masses
While hyperscale cloud service providers drove the industry to develop 25G Ethernet technology, the enterprise remains
a much larger market and Dell is well positioned to deliver this hyperscale technology to the masses with their line-up of
PowerEdge servers, Z9100 open networking switches and QLogic 45212 server adapters.
The bottom line is Dell’s ability to deliver server and rack solutions with 25G technology will be a key competitive edge
for as long as other server vendors lag behind.
Related Links
Dell PowerEdge Servers
Dell Networking Z9100-ON Multi-rate Fabric Switch
QLogic 45000 Series Server Adapters
Ethernet Alliance 2016 Ethernet Road Map
IT Brand Pulse
About the Author
Frank Berry is founder and senior analyst for IT Brand Pulse, a trusted source of data and
analysis about IT infrastructure, including servers, storage and networking. Mr. Berry has over
30 years experience in the development and marketing of IT infrastructure. If you have any
questions or comments about this report, contact frank.berry@itbrandpulse.com.