PCI Express* based Storage: Data Center NVM Express* Platform Topologies

PCI Express* based Storage: Data Center
NVM Express* Platform Topologies
Michael Hall
Director of Technology Solutions Enabling, Data Center Group, Intel Corporation
Jonmichael Hands
Technical Program Manager, Non-Volatile Memory Solutions Group, Intel Corporation
SSDS004

2
Agenda
•PCI Express*SSD Data Center Ecosystem –what is the opportunity?
•Platform topology options
•Validation tools and methodologies
•Hot plug support for Intel® Xeon® processor based servers
•Upcoming workshops

3
Agenda

4
PCI Express*and NVM Express*SSD Advantages Over SATA
Lower latency: Direct connection to CPU, increased CPU efficiency
Scalable performance:1 GB/s per lane –4 GB/s, 8 GB/s, … in one SSD
Industry standards: NVM Express*and PCI Express*(PCIe*) 3.0
Increased I/O: Up to 40 PCIelanes per CPU socket
Security protocols: Trusted Computing Group Opal
Low Power features: Low power link (L1.2), NVM Express*power states
Form factors: SFF-8639, SATA Express*, M.2, Add in card
Future: BGA (PCI-SIG), high density FF (SSD Form Factor WG)
Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark*and MobileMark*, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products.

5
Form Factors for PCI Express*
Data Center
Client
SFF-8639
SATA Express
AIC
SFF-8639
SATA Express*
M.2
Add in Card
M.2
BGA
HD SSD FF

6
80% increase in
Data Center SSD
revenue projected
SSD Market is Exploding
Source: Forward Insight and Intel
$0
$5
$10
$15
$20
2014
2017
SSD Market Billions $
Client
Data Center

7
PCIeSSDs
are replacingSATA
in the Data Center
PCI Express*SSD Adoption in the Data Center
13%
17%
27%
32%
46%
53%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2013
2014
2015
2016
2017
2018
Data Center SSD Capacity (GB) by Interface
SATA
SAS
PCIe
PCI Express* (PCIe*)
Source: Forward Insight and Intel

8
What do I need to start using an SSD?
PCI Express*support
Software and drivers
Infrastructure

9
Agenda

10
Drive Connectors
SATA Signal Pins
Power and Control Pins
SATA
•Keyed only for SATA drives
•Separate power and data
Key
Signal Pins(port A)
Signal Pins (Dual port, B)
SAS
•Backwards compatible with SATA
•Dual port
SFF-8639
•Supports SATA, SAS, and PCI Express* (PCIe*) x4 or two x2
•PCIedata, reference clock, and side band
SAS / SATA
SAS
RefClk 0 & Lane 0
Lanes 1-3, SMBus, & Dual Port Enable
Refclk 1, 3.3V Aux, & Resets

11
SATA Express*and SFF-8639 Comparison
SATAe
SFF-8639
SATA
Yes
Yes
PCIExpress*
x2
x2 or x4
Host Mux
Yes
No
Ref Clock
Optional
Required
EMI
SRIS
Shielding
Height
7mm
15mm
MaxPerformance
2 GB/s
4 GB/s
Bottom Line
Flexibility& Cost
Performance
SFF-8639 designed for data center, SATAedesigned for Client
Source: Seagate*(with permission)

12
M.2 Form Factor Comparison
Host Socket 2
Host Socket 3
Device w/ B&M Slots
M.2 Socket 2
M.2 Socket 3
SATA
Yes, Shared
Yes, Shared
PCIex2
PCIex4
No
Yes
CommsSupport
Yes
No
Ref Clock
Required
Required
Max Performance
2 GB/s
4 GB/s
Bottom Line
Flexibility
Performance
M.2 Socket 3 is the best option for Data Center PCI Express*(PCIe*) SSDs

13
Cabling Options for Data Center PCI Express*SSD Topologies
Reference Clock
PCIe Reset
SMBUS
miniSASHD cables lightly modified for PCI Express*(PCIe*)
Reference Clock

14
Basic PCI Express*SSD Topology –1 Connector
SFF-8639 Connector directly attached to board
1

15
1
2
miniSASHD Connector
PCI Express* (PCIe*) Cable
PCIe 3.0 x4 Enterprise SSD
SFF-8639 Connector
External Power

16
Motherboard
1
miniSASHD Connector
miniSASHD Connector
3
Backplane
SSD Drive Carrier
SFF-8639 Connector
2

17
Link Extension Devices –Switches and Retimers
Use Link Extension Devices for longer topologies
Retimer
PCIe3.0 x8 link
x8 link
PCIeSSD
x4link
Switch
PCI Express*(PCIe*) 3.0 x16 link
x32 link
PCIeSSD
x4link
Intel CPU

18
Complex PCI Express*Topology –4 Connector
PCIe x16 slot
Cabled Add in card with Link Extension
miniSASHD for PCIe
Backplane
SSD Drive Carrier
1
2
3
4
SFF-8639 Connector

19
Complex PCI Express*Topology –5 Connector
PCIe x16 slot
PCIe Cable
Cabled Add in card with Link Extension
miniSASHD for PCIe
Backplane
SSD Drive Carrier
1
2
4
SFF-8639 Connector
PCI Express* (PCIe*) x16 Riser
3
5

20
PCI Express*cabling for future topologies -OCuLink*
Category
OCuLink*
Standard Based
PCI-SIG
PCI Express* (PCIe*) Lanes
X4
Layout
Smaller footprint
Signal Integrity
Similaron loss dominated channels
PCIe4.0ready
16GT/s target
Clock, power
Supports clock and 3.3/5V power
Production Availability
Mid2015
12.85mm
2.83mm
Source:
OCuLinkinternal cables and connectors

21
OCuLink*Provides Flexible Data Center Topologies
Board to board connections
Cabled add in card
Backplane
SFF-8639 Connector
PCI Express* (PCIe*) SSD
Source:

22
Intel® Server Board S2600WT System with NVM Express*Support
Cabled PCIe 3.0 x16 AIC
SFF-8639
Backplane
miniSASHD for PCI Express*(PCIe*)
x16 Riser
miniSASHD for PCIe Cables
Drive
Carriers
2U Server

23
Agenda

24
Industry goal is to get to the point where add in cards are today – they just work!
1.Physical Layer
•New fixtures required for SFF-8639
2.Configuration Space –no change
3.Link & Transaction Layer –no change
4.Platform Interop at Workshops
•Use adapters for M.2 and SFF-8639
PCI Express*Electrical Testing for SFF-8639
3.0 Compliance

25
Agenda

26
What is required to support hot plug?
+
Server (Hardware + BIOS)
NVM Express*and PCI bus driver
SSD that supports unplanned power loss

27
Hot AddHot Remove
Insert PCIeSSD Drive
BIOS configures PCI Express*(PCIe*) Port for Hot Plug
OS’s PCIeBus Driver setup
Hardware Presence detect
Vendor PCIeSSD Driver loaded
Storage Software & User determines usage
Drivers in known statePCIeSSD Drive inactive
Remove PCIeSSD Drive
BIOS configures PCIePort for Hot Plug
OS’s Disk driver disable, unloaded driver
Vendor PCIeSSD Driver – Failed LED
Storage Software or Driver determines Failure Replace
OS’s PCIeBus Driver cleanup

28
Presence Detect
IO Timeout
Drive Active
Surprise Hot Remove
BIOS configures PCI Express*(PCIe*) Port for Hot Plug
Failed Access in Vendor PCIeSSD Driver
Storage SW or Driver determines Failure Replace
OS’s PCIeBus disable, unload driver
IO timeout in Vendor PCIeSSD Driver
Race
Master Abort
OS’s PCIeBus Driver cleanup
Remove PCIeSSD Drive

29
Agenda

3020132014Q1Q2Q3Q4Q1Q2Q3Q4
NVM Express* (NVMe) Community IDF
NVM Express Community IDF
SFF 8639 Spec
Platform testing Taiwan
Platform testing
US and Taiwan
Non-Sig Compliance boards available
SFF-8639
Plugfest#1
UNH NVMePlugfest#1
UNH NVMePlugfest#2
UNH NVMePlugfest#3
Nov 2014
First PCI Express* 3.0 Integrators list
Testing Events
PCI-SIG and Compliance
NVMeCommunities at IDF
Form Factor
PCI Express*Ecosystem Workshops and Plugfests
NVMePlugfests
UNH –University of New Hampshire

31
•NVM Express*(NVMe) Solid-State Drives are going to become pervasive in the data center
•Intel is accelerating the ecosystem to make it easier to deploy complex PCI Express*(PCIe*) SSD topologies with Intel® Xeon® processor based platforms
•PCIeprovides multiple form factors and flexible topologies for designing into servers and market segments with different requirements
•Start designing new PCI Express form factors into servers to take full advantage of NVMe!
Summary

32
•Design servers to support PCI Express*(PCIe*) Solid-State Drives to take advantage of the performance and efficiency of NVM Express*(NVMe) SSDs
•Get involved with NVMeat www.nvmexpress.organd participate with PCI-SIG at www.pcisig.comfor developments of new storage technology
•See your Intel representative for more information about what Intel is doing to accelerate PCIeSSDs in the data center
•Participate in industry events to advance the PCIeecosystem to support new form factors and topologies
Next Steps

33
A PDF of this presentation is available from our Technical Session Catalog: www.intel.com/idfsessionsSF. This URL is also printed on the top of Session Agenda Pages in the Pocket Guide.
Demos in the showcase –Booths #175 and #259
Additional info in the NVM Express*community –Booths #161-178
More web based info: www.intel.comssd
Additional Sources of Information

34
Legal Disclaimer
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS.NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT.EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death.SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY,OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice.Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined".Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them.The information here is subject to change without notice.Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications.Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your product order.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to:http://www.intel.com/design/literature.htm
Intel, Xeon, Look Inside and the Intel logo are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright ©2014 Intel Corporation.

35
Risk Factors
The above statements and any others in this document that refer to plans and expectations for the second quarter, the year and the future are forward- looking statements that involve a number of risks and uncertainties. Words such as “anticipates,” “expects,” “intends,” “plans,”“believes,” “seeks,” “estimates,” “may,” “will,” “should” and their variations identify forward-looking statements. Statements that refer to or are based on projections, uncertain events or assumptions also identify forward-looking statements. Many factors could affect Intel’s actual results, and variances from Intel’s current expectations regarding such factors could cause actual results to differ materially from those expressed in these forward-looking statements. Intel presently considers the following to be important factors that could cause actual results to differ materially from thecompany’s expectations. Demand for Intel's products is highly variable and, in recent years, Intel has experienced declining orders in the traditional PC market segment. Demand could be different from Intel's expectations due to factors including changes in business and economic conditions; consumer confidence or income levels; customer acceptance of Intel’s and competitors’ products; competitive and pricing pressures, including actions taken by competitors; supply constraints and other disruptions affecting customers; changes in customer order patterns including order cancellations; and changes in the level of inventory at customers. Intel operates in highly competitive industries and its operations have high costs that are either fixedor difficult to reduce in the short term. Intel's gross margin percentage could vary significantly from expectations based on capacity utilization; variationsin inventory valuation, including variations related to the timing of qualifying products for sale; changes in revenue levels; segment product mix; the timing and execution of the manufacturing ramp and associated costs; excess or obsolete inventory; changes in unit costs; defects or disruptions in the supply of materials or resources; and product manufacturing quality/yields. Variations in gross margin may also be caused by the timing of Intel product introductions and related expenses, including marketing expenses, and Intel's ability to respond quickly to technological developments and to introduce new products or incorporate new features into existing products, which may result in restructuring and asset impairment charges. Intel's resultscould be affected by adverse economic, social, political and physical/infrastructure conditions in countries where Intel, its customers or its suppliers operate, including military conflict and other security risks, natural disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Intel’s results could be affected by the timing of closing of acquisitions, divestitures and other significant transactions. Intel's results could be affected by adverse effects associated with product defects and errata (deviations from published specifications), and by litigation or regulatory matters involving intellectual property, stockholder, consumer, antitrust, disclosure and other issues, such as the litigation and regulatory matters described in Intel's SEC filings. An unfavorable ruling could include monetary damages or an injunction prohibiting Intel from manufacturing or selling one or more products, precluding particular business practices, impacting Intel’s ability to design its products, or requiring other remedies such as compulsory licensing of intellectual property. A detailed discussion of these and other factors that could affect Intel’s results is included in Intel’sSEC filings, including the company’s most recent reports on Form 10-Q, Form 10-K and earnings release.
Rev. 4/15/14

37
PCI Express*(PCIe*) Switches and Retimers
PCI Express*(PCIe*) Switches
•User configurable lane distribution
•Ease of implementation and hotplugsupport
•Less BIOS development needed
•Slot configurability
•Acts like PCIeHBA
•Extra software features
•Switches available from Avago*– PLX at www.plxtech.com
PCIeRetimers
•Channel has > -20db loss: at 8GT/s PCIe 3.0
•Intel co-authored ECN spec in PCI-SIG
•Retimersavailable from www.IDT.com
Definitions
•Repeater: A Retimeror a Re-driver
•Re-driver: Analog and not protocol aware
Retimer: Physical Layer protocol aware, software transparent, Extension Device. Forms two separate electrical sub-links.

38
PCI Express*Hot Plug: Supported on Intel® Xeon® Processor Based Servers
Terminology
•Hot Plug: general term to describe adding and removing devices while system is running
•Hot Add –Also known as Hot Insertion
•Hot Removal –Software Managed Hot Removal (orderly)
•Surprise Hot Removal –possible outstanding IO transactions
•Hot Swap (Hot Add + Removal)
Requirements for Surprise Removal
•Hardware: registers and drive status, master abort, and disable link
•Software: PCI Bus Driver and NVM Express*Driver
•Drive: Support unplanned power loss
•LER, DPC, eDPC–not required but make it easier to validate

39
Hot Plug Requirements –System
•PCI Express*(PCIe*) Slot Capability register: Hot Plug Capableand Hot Plug Surprise
•PCIeSlot Status: Presence Change Interruptto notify PCIebus driver
•Backplane, pre-charge circuit to limit in- rush current, isolated Reset, Refclk, and Smbus, presence detect via IfDet# (pin 4) and PRSNT# (pin10)
•Drive Identifyand FailIndicators
•PCIeLink Down Interrupt –for link down, uses PCIeAER
•BIOS: UEFI 2.3.1 or later, pre-allocate memory resources
•Pre-allocate slot resources (Bus IDs, interrupts, memory regions) using ACPI tables

PCI Express* based Storage: Data Center NVM Express* Platform Topologies

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie PCI Express* based Storage: Data Center NVM Express* Platform Topologies

Ähnlich wie PCI Express* based Storage: Data Center NVM Express* Platform Topologies (20)

Mehr von Odinot Stanislas

Mehr von Odinot Stanislas (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

PCI Express* based Storage: Data Center NVM Express* Platform Topologies