When Amazon EC2 launched in 2006 there was a single instance size: m1.small. Over the past eleven years EC2 has evolved to provide an extensive selection of compute resources to customers including specialized resources such as NVMe SSDs, GPUs, and FPGAs. Under the hood, the servers used to host EC2 instances have transformed from off the shelf designs running virtualization software on the host CPUs to purpose built servers with AWS network and storage components implemented in hardware. Now we are happy to announce a new category of EC2 instances: Amazon EC2 Bare Metal Instances. These instances provide customers access to the physical compute resources of the host processors along with the security, scale, and services of EC2. This session will provide an overview of Bare Metal instances, how VMware used EC2 Bare Metal instances to build VMware Cloud on AWS, and other customer use cases for this new EC2 capability.

1. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
AWS re:INVENT
Amazon EC2 Bare Metal Instances
A a r o n B l a s u i s , P r i n c i p a l P r o d u c t M a n a g e r , E C 2
M a t t h e w S . W i l s o n , S r . P r i n c i p a l E n g i n e e r , E C 2
C M P 3 3 0
N o v e m b e r 2 9 , 2 0 1 7

2. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Introductions

3. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

4. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

5. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Narayan Bharadwaj
Senior Director, Product Management
VMware Cloud on AWS

7. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Day 1: The VMware Cloud on AWS vision
• VMware SDDC running on AWS
bare-metal
• Delivered, operated & supported
by VMware
• On-demand capacity & flexible
consumption
• Operational consistency with on-
prem SDDC
• Enterprise-class SDDC capabilities
• Global AWS footprint and
availability
• Direct access to native AWS
services
SERVICE HIGHLIGHTS
AWS GLOBAL
INFRASTRUCTURE
NSXvSphere vSAN
VMWARE CLOUD ON AWS
POWERED BY VMWARE CLOUD FOUNDATION
vREALIZE, vSPHERE INTEGRATED CONTAINERS, ISV
ECOSYSTEM
CUSTOMER
DATA CENTER
VMWARE SDDC
ON-PREMISES
vCENTER vCENTERvCENTER
AWS GLOBAL INFRASTRUCTURE
AWS
SERVICES
Amazon
EC2
AWS
IoT
AWS
Storage
Amazon
RDB
80+ Services

8. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Elastic Compute Cloud (EC2)—
Elastic virtual servers in the cloud
Physical Servers in
AWS Global Regions
Host server
AWS Hypervisor
Guest 1 Guest 2 Guest nCustomer
AWS

9. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Amazon Elastic Compute Cloud (EC2)—
Elastic virtual servers in the cloud
Physical Servers in
AWS Global Regions
Host server
ESXi + vSAN + NSXCustomer
AWS

10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
I want VMware as a customer

11. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Yes, we can build that!

12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
We’ve been building it for a while…

13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 instance
Nov. 2013: 10 Gbps Enhanced Networking
Xen dom0
Xen Hypervisor
Amazon
EC2
Amazon
S3
Amazon
RDS
AWS Direct
Connect
IAMAWS IoT
82599

14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
C4
• Missionary, passionate,
and uniquely talented
team
• Shared our vision for
hardware innovation and
the cloud
• Deep experience in both
networking and storage
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

15. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
NVMe: Efficient and scalable storage

16. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Jan. 2015: EBS on Nitro Card
EBS
gp2 sc1
io1
st1
blkfront
EC2 instance
blkback
Xen dom0
Xen Hypervisor

17. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local
NVMe
Nitro
Card
Feb. 2017: Local NVMe storage Nitro Cards
EC2 instance
Xen Hypervisor
Xen dom0

18. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local NVMe storage encryption
Local NVMe Nitro CardEC2 instance

19. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Local NVMe storage encryption
Local NVMe Nitro CardEC2 instance

20. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
ENA: Efficient and scalable networking

21. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
2016: 25 Gbps Enhanced Networking
Xen Hypervisor
Amazon
EC2
Amazon
S3
Amazon
RDS
AWS Direct
Connect
IAMAWS IoT
EC2 instanceXen dom0

22. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 instanceEC2 instance
Nov. 2017: 100% Nitro system—C5
EBS
gp2 sc1
io1
st1
Nitro Hypervisor
Amazon
EC2
Amazon
S3
Amazon
RDS
AWS Direct
Connect
IAMAWS IoT
EBS NVMe ENA

23. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
EC2 instance
Nov. 2017: 100% Nitro system—i3.metal
EBS
gp2 sc1
io1
st1
Amazon
EC2
Amazon
S3
Amazon
RDS
AWS Direct
Connect
IAMAWS IoT
EBS NVMe ENA
Aug.

24. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
QPI
Amazon EC2 I3 host

25. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
QPI
I3 host with Nitro Security Chip

26. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
$ aws ec2 run-instances --instance-type i3.metal
--key msw --image-id ami-bf4193c7
Running Bare Metal instances
AMI requirements
• E NA netwo rki ng
• NVMe as boot d evi ce
• PCIe nati ve hot pl ug for l i ve E NI and E BS vol ume
attachment/d etachment
• Most AMIs that run on C5 are compati bl e

27. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
[ec2-user@ip-172-31-3-87 ~]$ lspci -q -d 1d0f:* | grep -v bridge
02:00.0 Ethernet controller: Amazon.com, Inc. Device ec20
05:00.0 Serial controller: Amazon.com, Inc. Device 8250
24:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device 0061
25:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
25:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
28:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
28:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
81:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
81:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
84:00.0 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
84:00.1 Non-Volatile memory controller: Amazon.com, Inc. Device cd00
I3.metal PCIe devices

28. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
[ec2-user@ip-172-31-3-87 ~]$ sudo nvme list | cut -c1-78
Node SN Model
---------------- -------------------- ---------------------------------------
/dev/nvme0n1 vol02a523c58b21e0a9e Amazon Elastic Block Store
/dev/nvme1n1 AWS0516F8A134E5F22E5 Amazon EC2 NVMe Instance Storage
/dev/nvme2n1 AWS5B42954687FCCFBF9 Amazon EC2 NVMe Instance Storage
/dev/nvme3n1 AWS086636B9443B87461 Amazon EC2 NVMe Instance Storage
/dev/nvme4n1 AWS510025976B1F9CB14 Amazon EC2 NVMe Instance Storage
/dev/nvme5n1 AWS0198F05BB235493C1 Amazon EC2 NVMe Instance Storage
/dev/nvme6n1 AWS5A90075D54B15128D Amazon EC2 NVMe Instance Storage
/dev/nvme7n1 AWS02D65EB3F8ACE17C8 Amazon EC2 NVMe Instance Storage
/dev/nvme8n1 AWS5153F263B82DC1786 Amazon EC2 NVMe Instance Storage
NVMe: Both EBS and local storage

29. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Bare Metal instances: What will you build?

30. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Bare Metal instances: What will you build?
Hardware virtualization secured containers
Clear Linux
Project

31. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Bare Metal instances: What will you build?
Applications that benefit from deep performance analysis
“I think ‘some success’ puts it lightly. The [i3.metal]
platform was invaluable for performance debug.
Intel vTune just worked, and it allowed me to find
and fix several issues that would have been difficult
or impossible to locate on our standard clusters.”
- Amazon Redshift team

32. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Next steps

33. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Call to action

34. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

35. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.

36. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Thank you!
M a t t h e w S . W i l s o n A a r o n B l a s i u s
@ _ m s w _ @ A a r o n B l a s i u s