AWS Black Belt Online Seminar 2018 re:Invent Recap: Compute, Container and Network

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.2018.01.17
【AWS Black Belt Online Seminar】
re:Invent Recap: Compute, Container and Network

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
• 2018 1 17
AWS (http://aws.amazon.com)
• AWS
AWS
•
• AWS does not offer binding price quotes. AWS pricing is publicly available and is subject to
change in accordance with the AWS Customer Agreement available at
http://aws.amazon.com/agreement/. Any pricing information included in this document is
provided only as an estimate of usage charges for AWS services based on certain information
that you have provided. Monthly charges will be based on your actual use of AWS services,
and may vary from the estimates provided.

Who am I ?
Keisuke Nishitani
Specialist Solutions Architect, Serverless
Amazon Web Service Japan K.K
@Keisuke69 Keisuke69 Keisuke69 Keisuke69Keisuke69x

• API Gateway, Lambda, Cognito, S3,
Kinesis, DynamoDB, Rekognition etc…
2018 2
Amazon
http://amzn.asia/0QzrGYh

Compute update

AMAZON EC2 HOST

NITRO SYSTEM

CPU
C3
re:Invent 2013
AMAZON EC2 C3

EC2
re:Invent 2014
AMAZON EC2 C4

C5 EC2
Annapurna AWS
Linux KVM
EC2
AMAZON EC2 C5

AMAZON EC2 C5
CPU
HPC
Skylake Xeon Platinum 8000 3.0GHz
Turbo Boost 3.5GHz AVX-512
(C4) 25% /
NITRO CPU
Elastic Network Adaptor(ENA) 25Gbps
3

N EW
!
Bare Metal
Instances
AWS

Bare Metal Instances
•
• AWS AWS
OS
• Bare Metal i3.metal
• CPU:Intel Xeon E5-2686v4 2Processers(2.3Ghz), 36core(HT
72core)
• :512GiB
• : 15.2TB SSD
NVMe
• :25Gbps (ENA )

Burstable
T2
Big Data
Optimized
H1
Memory
Optimized
R4
In-memory
X1
High
I/O
I 3
Compute
Intensive
C5
Graphics
Intensive
G3
General
Purpose GPU
P3
Memory
Intensive
X1e
General
Purpose
M5
Virtual
Private
Servers
Bare Metal
High I/O
I 3m
Dense
Storage
D2 F1
FPGA
Amazon
Lightsail
EC2 Elastic GPUs
Graphics accel eration
for EC2 instances
EC2 Spot Instances
• Hibernation
• No Bid Pricing
N E
W !
NEW! NEW! NEW!
NEW!

Amazon EC2 H1
•
• D2 CPU
• I/O
CPU Intel Xeon E5-2686 v4
•
vCPU NW
h1.2xlarge 8 32GiB 2TB 10Gbps
H1

Amazon EC2 M5
•
• 2.5GHz Xeon Platinum 8000 Skylake M4
14%
• NITRO
• EBS
• Intel-AVX512
•
vCPU EBS NW
m5.large 2 8GiB 2,120Mbps 10Gbps
m5.xlarge 4 16GiB 2,120Mbps 10Gbps
m5.2xlarge 8 32GiB 2,120Mbps 10Gbps
New Hypervisor!
M5

T2 Unlimited
• CPU
CPU
• 24
• Cloudwatch
• 1vCPU Linux $0.05 Windows $0.096
•

Nitro Hypervisor(C5&M5)
OS ENA + NVMe ACPI OS
OS ( https://aws.amazon.com/jp/ec2/faqs/)
Amazon Linux 2014.03 or newer Ubuntu 14.04 or newer
SUSE Linux Enterprise Server 12 or newer Red Hat Enterprise Linux 7.4 or newer
CentOS 7 or newer FreeBSD 11.1-RELEASE
Windows Server 2012 R2 Windows Server 2016
Amazon Linux AMI
• AMI acpid (yum
install acpid) stop
CentOS
• CentOS 6: ENA
• CentOS 7: Marketplace AMI M5 (C5 )
SLES 12
• AMI NVMe initramfs initramfs
AMI
M5C5

Container update

DAY ONE!

450+%
(2016 )
50+
2015 GA

=
API

Scheduling and Orchestration
Cluster Manager Placement Engine
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance
ECS
AMI
Docker
agent
ECS
agent
EC2 Instance

AWS
AWS VPC
AWS
ECS CLI{ }
(2018 )
CloudWatch

{
”cpu": “1 vCPU”,
”memory": “2 gb”,
"networkMode": ”AWSVPC",
"compatibilities": [”FARGATE",
”EC2"],
"placementConstraints": [],
"containerDefinitions": [
{
<snip>…....
• CPU
: CPU Task
Level
Resources

CPU
50 CPU/
CPU Memory
256 (.25 vCPU) 512MB, 1GB, 2GB
512 (.5 vCPU) 1GB to 4GB (1GB )
1024 (1 vCPU) 2GB to 8GB (1GB )
2048 (2 vCPU) 4GB to 16GB (1GB )
4096 (4 vCPU) 8GB to 30GB (1GB )

FARGATE: UNDER THE
HOOD

HOW DO I RUN
CONTAINERS
ON FARGATE?

Fargate

ECS CONSTRUCTS
Define application containers:
Image URL, CPU & Memory
requirements, etc.
register
Task Definition
create
Cluster
• Infrastructure Isolation boundary
• IAM Permissions boundary
run
Task
• A running instantiation of a
task definition
• Use FARGATE launch type
create
Service
Elastic Load
Balancer
• Maintain n running copies
• Integrated with ELB
• Unhealthy tasks automatically
replaced

Fargate
ECS API
- Fargate EC2
Task Definition

VPC INTEGRATION 172.31.0.0/16
Subnet
172.31.1.0/24
Internet
Other Entities in VPC
EC2 LB DB etc.
Private IP
172.31.1.164
• AWS VPC Networking Mode
• Fargate VPC/
• /
• Public IP
• AZ
us-east-1a
us-east-1b
us-east-1c
ENI Fargate
TaskPublic /
208.57.73.13 /

LOAD BALANCING
APPLICATION LOAD BALANCER
NETWORK LOAD BALANCER
Task network mode awsvpc
ALB/NLB target type ip ECS/Fargate

SECURITY

PROD Cluster Infrastructure
DEV Cluster Infrastructure
BETA Cluster Infrastructure
QA Cluster Infrastructure
Web Web
Shopping
Cart
Shopping
Cart
Notifications NotificationsWeb
Shopping
Cart NotificationsWeb
Shopping
Cart
Shopping
Cart
Notifications NotificationsWeb Web
PROD CLUSTER BETA CLUSTER
DEV CLUSTER QA CLUSTER

PERMISSION TIERS
Cluster
Permissions
Application
Permissions
Task
Housekeeping
Permissions
Cluster
Fargate Task
Cluster Permissions:
/
Application (Task) Permissions:
AWS
Housekeeping Permissions:
ECS
e.g.
• ECR Image Pull
• CloudWatch Logs pushing
• ENI creation
• Register/Deregister targets into ELB

CONTAINER REGISTRIES

REGISTRY SUPPORT
3rd Party Private Repositories (coming soon!)
Public Repositories supported
Amazon Elastic Container Registry (ECR)

VISIBILITY AND MONITORING
CloudWatch Logs
CloudWatch Events

STORAGE
Container Storage Space 10GB
Ephemeral storage backed by EBS
4GB

Fargate EC2
• Fargate
• Windows Containers
• GPU Support
• docker exec
• Spot RI
• Task
• Fargate OK

Fargate Lambda

99.99%

What is Kubernetes?

63% of Kubernetes workloads
run on AWS today
—CNCF survey

3x Kubernetes masters for HA
Kubernetes on AWS

API
server
Cloud
controller
Controller
manager
Scheduler Add-onsKubeDNS
Kubernetes master

Availability
Zone 1
Etcd
Master
Etcd
Master
Etcd
Master
Availability
Zone 2
Availability
Zone 3

Availability
Zone 1
Etcd
Master
Etcd
Master
Availability
Zone 2
Availability
Zone 3
Etcd
Master

“Run Kubernetes for me.”

“Native AWS Integrations.”

”An Open Source Kubernetes Experience.”

Amazon Elastic Container Service
for Kubernetes (EKS)

Tenet 1
EKS
Tenet 2
EKS
Kubernetes
Tenet 3
EKS AWS
Tenet 4
EKS Kubernetes

Availability
Zone 1
Etcd
Master
Etcd
Master
Availability
Zone 2
Availability
Zone 3
Etcd
Master

mycluster.eks.amazonaws.com
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
Kubectl

APIAPIAPIAPI
EKS

aws eks create-cluster –cluster-name reinvent2017 –desired-master-version
1.7.1 –role-arn arn:aws:iam::account-id:role/role-name

HTTP/1.1 200 Content-type:
application/json
{ "cluster":
{
"clusterName": "string",
"createdAt": number,
"currentMasterVersion": "string",
"desiredMasterVersion": "string",
"masterEndpoint": "string",
"roleArn": "string",
"status": "string",
"statusMessage": "string"
}
}
aws eks create-cluster

aws eks describe-cluster –cluster-name reinvent2017

application/json
{ "cluster":
{ "clusterName": "string",
"status": "string",
"statusMessage": "string" }
}
aws eks describe-cluster –cluster-name reinvent2017

aws eks list-clusters

HTTP/1.1 200
Content-type: application/json
{
"clusterArns": [ "string" ],
"nextToken": "string"
}
aws eks list-clusters

aws eks delete-cluster –cluster-name
reinvent2017

application/json
{ "cluster":
{ "clusterName": "string",
"status": "string",
"statusMessage": "string" }
}
aws eks delete-cluster –cluster-name reinvent2017

Fargate EKS 2018

Native VPC networking
with CNI plugin
Pods have the same VPC
address inside the pod
as on the VPC
Simple, secure networking
Open source and
on Github
…{ }

Kubernetes Network
Policies enforce network
security rules
Calico is the leading
implementation of the
network policy API
Open source, active
development (>100
contributors)
Commercial support
available from Tigera

Heptio IAM Authenticator
An open source approach to integrating
AWS IAM authentication with Kubernetes

Kubectl
3) Authorizes AWS Identity with RBAC
K8s API
1) Passes AWS Identity
2) Verifies AWS Identity
4) K8s action
allowed/denied
AWS Auth

Network update

( )
Network
ISP,IXP,VPN
SE AWS
AWS : ELB,Direct Connect,VPC,Market Place

Agenda
• AWS Direct Connect Gateway
• AWS PrivateLink
• Amazon VPC(Inter-region VPC peering)
• AWS Time Sync Service

• Direct Connect Gateway Hub
VPC
• Direct Connect VPC
• 1 Direct Connect VPC
• Direct Connect Direct Connect Gateway
Direct Connect Gateway
1 Direct Connect
VPC

1. Direct Connect Gateway Direct Connect
•
2. Direct Connect Gateway VPC
• VPC
3. VPC
4. VIF VPC
us-east-1
(
ap-northeast-1
( )
1. 2. 172.16.0.0/16
10.0.0.0/16
3.
4.

AWS PrivateLink

AWS PrivateLink for AWS
• https://aws.amazon.com/jp/about-aws/whats-
new/2017/11/introducing-aws-privatelink-for-aws-services/
• IP
Amazon Virtual Private
Cloud (VPC) AWS
•
• Amazon Kinesis Streams
• Elastic Load Balancing API
• Amazon Elastic Compute Cloud (Amazon EC2) API
• Amazon EC2 Systems Manager (SSM)
• AWS Service Catalog

VPC Endpoint
VPC Endpoint
IP AWS
VPC
S3
Dynamo DB
VPC
Endpoint

VPC Endpoint
VPC Endpoint VPC Endpoint
VPC Endpoint for S3
VPC Endpoint for DynamoDB
VPC Endpoint for S3
VPC Endpoint for DynamoDB
Gateway
EC2
PrivateLink:Interface
SSM Kinesis Stream
ServiceCatalog ELB
VPC Endpoint Gateway Privatelink(Interface )

•
VPC
• IP
IP
Gateway
S3
Dynamo DB
VPC Endpoint
(vpce-1a2b3c)
S3 vpce-1a2bc3
S3 IP pl-xxxxx
Destination AWS
Amazon Provided DNS
S3 IP 52.219.68.108
52.219.68.108

•
IP
• VPC DNS
IP
•
IP
PrivateLink (Interface )
EC2 API
Kinesis Streams API
Amazon Provided DNS
API
EC2 VPC
10.0.0.100
EC2 VPC
10.0.1.100
EC2 API IP 10.0.0.100 or
10.0.1.100
10.0.0.100
AWS
EC2 API
API
IP
IP
ELB
RDS

Gateway PrivateLink(Interface )
IAM Policy
IP
1 IP
0.014 USD/ + 0.01 USD/ GB
https://aws.amazon.com/jp/vpc/pricing/
AZ
AZ

AWS PrivateLink
VPC
AWS PrivateLink
AWS
Network Load Balancer TCP
AWS Marketplace
SaaS
• CA Technologies – CA Infrastructure Management Essentials, CA App Experience Analytics
Essentials, CA Application Performance Management Essentials.
• Aqua Security – Aqua Container Image Security Scanner.
• Dynatrace – Could-Native Monitoring powered by AI.
• Cisco Stealthwatch – Public Cloud Monitoring – Metered, Public Cloud Monitoring – Contracts.
• Sig Opt – ML Optimization & Tuning.
& VPC

Privatelink
A
10.1.1.0/24
VPC CIDR: 10.1.0.0 /16
B
10.2.1.0/24
VPC CIDR: 10.1.0.0 /16 VPC CIDR: 10.1.0.0 /16 NLB
•
•
•
• IP

Inter-region VPC peering

VPC
VPC
• VPC
VPC
• AWS
•
•

VPC Peering (VPC )
A
10.1.1.0/24
VPC CIDR: 10.1.0.0 /16
B
10.2.1.0/24
Route Table
Destination Target
10.2.0.0/16 local
0.0.0.0/0 pcx-xxxxxx
VPC CIDR: 10.1.0.0 /16 VPC CIDR: 10.2.0.0 /16
pcx-xxxxxx
VPC
AWS
AWS (
) VPC
MTU (VPC Peering 1,500)
Route Table
Destination Target
10.0.0.0/16 local
0.0.0.0/0 pcx-xxxxxx
http://docs.aws.amazon.com/ja_jp/AmazonVPC/latest/UserGuide/vpc-peering.html

Amazon Time Sync Service

Amazon Time Sync Service
• VPC NTP
• EC2 NTP IP
169.254.169.123
–
• Leap Smearing
•

Leap Smearing (2015/07/01)
UTC AWS Adjusted Clock AWS vs. UTC Notes
11:59:59 AM June 30th, 2015 11:59:59 AM June 30th, 2015 +0
AWS clocks are synchronized to UTC.
12:00:00 PM 12:00:00 PM +0
12:00:01
Each second is 1/86400 longer and AWS clocks fall behind UTC.
The gap gradually increases to up to 1/2 second.
12:00:01 +1/86400
12:00:02
12:00:02 +2/86400
… … …
23:59:59
23:59:59 +43199/86400
23:59:60 Leap second injected to UTC.
00:00:00 AM July 1st, 2015 -1/2 AWS clocks gain 1/2 second ahead of UTC.
00:00:00 AM July 1st, 2015
AWS clocks keep falling behind and the gap with UTC shrinks
gradually.
00:00:01 -43199/86400
00:00:01
00:00:02 -43198/86400
… … …
11:59:59 AM -1/86400
11:59:59 AM
12:00:00 PM July 1st ,2015 12:00:00 PM July 1st ,2015 +0
The gap shrinks to zero. AWS clocks synchronize to UTC again.
12:00:01 12:00:01 +0
… … …
AWS 12
1 1/864000
(Smearing )
1/2
AWS 1/864000
0.5
12
UTC
Smearing
https://aws.amazon.com/jp/blogs/aws/look-before-you-leap-the-coming-leap-second-and-aws/

AWS
• http://aws.amazon.com/jp/aws-jp-introduction/
AWS Solutions Architect
• Q&A
• http://aws.typepad.com/sajp/

Twitter/Facebook
AWS
@awscloud_jp
http://on.fb.me/1vR8yWm

AWS
AWS
https://aws.amazon.com/jp/contact-us/aws-sales/
AWS

AWS Black Belt Online Seminar 2018 re:Invent Recap: Compute, Container and Network

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to AWS Black Belt Online Seminar 2018 re:Invent Recap: Compute, Container and Network

Similar to AWS Black Belt Online Seminar 2018 re:Invent Recap: Compute, Container and Network (20)

More from Amazon Web Services Japan

More from Amazon Web Services Japan (20)

Recently uploaded

Recently uploaded (20)

AWS Black Belt Online Seminar 2018 re:Invent Recap: Compute, Container and Network