AWS公式オンラインセミナー: https://amzn.to/JPWebinar 過去資料: https://amzn.to/JPArchive

1. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 1
( )
EC2
2019-03-18
Amazon EC2

2. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2
•
•
•
•
•
•
•
•
•
•
•

3. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 3

4. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 4
https://www.itmedia.co.jp/enterprise/articles/1902/22/news006.html

5. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 5
AI AWS (1/4) - ITmedia
https://www.itmedia.co.jp/enterprise/articles/1902/22/news006.html

6. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 6

7. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 7

8. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 8

9. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 9

10. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 10
( )
( 5 )
2019-03-18
Amazon EC2

11. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 11

12. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 12

13. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 13
•
•
•
•
•
•
•
•
•
•
•

14. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 14

15. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 15

16. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 16

17. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 17
•
•
•
•
•
•
•

18. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 18
•
•
•
•
•
•
•

19. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 19

20. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 20
AWS
CloudFormation
AWS
OpsWorks
Amazon
EMR
Amazon
ECS
AWS Data
Pipeline
AWS BatchAuto Scaling
AWS Thinkbox
Deadline

21. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 21
•
•
•
•
•

22. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 22

23. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 23

24. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 24
•
•
•
•

25. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 25

26. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 26
•
•

27. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 27
•
•
•
•

28. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 28

29. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 29

30. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 30

31. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 31
•

32. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 32
• 1
• ( )

33. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 33

34. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 34
AWS
CLI
(aws ec2 ...) (EC2)
run-instances
(request-spot-
instances)
one-time (sync) /
persistent (sync)
request-spot-
fleet
/
vCPU
( )
request (async) /
maintain (async)
EC2
create-fleet /
vCPU
( )
instant (sync) /
request (async) /
maintain (async)
Auto Scaling
(New!)

35. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 35
AWS
CLI
(aws ec2 ...) (EC2)
run-instances
(request-spot-
instances)
one-time (sync) /
persistent (sync)
request-spot-
fleet
/
vCPU
( )
request (async) /
maintain (async)
EC2
create-fleet /
vCPU
( )
instant (sync) /
request (async) /
maintain (async)
Auto Scaling
(New!)

36. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 36
AWS
CLI
(aws ec2 ...) (EC2)
run-instances
(request-spot-
instances)
one-time (sync) /
persistent (sync)
request-spot-
fleet
/
vCPU
( )
request (async) /
maintain (async)
EC2
create-fleet /
vCPU
( )
instant (sync) /
request (async) /
maintain (async)
Auto Scaling
(New!)

37. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 37
AWS
CLI
(aws ec2 ...) (EC2)
run-instances
(request-spot-
instances)
one-time (sync) /
persistent (sync)
request-spot-
fleet
/
vCPU
( )
request (async) /
maintain (async)
EC2
create-fleet /
vCPU
( )
instant (sync) /
request (async) /
maintain (async)
Auto Scaling
(New!)

38. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 38
m4.large Spot ASG Min: 1 Max: 10
m5.large Spot ASG Min: 1 Max: 10
c4.xlarge O-D ASG Min: 1 Max: 10
Availability
Zone 1
Availability
Zone 2
Availability
Zone 3
-
ASG
ASG
-
ASG
ASG: Auto Scaling Group
O-D: On Demand Instance

39. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 39
m4.large Spot Instances
m5.large Spot Instances
c4.xlarge On-Demand instances
Availability
Zone 1
Availability
Zone 2
Availability
Zone3
ASG
Single ASG
ASG: Auto Scaling Group
O-D: On Demand Instance
RI: Reserved Instance

40. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 40
•
Auto Scaling
Group

41. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 41
EC2 Auto Scaling Spot Fleet
AZ
[1]
1. Total target capacity
2. "Optional On-demand
portion
3.
[2]
1. Desired Capacity
2. "On-Demand base"
3. OD:Spot Ratio
ELB
Launch Template
$latest, $default
[1] https://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/spot-fleet.html#on-demand-in-spot
[2] https://docs.aws.amazon.com/ja_jp/autoscaling/ec2/userguide/AutoScalingGroup.html#asg-purchase-options

42. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 42
•
•
•
•
•
•
•
•
•

43. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 43

44. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 44
On Demand Auto-
Scaling group
Output S3
bucket
Availability Zone A
Availability Zone B
Spot Fleet
Upload object
into input S3
bucket
Job SQS
Queue
Auto Scaling group and Spot fleet
scale up based on queue depth and
scale down based on CPU utilization
CW metrics
Workers will
check for jobs
in the queue
Workers will update Job status
(start time, SLA end time, etc) in
DynamoDB
Uploads to S3 will trigger
a Lamda function to put
jobs in SQS and
DynamoDB
EFSEC2 instance
worker fleet
Cost-effective Batch Processing with Amazon EC2 Spot |
AWS Compute Blog
https://aws.amazon.com/blogs/compute/cost-effective-
batch-processing-with-amazon-ec2-spot/

45. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 45
JobQueue
submit-job
list-jobs
describe-jobs
cancel-jobs
terminate-jobs
Compute Environment
ECR/Docker
Registry
S3/EFS/NFS
Autoscaling
ECS
Docker Image
JobDefinitio
n
job
AWS Batch

46. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 46
ビッグデータ領域のリファレンスアーキテクチャ
Master
Node (OD
or RI)
S3
EMRFS
Master Fleet
Task
Node
(Spot)
Task
Node
(Spot)
Task
Node
(Spot)
Task Fleet
Instance
Store
EBS
Volume
Instance
Store
EBS
Volume
Instance
Store
EBS
Volume
Core Fleet
HDFS
Instance Fleet
Core Node
(OD or RI)
Core Node
(OD or RI)
Core Node
(OD or RI)
Amazon EMR

47. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 47
コンテナを活用したステートレスウェブアプリケー
ション
Stateless Web
Server (Spot)
Stateless Web
Server (Spot)
Session State Data
Stateless Web Server
(On-Demand)
Elastic Load
Balancing
Amazon EC2
Auto Scaling
ECS
container 1
Stateless Web Server
(On-Demand)
ECS
container 2
ECS
container 1
ECS
container 2
https://github.com/awslabs/ec2-spot-labs/tree/master/workshops/ec2-spot-fleet-web-app

48. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 48
•
•
•
•
•

49. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 49

50. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 50

51. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 51

52. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 52
•
•
•
•
•
•
•

53. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 53

54. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 54
•
•
•
•
•
•
•
•
•
•
•

55. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 55
•
•
•
•
•
•
•
•
•
•
•

56. © 2019, Amazon Web Services, Inc. or its Affiliates. All rights reserved. 56