Masterclass Advanced Usage of the AWS CLI
•
3 gefällt mir•1,284 views
A tutorial from beginner to expert on how to use the AWS CLI.
Empfohlen
Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Andere mochten auch
Andere mochten auch (20)
Ähnlich wie Masterclass Advanced Usage of the AWS CLI
Ähnlich wie Masterclass Advanced Usage of the AWS CLI (20)
Mehr von Danilo Poccia
Mehr von Danilo Poccia (15)
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
Masterclass Advanced Usage of the AWS CLI
- 1. Masterclass Advanced Usage of the AWS CLI Danilo Poccia AWS Technical Evangelist @danilop danilop
- 2. Masterclass A technical deep dive beyond the basics Help educate you on how to get the best from AWS technologies Show you how things work and how to get things done Broaden your knowledge in ~45 minutes
- 3. Crash Course Foundation Advanced Scenarios Intro to the AWS CLI Exploring Key Functionality LookingatAdvancedCLIFeatures
- 4. Novice Proficient AdvancedCrash Course Intro to the AWS CLI
- 5. AWS Command Line Interface Unified tool to manage your AWS services
- 6. MSI (Windows) Bundled (cross platform) pip (cross platform)
- 8. $ aws ec2 describe-‐instances service (command) operation (subcommand)
- 9. $ aws iam list-‐access-‐keys service (command) operation (subcommand)
- 10. --output json { "Places": [ { "City": "Seattle", "State": "WA" }, { "City": "Las Vegas", "State": "NV" } ] }
- 11. --output text PLACES Seattle WA PLACES Las Vegas NV
- 12. --output table -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | SomeOperationName | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ || Places || |+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐+| || City | State || |+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐+| || Seattle | WA || || Las Vegas | NV || |+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐+|
- 13. All Outputs { "Places": [ { "City": "Seattle", "State": "WA" }, { "City": "Las Vegas", "State": "NV" } ] } PLACES Seattle WA PLACES Las Vegas NV -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | SomeOperationName | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ || Places || |+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐+| || City | State || |+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐+| || Seattle | WA || || Las Vegas | NV || |+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐+| JSON Text Table
- 14. Basic AWS CLI Usage DEMO
- 16. Configuration
- 17. aws configure
- 18. aws configure AWS Access Key ID [**ABCD]: AWS Secret Access Key [****************EFGH]: Default region name [us-‐west-‐2]: Default output format [None]:
- 20. aws configure <subcommand> list - List common configuration sources get - Get the value of a single config var set - Set the value of a single config var
- 21. aws configure get region
- 22. aws configure set profile.prod.region us-‐west-‐2
- 23. A profile is a group of configuration values
- 24. aws configure -‐-‐profile prod
- 25. Configuration Files ~/.aws/credentials ~/.aws/config • supported by all AWS SDKs • only contains credentials • Used only by the CLI • Can contain credentials (but not the default behavior)
- 27. aws configure set profile.prod.aws_access_key_id foo ~/.aws/credentials ~/.aws/config
- 28. aws configure set profile.prod.aws_access_key_id foo [prod] aws_access_key_id = foo ~/.aws/credentials ~/.aws/config
- 29. aws configure set profile.prod.aws_secret_access_key bar [prod] aws_access_key_id = foo ~/.aws/credentials ~/.aws/config
- 30. [prod] aws_access_key_id = foo aws_secret_access_key = bar ~/.aws/credentials ~/.aws/config aws configure set profile.prod.aws_secret_access_key bar
- 31. aws configure set profile.prod.region us-‐west-‐2 [prod] aws_access_key_id = foo aws_secret_access_key = bar ~/.aws/credentials ~/.aws/config
- 32. [profile prod] region = us-west-2 [prod] aws_access_key_id = foo aws_secret_access_key = bar ~/.aws/credentials ~/.aws/config aws configure set profile.prod.region us-‐west-‐2
- 33. aws configure set profile.prod.output text [profile prod] region = us-west-2 [prod] aws_access_key_id = foo aws_secret_access_key = bar ~/.aws/credentials ~/.aws/config
- 34. [profile prod] region = us-west-2 output = text [prod] aws_access_key_id = foo aws_secret_access_key = bar ~/.aws/credentials ~/.aws/config aws configure set profile.prod.output text
- 35. create-new-user.sh #!/bin/bash # Create a new user and create a new profile. aws iam create-‐user -‐-‐user-‐name masterclass-‐user credentials=$(aws iam create-‐access-‐key -‐-‐user-‐name masterclass-‐user -‐-‐query 'AccessKey.[AccessKeyId,SecretAccessKey]' -‐-‐output text) access_key_id=$(echo $credentials | cut -‐d' ' -‐f 1) secret_access_key=$(echo $credentials | cut -‐d' ' -‐f 2) aws configure set profile.masterclass.aws_access_key_id "$access_key_id" aws configure set profile.masterclass.secret_access_key "$secret_access_key"
- 36. create-new-user.sh #!/bin/bash # Create a new user and create a new profile. aws iam create-‐user -‐-‐user-‐name masterclass-‐user credentials=$(aws iam create-‐access-‐key -‐-‐user-‐name masterclass-‐user -‐-‐query 'AccessKey.[AccessKeyId,SecretAccessKey]' -‐-‐output text) access_key_id=$(echo $credentials | cut -‐d' ' -‐f 1) secret_access_key=$(echo $credentials | cut -‐d' ' -‐f 2) aws configure set profile.masterclass.aws_access_key_id "$access_key_id" aws configure set profile.masterclass.secret_access_key "$secret_access_key"
- 37. create-new-user.sh #!/bin/bash # Create a new user and create a new profile. aws iam create-‐user -‐-‐user-‐name masterclass-‐user credentials=$(aws iam create-‐access-‐key -‐-‐user-‐name masterclass-‐user -‐-‐query 'AccessKey.[AccessKeyId,SecretAccessKey]' -‐-‐output text) access_key_id=$(echo $credentials | cut -‐d' ' -‐f 1) secret_access_key=$(echo $credentials | cut -‐d' ' -‐f 2) aws configure set profile.masterclass.aws_access_key_id "$access_key_id" aws configure set profile.masterclass.secret_access_key "$secret_access_key"
- 38. create-new-user.sh #!/bin/bash # Create a new user and create a new profile. aws iam create-‐user -‐-‐user-‐name masterclass-‐user credentials=$(aws iam create-‐access-‐key -‐-‐user-‐name masterclass-‐user -‐-‐query 'AccessKey.[AccessKeyId,SecretAccessKey]' -‐-‐output text) access_key_id=$(echo $credentials | cut -‐d' ' -‐f 1) secret_access_key=$(echo $credentials | cut -‐d' ' -‐f 2) aws configure set profile.masterclass.aws_access_key_id "$access_key_id" aws configure set profile.masterclass.secret_access_key "$secret_access_key"
- 39. create-new-user.sh #!/bin/bash # Create a new user and create a new profile. aws iam create-‐user -‐-‐user-‐name masterclass-‐user credentials=$(aws iam create-‐access-‐key -‐-‐user-‐name masterclass-‐user -‐-‐query 'AccessKey.[AccessKeyId,SecretAccessKey]' -‐-‐output text) access_key_id=$(echo $credentials | cut -‐d' ' -‐f 1) secret_access_key=$(echo $credentials | cut -‐d' ' -‐f 2) aws configure set profile.masterclass.aws_access_key_id "$access_key_id" aws configure set profile.masterclass.secret_access_key "$secret_access_key"
- 40. create-new-user.sh #!/bin/bash # Create a new user and create a new profile. aws iam create-‐user -‐-‐user-‐name masterclass-‐user credentials=$(aws iam create-‐access-‐key -‐-‐user-‐name masterclass-‐user -‐-‐query 'AccessKey.[AccessKeyId,SecretAccessKey]' -‐-‐output text) access_key_id=$(echo $credentials | cut -‐d' ' -‐f 1) secret_access_key=$(echo $credentials | cut -‐d' ' -‐f 2) aws configure set profile.masterclass.aws_access_key_id "$access_key_id" aws configure set profile.masterclass.secret_access_key "$secret_access_key"
- 41. Use the aws configure suite of subcommands
- 42. Query
- 43. -‐-‐query (string) A JMESPath query to use in filtering the response data.
- 44. Implementation Details --query Processing aws ec2 ... parse params HTTP request parse response display response format response Retry Pagination
- 45. Implementation Details --query Processing aws ec2 ... parse params HTTP request parse response display response format response Retry Pagination parse response parse response body apply --query
- 46. Implementation Details --query Processing <ListUsersResponse xmlns="..."> <ListUsersResult> <Users> <member> <UserId>userid</UserId> <Path>/</Path> <UserName>james</UserName> <Arn>arn:aws:iam::..:user/james</Arn> <CreateDate>2013-‐03-‐09T23:36:32Z</CreateDate> </member> <Users> </ListUsersResult> <ListUsersResponse>
- 47. Implementation Details --query Processing { "Users": [ { "Arn": "arn:aws:iam::...:user/james", "UserId": "userid", "CreateDate": "2013-‐03-‐09T23:36:32Z", "Path": "/", "UserName": "james" } ] }
- 48. Implementation Details --query Processing -‐-‐query Users[0].[UserName,Path,UserId] { "Users": [ { "Arn": "arn:aws:iam::...:user/james", "UserId": "userid", "CreateDate": "2013-‐03-‐09T23:36:32Z", "Path": "/", "UserName": "james" } ] }
- 49. Implementation Details --query Processing [ [ "james", "/", "id" ], ] -‐-‐query Users[0].[UserName,Path,UserId]
- 50. Implementation Details --query Processing -‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐ | ListUsers | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+ | james | / | userid | +-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+-‐-‐-‐-‐+-‐-‐-‐-‐-‐-‐-‐-‐-‐-‐+
- 51. Implementation Details --query Processing aws ec2 ... parse params HTTP request parse response display response format response Retry Pagination
- 52. http://jmespath.org A Query Language For JSON
- 53. Waiters
- 54. Amazon EC2 Instance State Transitions pending running shutting-down terminated stopping stoppedrebooting Stop Terminate Reboot Terminate Start
- 55. ec2-instance-running.sh #!/bin/bash instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') while [ "$instance_state" != "running" ] do sleep 1 instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') done
- 56. ec2-instance-running.sh #!/bin/bash instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') while [ "$instance_state" != "running" ] do sleep 1 instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') done
- 57. ec2-instance-running.sh #!/bin/bash instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') while [ "$instance_state" != "running" ] do sleep 1 instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') done
- 58. ec2-instance-running.sh #!/bin/bash instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') while [ "$instance_state" != "running" ] do sleep 1 instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') done
- 59. ec2-instance-running.sh #!/bin/bash instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') while [ "$instance_state" != "running" ] do sleep 1 instance_state=$(aws ec2 describe-‐instances -‐-‐instance-‐ids $instance_id -‐-‐query 'Reservations[].Instances[].State.Name') done • No timeouts • Failure states • Hand-written code
- 60. ec2-waiters.sh instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) aws ec2 wait instance-‐running -‐-‐instance-‐ids $instance_id
- 61. ec2-waiters.sh instance_id=$(aws ec2 run-‐instances -‐-‐image-‐id ami-‐12345 -‐-‐query Reservations[].Instances[].InstanceId -‐-‐output text) aws ec2 wait instance-‐running -‐-‐instance-‐ids $instance_id subcommand waiter name describe-instances options
- 62. Novice Proficient AdvancedAdvanced Scenarios Looking at advanced AWS CLI features
- 63. Templates
- 64. The AWS CLI is data driven
- 65. Implementation Details JSON Models "RunInstancesRequest":{ "type":"structure", "required":[ "ImageId", "MinCount", "MaxCount" ], "members":{ "ImageId":{"shape":"String"}, "MinCount":{"shape":"Integer"}, "MaxCount":{"shape":"Integer"}, "KeyName":{"shape":"String"}, "SecurityGroups":{ "shape":"SecurityGroupStringList", "locationName":"SecurityGroup" },
- 66. Implementation Details JSON Models "RunInstancesRequest":{ "type":"structure", "required":[ "ImageId", "MinCount", "MaxCount" ], "members":{ "ImageId":{"shape":"String"}, "MinCount":{"shape":"Integer"}, "MaxCount":{"shape":"Integer"}, "KeyName":{"shape":"String"}, "SecurityGroups":{ "shape":"SecurityGroupStringList", "locationName":"SecurityGroup" }, -‐-‐image-‐id -‐-‐min-‐count -‐-‐max-‐count -‐-‐key-‐name -‐-‐security-‐groups
- 67. Implementation Details JSON Models "RunInstancesRequest":{ "type":"structure", "required":[ "ImageId", "MinCount", "MaxCount" ], "members":{ "ImageId":{"shape":"String"}, "MinCount":{"shape":"Integer"}, "MaxCount":{"shape":"Integer"}, "KeyName":{"shape":"String"}, "SecurityGroups":{ "shape":"SecurityGroupStringList", "locationName":"SecurityGroup" }, { "ImageId": "ami-‐12345", "MinCount": 1, "MaxCount": 1, "KeyName": "id_rsa", "SecurityGroups": ["SSH", "web"], } arguments.json
- 68. aws ec2 run-‐instances -‐-‐cli-‐input-‐json file://arguments.json
- 69. What else can we do?
- 70. aws ec2 run-‐instances -‐-‐generate-‐cli-‐skeleton
- 71. Creating and using JSON templates DEMO
- 73. export AWS_ACCESS_KEY_ID=... ~/.aws/credentials ~/.aws/configCredentialLocator Amazon EC2 Instance Metadata
- 74. CredentialLocator export AWS_ACCESS_KEY_ID=... ~/.aws/credentials ~/.aws/config Amazon EC2 Instance Metadata
- 75. CredentialLocator export AWS_ACCESS_KEY_ID=... ~/.aws/credentials ~/.aws/config Amazon EC2 Instance Metadata
- 76. CredentialLocator export AWS_ACCESS_KEY_ID=... ~/.aws/credentials ~/.aws/config Amazon EC2 Instance Metadata
- 77. CredentialLocator export AWS_ACCESS_KEY_ID=... ~/.aws/credentials ~/.aws/config Amazon EC2 Instance Metadata
- 78. export AWS_ACCESS_KEY_ID=... ~/.aws/credentials ~/.aws/config Amazon EC2 Instance Metadata CredentialLocator AssumeRole
- 79. AWS Identity and Access Management (IAM) roles Delegate access to AWS resources using
- 85. configure-role.sh aws configure set profile.prodrole.role_arn arn:aws:iam... aws configure set profile.prodrole.source_profile dev
- 86. ~/.aws/credentials ~/.aws/config [dev] aws_access_key_id = foo aws_secret_access_key = bar [profile prodrole] role_arn = arn:aws:iam source_profile = dev
- 87. [dev] aws_access_key_id = foo aws_secret_access_key = bar [profile prodrole] role_arn = arn:aws:iam source_profile = dev ~/.aws/credentials ~/.aws/config
- 89. aws s3 cp
- 90. We want to avoid disk
- 91. aws s3 cp -‐ s3://bucket/key
- 92. aws s3 cp s3://bucket/key -‐
- 93. Compress
- 94. s3-compress.sh aws s3 cp s3://bucket/key -‐ | gzip | aws s3 cp -‐ s3://bucket/key.gz
- 97. Mapping one output to N AWS calls
- 98. Mapping one output to N AWS calls $ aws iam list… [ "a", "b", ”c", "d", "e" ] $ aws iam … -‐-‐foo <here> $ aws iam … -‐-‐foo <here> $ aws iam … -‐-‐foo <here> $ aws iam … -‐-‐foo <here> $ aws iam … -‐-‐foo <here>
- 99. Mapping one output to N AWS calls for name in $(aws iam list-‐users -‐-‐query "Users[].[UserName]" -‐-‐output text); do aws iam delete-‐user -‐-‐user-‐name "$name" done
- 100. Mapping one output to N AWS calls for name in $(aws iam list-‐users -‐-‐query "Users[].[UserName]" -‐-‐output text); do aws iam delete-‐user -‐-‐user-‐name "$name" done User1 User2 User3 User4 User5
- 101. Mapping one output to N AWS calls for name in $(aws iam list-‐users -‐-‐query "Users[].[UserName]" -‐-‐output text); do aws iam delete-‐user -‐-‐user-‐name "$name" done
- 102. Mapping one output to N AWS calls aws iam list-‐users -‐-‐query "Users[].[UserName]" -‐-‐output text | xargs -‐I {} -‐P 10 aws iam delete-‐user -‐-‐user-‐name "{}"
- 103. Mapping one output to N AWS calls aws iam list-‐users -‐-‐query "Users[].[UserName]" -‐-‐output text | xargs -‐I {} -‐P 10 aws iam delete-‐user -‐-‐user-‐name "{}"
- 104. Mapping one output to N AWS calls aws iam list-‐users -‐-‐query "Users[].[UserName]" -‐-‐output text | xargs -‐I {} -‐P 10 aws iam delete-‐user -‐-‐user-‐name "{}" Parallel
- 105. Mapping one output to N AWS calls -‐+= 72730 james -‐bash -‐+-‐ 76343 james xargs -‐I {} –P 9 aws iam delete-‐user -‐-‐user-‐name {} |-‐-‐-‐ 76348 james aws iam delete-‐user -‐-‐user-‐name user1 |-‐-‐-‐ 76349 james aws iam delete-‐user -‐-‐user-‐name user2 |-‐-‐-‐ 76350 james aws iam delete-‐user -‐-‐user-‐name user3 |-‐-‐-‐ 76351 james aws iam delete-‐user -‐-‐user-‐name user4 |-‐-‐-‐ 76352 james aws iam delete-‐user -‐-‐user-‐name user5 |-‐-‐-‐ 76353 james aws iam delete-‐user -‐-‐user-‐name user6 |-‐-‐-‐ 76354 james aws iam delete-‐user -‐-‐user-‐name user7 |-‐-‐-‐ 76355 james aws iam delete-‐user -‐-‐user-‐name user8 -‐-‐-‐ 76356 james aws iam delete-‐user -‐-‐user-‐name user9
- 106. Mapping one output to N AWS calls • Use a for loop for functions • If you’re using xargs, use -‐I {} • Use xargs –P N parallel execution • Use “.[UserName]” instead of “.UserName” to get newline separated output.. • This works because nothing is written to stdout on error
- 107. Wrapping Up • Configuration • Waiters • Query • Templates • Credential Providers • Amazon S3 Streaming
- 108. For More Information • https://github.com/aws/aws-cli • http://docs.aws.amazon.com/cli/latest/userguide/ • http://docs.aws.amazon.com/cli/latest/reference/ • https://forums.aws.amazon.com/forum.jspa?forumID=150 • http://jmespath.org/
- 109. Masterclass Advanced Usage of the AWS CLI Danilo Poccia AWS Technical Evangelist @danilop danilop