DevOps | AWS Loft Architecture Week | Wednesday, August 17

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Henrik Johansson, Security Solutions Architect
August 2016
DevOps on AWS:
Improving Infrastructure
Governance on AWS

2. https://secure.flickr.com/photos/mgifford/4525333972
Why are we
here today?

3. Why are we here today?
Using cloud based infrastructure changes how to think about
governing our infrastructure:
 Infrastructure can be provisioned in seconds.. and go
away just as quickly!
 Development teams expect a higher level of flexibility and
self control in interfacing with their infrastructure needs
 Being API driven means that the way people provision and
manage infrastructure in the cloud has changed compared
to on-premises

4. Why are we here today?
That doesn’t mean that our basic governance
needs change:
 We still need to have some ability to drive best
practices/patterns in our organizations
 We need to make sure that we’re able to audit
and track changes to our infrastructure for both
regulation and security purposes
 We need to make sure that we understand
how resources are related and integrated

5. What can we do?
There are a few areas to focus on that can help us accomplish
both the freedom to rapidly provision, manage, and update our
infrastructure while meeting our governance needs:
 Policy as Code
 Infrastructure standardization (via code!)
 Self service environments
 Logging/Auditing/Reacting to infrastructure change

6. Policy as Code builds off of
infrastructure as code practices by
allowing organizations to codify
infrastructure and system configurations
allowing them to monitor and enforce
compliance dynamically and at scale.

7. Infrastructure as Code is a practice
in which infrastructure is provisioned
and managed using code and
software development techniques,
such as version control and
continuous integration.

8. Infrastructure as Code “levels”
AWS Resources
Operating System and Host Configuration
Application Configuration

9. Infrastructure as Code “levels”
AWS Resources
Operating System and Host Configuration
Application Configuration
allOfThis == $Code

10. Browse and launch
AWS ConfigAWS CloudTrail
Use and modify
Users Admin
Putting the AWS Management services together
AWS Service Catalog
Provision with Tags
API calls Configuration checks and
reactions to change
Troubleshoot and Audit

11.  Create templates of your infrastructure
 CloudFormation provisions AWS
resources based on dependency needs
 Version control/replicate/update
templates like code
 Integrates with development, CI/CD,
management tools
AWS
CloudFormation

12. Template CloudFormation Stack
JSON formatted file
Parameter definition
Resource creation
Configuration actions
Configured AWS resources
Comprehensive service support
Service event aware
Customizable
Framework
Stack creation
Stack updates
Error detection and rollback
CloudFormation – Components & Technology

13. Template File
Defining Stack
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Many Stacks & Environments from One Template

14. Template File
Defining Stack
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Use the version
control system of
your choice to
store and track
changes to this
template
Many Stacks & Environments from One Template
Git
Perforce
SVN
…

15. Template File
Defining Stack
Git
Perforce
SVN
…
Dev
Test
Prod
The entire infrastructure can be
represented in an AWS
CloudFormation template.
Use the version
control system of
your choice to
store and track
changes to this
template
Build out multiple
environments, such
as for Development,
Test, Production and
even DR using the
same template
Many Stacks & Environments from One Template

16. CloudFormation example use cases:
Have “full stack” templates that can be used to stand up common
application patterns inside your organization such as a 3-tier application
template that:
 uses Lambda custom resources to look up appropriate VPC
information (VPC ID, Subnets, etc) based on tags
 creates an Elastic Beanstalk environment that supports Multi-AZ,
AutoScaling, CloudWatch Metrics, and Elastic Load Balancing
 contains security controls such as AWS Identity and Access
Management (IAM) roles, profiles, and policies, and Security Groups
 allows the user to specify the language of their application
 allows a user to specify which database they want (SQL or NoSQL)
and then creates the appropriate resource

17. Using Parameters and Conditionals are two key ways
to make a single template much more dynamic:
"Parameters" : {
"Database": {
"Type" : "String",
"Default" : "RDS",
"AllowedValues" : ["RDS", "DynamoDB", "None"],
"Description" : "Database to create. Select None if using an existing database.”
}
},
"Conditions" : {
"CreateRDS" : {"Fn::Equals" : [{"Ref" : "Database"}, "RDS"]},
"CreateDynamoDB" : {"Fn::Equals" : [{"Ref" : "Database"}, "DynamoDB"]},
"CreateNone" : {"Fn::Equals" : [{"Ref" : "Database"}, "None"]}
},
“Resources” : {
”RDSdb01" : {
"Condition" : " CreateRDS ",
"Type" : "AWS::RDS::Instance",

18. https://secure.flickr.com/photos/wscullin/3770015991
Now that we have these
templates, what can we do to
help simplify our developers’
lives further and increase our
infrastructure standardization?

19.  Customized catalogs of products
 Manage products centrally
 Personalized, self-service portal
 Integrate with existing systems
AWS
Service Catalog

20. What is AWS Service Catalog?
AWS Service Catalog allows organizations to create and manage
catalogs of IT services. It enables users to quickly deploy the approved
IT services they need in a self-service manner.
Organizations Developers
Control
Standardization
Governance
Agility
Self-service
Time to market

21. Creates portfolio
Adds constraints
and grant access
1
4
5
Administrator
Portfolio
Users
Browse Products
6Launch ProductsAWS CloudFormation
template
Creates
product3Authors template2
ProductX ProductY ProductZ
7
Deploys
stacks
Events
Events
8
8
Service Catalog
Create custom
services
and grant access
Use a
personalized
portal to find and
launch services

22. Service Catalog use cases:
You can remove the need for developers to understand how all
AWS services work. Treat infrastructure provisioning like buying
components from a retail site:
 provide standardized Service Catalog products around
common internal application frameworks/architectural patterns
 provide common application component products such as
databases, queues, caches, worker tiers, etc
 build logging, monitoring, metrics into these stacks
 leverage service discovery tools when possible
 build in the same best practices across development, staging,
production environments with these provided products

23. We’ve helped solve some of our
developer’s access and
standardization issues, but how can
we now go about auditing changes
to our infrastructure?
https://www.flickr.com/photos/atoach/7623237104

24. AWS
CloudTrail
 Records AWS API calls for your account
 Delivers log files of API calls to S3
 Delivery typically within 15 minutes of API call
 Logs contain detailed information
 Log files can be encrypted and have their
integrity verified by you

25. AWS CloudTrail
CloudTrail can help you achieve many tasks
 Security analysis
 Track changes to AWS resources, for
example VPC security groups and NACLs
 Compliance – log and understand AWS API
call history
 Prove that you did not:
 Use the wrong region
 Use services you don’t want
 Troubleshoot operational issues – quickly
identify the most recent changes to your
environment

26. AWS CloudTrail logs can be delivered cross-account
CloudTrail can help you achieve many tasks
 Accounts can send their trails to a central
account
 Central account can then do analytics
 Central account can:
 Redistribute the trails
 Grant access to the trails
 Filter and reformat Trails (to meet
privacy requirements)

27. CloudTrail

28. CloudTrail – Amazon CloudWatch Logs Integration

29.  Continuous recording
 Inventory of AWS resources
 New & deleted resources
 Configuration change & compliance
notifications
AWS
Config

30. Relationships
Bi-directional map of dependencies
automatically assigned
Change to a resource propagates
to create Configuration Items for
related resources

31. Configuration Item
 All configuration attributes
 Normalized
 Point in time
 Captured on configuration change

32. Component Description Contains
Metadata Information about this configuration
item
Version ID, Configuration item ID,
Time when the configuration item
was captured, State ID indicating
the ordering of the configuration
items of a resource, MD5Hash, etc.
Common Attributes Resource attributes Resource ID, tags, Resource type.
Amazon Resource Name (ARN)
Availability Zone, etc.
Relationships How the resource is related to other
resources associated with the
account
EBS volume vol-1234567 is
attached to an EC2 instance i-
a1b2c3d4
Current Configuration Information returned through a call
to the Describe or List API of the
resource
e.g. for EBS Volume
State of DeleteOnTermination flag
Type of volume. For example, gp2,
io1, or standard
Related Events The AWS CloudTrail events that are
related to the current configuration
of the resource
AWS CloudTrail event ID
Configuration Item

34. Sample Configuration Item
"configurationItemVersion": "1.0",
"configurationItemCaptureTime": "2014…",
"configurationStateID": “….",
"configurationItemStatus": "OK",
"resourceId": "vol-ce676ccc",
"arn": "arn:aws:us-west-………",
"accountId": "12345678910",
"availibilityZone": "us-west-2b",
"resourceType": "AWS::EC2::Volume",
"resourceCreationTime": "2014-02..",
"tags": {},
"relationships": [
{
"resourceId": "i-344c463d",
"resourceType": "AWS::EC2::Instance",
"name": "Attached to Instance"
}
],
"relatedEvents": [
"06c12a39-eb35-11de-ae07-db69edbb1e4",
],
Metadata
Common Attributes
Relationships
Related Events

35. Sample Configuration Item
"configuration": {
"volumeId": "vol-ce676ccc",
"size": 1,
"snapshotId": "",
"availabilityZone": "us-west-2b",
"state": "in-use",
"createTime": "2014-02-……",
"attachments": [
{
"volumeId": "vol-ce676ccc",
"instanceId": "i-344c463d",
"device": "/dev/sdf",
"state": "attached",
"attachTime": "2014-03-",
"deleteOnTermination": false
}
],
"tags": [
{
"tagName": "environment",
"tagValue": "PROD"
Configuration

36. Having this data is one
thing, but how can we
react to change?
https://www.flickr.com/photos/livenature/204420128/

37. CloudTrail – Amazon CloudWatch Logs Integration

38. CloudTrail – Amazon CloudWatch Logs Integration
Trigger a
CloudWatch
Alarm based
on API call!

39.  Check configuration changes
 Pre-built rules provided by AWS
 Custom rules using AWS Lambda
 Continuous assessment
 Dashboard
 Compliance visualization
 Identify offending changes
 GitHub repo: Community sourced rules
AWS
Config Rules

40. NormalizeRecordChanging
Resources
AWS Config & Config Rules
Deliver
Stream
Snapshot (ex. 2014-11-05)
AWS Config
APIs
Store
History
Rules

41. Config Rules

42. Config Rules

43. Config Rules GitHub repository

44. AWS Management Tools Partners

45. CLOUD SECURITY PLATFORM for Start-ups, High-Growth, & Enterprise
Modern Times Require Modern Security.
Vulnerability
Management
Threat Intelligence Compliance
Reporting
Infrastructure
Monitoring
Workloa
d
Insights
Cloud Native. Platform Independent. Fully Integrated.

46. INCREASE VELOCITY OF YOUR SECURITY OPERATIONS
Time to detection: Go from 4 hrs to 4 minutes
GAIN COMPLETE VISIBILITY INTO BEHAVIOR
Know Who, What, Where, When across your entire
environment
CONTINUOUS SECURITY MONITORING & VISIBILITY, IN ONE PLACE
No need to chase down fragmented data points from multiple
tools
ASSURE COMPLIANCE FOR YOU & YOUR CUSTOMERS
Automatically implement effective controls, policies
& procedures to protect data and meet compliance
Why Threat Stack?

47. AWS CloudTrail: Threat Stack Partner Integration
Threat Stack Cloud Security Platform
enables you to:
Automatically monitor and Alert changes to
AWS resources using Cloud Trail logs
 Audit changes to AWS Service against risk
and compliance goals
 Pre-built Rule sets with AWS and
community best practices
 Integrated Notification channels : Slack,
Pager Duty, VictorOps
Additional Integrated Capabilities
 Advanced Host Intrusion Detection
 Software Vulnerability Management
 Threat Intelligence based on IP
Reputation
 Compliance Management and Reporting

48. Threat Stack AWS CloudTrail Monitoring Flow
AWS S3
Designated
Bucket
AWS
CloudTrail
Amazon
SNS
Amazon
SQS
1
2
3
AWS Cloud Trail Events
Processing Threat Stack AWS
Cloud Trail Audit Rules
AWS Console Access
AWS IAM Service
AWS EC2 Service
AWS S3 Service
AWS RDS Service
AWS Route53 Service
…...........................
...............................
................................
6 7 8
Threat Stack
Cloud Trail
Alerts
Alert Notifications
 Audit configuration changes against Pre-built rules provided by Threat Stack
 Create Custom rules using Threat Stack Policy Engine
 Continuous risk assessment for all AWS Infrastructure Changes
 Gain Operational efficiencies via
 At a glance Dashboard visualization to Identify offending changes
 Seamless integration with operational collaboration tools

49. FIN, ACK
We’ve seen a quick run through today of the ways you can
improve your governance on AWS:
 Treat your infrastructure and host configuration as code!
 This lends itself to being able to use services like Service
Catalog to enable self-service in your organization
 Track, trend, and alert on CloudTrail API logs to keep on top
of access to your AWS resources
 Use Config and Config Rules to understand the relationship
between resources and react to policy violations
 Putting all this together is what gives you Policy as Code!

50. Browse and launch
AWS ConfigAWS CloudTrail
Use and modify
Users Admin
Putting the AWS Management services together
AWS Service Catalog
Provision with Tags
API calls Configuration checks and
reactions to change
Troubleshoot and Audit

51. But wait, there’s more!
Resources to learn more:
 More on DevOps: https://aws.amazon.com/devops/
 AWS Management Services: https://aws.amazon.com/products/management/
 AWS CloudFormation
 https://aws.amazon.com/cloudformation/
 AWS Service Catalog
 https://aws.amazon.com/servicecatalog/
 AWS CloudTrail
 https://aws.amazon.com/cloudtrail/
 AWS Config / Config Rules
 https://aws.amazon.com/config/
 GitHub repo: https://github.com/awslabs/aws-config-rules

52. Thank you!
Happy Deploying!