A lot of focus has been placed on securing the cloud, but the cloud can also be used to help secure applications. Find out how the same principles that apply to building cloud scale applications can also be used to deploy test environments in the cloud that support application security testing. Never again fear that your automated security testing, penetration testing, and customer A/B testing will collide. This talk will cover how applications that abide by the 12 Factors (https://12factor.net/) are easier to test. It will also discuss how the extreme flexibility of cloud resources allows easy separation of different types of application testing, ensuring that security tests can be run without interfering with business objectives.

1. Testing 12-Factor Cloud Apps
Phillip Marlow
October 2022
Approved for Public Release; Distribution Unlimited. Case Number 22-3215
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

2. Too Long; Didn’t Listen
 The flexibility and elasticity of cloud services allows better and more automated
testing – if applications are designed to take advantage of it
 Designing applications and services for the cloud provides increased testability
and security
 This makes applications more resilient against technical and environmental
failures as well as attacks
 It also improves the organization’s ability to deliver on their mission
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

3. > iam list-user-tags
 Cloud Engineer:
Designed and built both AWS and Azure environments for
large teams
 Systems Engineer:
Focus on the overall system and process to deliver the system
 Developer:
10+ years
 DevOps Engineer:
Automating build, test, deployment, and monitoring
 Security Engineer:
GSE #263, SANS Master’s Degree
 Hacker:
Speaker at DEF CON Cloud Village
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

4. Typical Application Promotion Process
Development.env Test.env Production.env
Application v1.0 Application v1.0
Application v1.0
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

5. Application Development Process
Development Test Production
Application v1.0-
katherine
Application v1.0-jenny
Application v1.1
Application v1.0-
katherine
Application v1.0-jenny
Application v1.1 Application v1.1
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

6. Mature Application Deployment Process
Development
Test
Production
Application v1.0-
katherine
Application v1.0-jenny
Application v1.1
Application v1.0-
katherine
Application v1.0-jenny
Application v1.1 –
instance 1
Application v1.1
Application v1.1 –
instance N
Test
App2 v2.1
App2 v2.1 App2 v2.1
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

7. The Big Problem
 Can multiple versions of an application be hosted in each environment?
 This design creates choke points on work at each environment
 Especially problematic for the test environment which may be shared by many users
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

8. Designing for the Cloud is Better
 The Twelve-Factor App, developed by Adam
Wiggins & Heroku
 https://12factor.net/
Apps that:
 Use declarative formats for setup automation,
to minimize time and cost for new developers
joining the project;
 Have a clean contract with the underlying
operating system, offering maximum
portability between execution environments;
 Are suitable for deployment on modern cloud
platforms, obviating the need for servers and
systems administration;
 Minimize divergence between development
and production, enabling continuous
deployment for maximum agility;
 And can scale without significant changes to
tooling, architecture, or development
practices.
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

9. Twelve-Factor Alternatives
 Microservices Reference Architecture from NGINX
 https://www.nginx.com/blog/introducing-the-nginx-microservices-
reference-architecture/
 Beyond the Twelve-Factor App by Kevin Hoffman
 https://www.oreilly.com/library/view/beyond-the-twelve-
factor/9781492042631/
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

10. I. Codebase
 Partially solves the big problem of multiple deploys in an environment
One codebase tracked in revision control, many deploys
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

11. II. Dependencies
 No reliance on dependencies installed in the deployment environment
makes it possible to scale the number of deployments and environments
as needed
Explicitly declare and isolate dependencies
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

12. III. Config
 Separating environment specific configuration allows consistent and
independent deployments
 It also ensures that no changes need to be made to the system between
environments, which could potentially compromise the integrity of
previously run tests
Store config in the environment
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

13. IV. Backing Services
 By treating backing services, such as databases or APIs, as attached
resources, we ensure the application is loosely coupled to those resources
 This enforcement of loose coupling of components makes testing those
components easier
 While this may increase the number of integration tests, this approach
ensures we have a thorough understanding of those integration points
making developing integration tests easier
Treat backing services as attached resources
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

14. V. Build, Release, Run
 Testing can be run more frequently when build is separated from run
 Ensures no code changes are possible at runtime, so earlier tests remain
valid in the production environment
Strictly separate build and run stages
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

15. X. Dev/Prod Parity
 Independent tests results are applicable to the final deployment
Keep development, staging, and production as similar as possible
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

16. Wins
 Tests can be run simultaneously AND independently
 It’s easy to add another instance of an app or a whole environment
 Applications are designed for easy integration with other tools, including test
orchestrators and cloud security platforms
 Common operational patterns can be used to make the application more resilient
against a variety of failures and attacks
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.

17. Phillip Marlow
@wolramp
linkedin.com/in/phillipmarlow
Thank You!
pmarlow@mitre.org
© 2022 THE MITRE CORPORATION. ALL RIGHTS RESERVED.