Applications running in a typical data center are static entities. But applications aren't static in the cloud. Dynamic scaling and resource allocation is the norm in AWS. Technologies such as Amazon EC2, AWS Lambda, and Auto Scaling provide flexibility but can add complexity to the enterprise application environment. New Relic helps manage that complexity to give the benefits of the cloud without sacrificing simplicity. In this session, we discuss some of the best practices we’ve learned working with New Relic customers on how to manage applications running in this environment and take advantage of the dynamic nature of the cloud to give you additional insights into your application performance.

1. © 2016, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Andy Wetzel, Director of Industry Solutions at New Relic, Inc.
Donald J. Patti, Program Manager & ScrumMaster, Telesis
20 June 2016
Monitoring Performance of Enterprise
Applications on AWS: Life’s Too Short…
... for Cloud without Analytics Monitoring the Dynamic Nature of Cloud Computing

2. Safe Harbor
This document and the information herein (including any information that may be incorporated by reference) is provided for
informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc.
(“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is
proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a
historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future
events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the
statement will include words such as “believes,” “anticipates,”, “expects” or words of similar import.
Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof,
and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting
business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success
of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs
and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results,
performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further
information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to
time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at http://ir.newrelic.com or the
SEC’s website at www.sec.gov.
New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New
Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

3. Who are We?
Specialize in:
Testing
Monitoring
Performance Optimization
22 years in industry
10 in Mercury Interactive / HP
2 in New Relic
Did DevOps when it was called “Lean Manufacturing”
@adwetzel andywetzel

4. Who are We?
Specialize in:
Agile w/Lean
Software Product Development
Coaching & Consulting
20+ years in industry
CSM©/CSP© (Scrum Alliance)
SAFe Agilist© (Scaled Agile Academy)
Lean Six Sigma Black Belt/CSSGB© (ASQ)
Project Management Professional© (PMI)
Balanced Scorecard Professional© (BSI)
donald.patti@telesishq.com

5. Avoiding the “Capitol Hill” moment
You have
to testify
If app launch /
performance is bad

6. We want better apps faster
Better
Data Center
Dynamic
Cloud

7. Cloud as a Better Data Center

8. Better Data Center
Resources
are allocated to
uses, just like in
a data center

9. Better Data Center
Resources
are allocated to
uses, just like in
a data center
Provisioning
process is faster

10. Better Data Center
Resources
are allocated to
uses, just like in
a data center
Lifetime of
components is
relatively long
Provisioning
process is faster

11. Better Data Center
Resources
are allocated to
uses, just like in
a data center
Lifetime of
components is
relatively long
Provisioning
process is faster
Capacity planning
is still important
and still applies

12. Why use a “Better Data Center”?
ComplianceImprove
Application
Availability
(redundancy)
Add new
Capacity
(faster)

13. Who is impacted?
Better Data Center
Faster Application
Launch / Deploy=
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development

14. How do I monitor it?
Similar to monitoring any other data center…

15. Monitoring an application
Typical Server / EC2 Instance
• Application &
Application Microservices
• Server OS
• Hardware (virtual)
EC2 Instance
Application &
Application
Microservices
Server OS
Server (Virtual)
Hardware

16. AWS Monitoring
CloudWatch
• Monitors:
– EC2 instance
– Virtualization
– Hardware
– [CPU/Disk/Networking]
• Doesn’t know about:
– Server OS
– Memory/Filesystem
– Processes
– Application
• Latency
• Error rates
• Internal insights
EC2 Instance
Application &
Application
Microservices
Server OS
Server (Virtual)
Hardware
Amazon
CloudWatch
AWS MANAGEMENT
CONSOLE

17. New Relic Monitoring
New Relic
• Monitors (Server):
– How O.S. is performing
– Processes
– Hardware
• Monitors (Application):
– App health
– App performance
– Microservices
• Doesn’t know
– Virtualization
EC2 Instance
Application &
Application
Microservices
New Relic
Application
Monitoring
Server OS
Server (Virtual)
Hardware
New Relic
Server
Monitoring
Amazon
CloudWatch
AWS MANAGEMENT
CONSOLE
DASHBOARDS

18. AWS  New Relic work together
EC2 Instance
Application &
Application
Microservices
New Relic
Application
Monitoring
Server OS
Server (Virtual)
Hardware
New Relic
Server
Monitoring
Amazon
CloudWatch
AWS MANAGEMENT
CONSOLE
DASHBOARDS
AWS / CloudWatch
• Visibility into virtualization
• CPU / Disk / Networking
New Relic
• CPU / Disk / Networking
• Memory / Filesystem
• Processes
• Infrastructure components
• Application / Microservices:
– Latency
– Error rates
– App insights CloudWatch
monitors
New Relic
monitors

19. The certify.SBA.gov Launch

20. About “Certify”
SBA’S BUSINESS CERTIFICATION PLATFORM
Hosted on the
AWS Cloud
FedRAMP
compliant
Replaces legacy
ColdFusion /
Oracle systems
Helps small
businesses to
win government
contracts

21. Preparation for Launch
Traditional load,
performance &
stress testing
Implementation
of New Relic

22. Key milestones
Initial
Launch
Chief Administrator’s
Press Conference

23. How we avoided “The Capitol Hill” moment
Used New Relic to
identify bottlenecks
Proactive real-time monitoring
via New Relic helped us to
better manage go-live

24. The outcome
Successful launch
to the cloud –
infrastructure and
new application –
in only 7 months
Increase in
new certification
applications during
the first month
No measurable
degradation in
performance for
either event
(MRT90 = 3 seconds)
600%0 7

25. A better data center for SBA
FedRAMP
Compliance &
Rapid ATO
Virtual
redundancy via
AWS Platform
Auto-scaling
of cloud
infrastructure

26. Cloud as a Dynamic Tool

27. Cloud as a “Dynamic Tool for Dynamic Apps”
Use only
the resources
you need
Resource
allocation is an
integral part of
your application
architecture
Allocate /
de-allocate
resources on
the fly

28. Dynamic Cloud
Docker containers:
• Quicker / cheaper startup / shutdown
than EC2 instances
• Easy to consume Docker containers
much like you do “processes”
• Perform jobs:
– In a script
– Off of some queue of job requests

29. Dynamic Cloud
100
10K
1M
3.7 M
83 days 333 days
Docker Container Age
(Count vs. Hours)

30. Dynamic Cloud
950,000
11% under one minute
Docker Container Age
(by Minute Under and Hour)

31. EC2 Auto Scaling
Dynamic Cloud Technologies
Mobile / IoT
Dynamic routing
Load balancing
Data transport
Queues and notifications
Docker
Dynamic Cloud is about scaling

32. Who Uses It?
Better Data
Center
Can I scale my server fleet?
Can apps run anywhere?
How do they perform in the cloud?
Operations
Data center is
a data center…
Development
Dynamic
Cloud
What is a container?
What is a Lambda?
It was just here, where did it go?
Cloud architecture is integral
to the application architecture
Developers deeply involved
in cloud activities

33. How do I monitor the Dynamic Cloud?
Dynamic Cloud has unique monitoring requirements…

34. Responsibility (Dynamic Cloud)
Server OS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
Responsible for the parts you care about
• Application &
Application Microservices
Let cloud manage rest
• Infrastructure
• Allocation / Provisioning
• Scaling
Server (Virtual)
Hardware

35. AWS Infrastructure  New Relic work together
Server OS
Server (Virtual)
Hardware
AWS CONSOLE
DASHBOARDS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
Amazon
CloudWatch

36. AWS Infrastructure  New Relic work together
Server OS
Server (Virtual)
Hardware
AWS CONSOLE
DASHBOARDS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
Amazon
CloudWatch
CloudWatch
& AWS monitoring

37. AWS Infrastructure  New Relic work together
Server OS
Server (Virtual)
Hardware
AWS CONSOLE
DASHBOARDS
Application &
Application
Microservices
Application &
Application
Microservices
Application &
Application
Microservices
Provisioning
CloudWatch
& AWS monitoring
New Relic
monitors
Amazon
CloudWatch

38. Monitoring the Dynamic Cloud
Very different than monitoring traditional Data Center components
Monitor the
Cloud Components
themselves
Monitor the
lifecycle of the
Cloud Components

39. Dynamic Cloud
Amazon EC2
Server running
application / processes
Docker container
Process running
a command
Amazon Lambda
Function performing a
task or operation

40. Monitoring the Dynamic Cloud
Your world is changing fast…

41. Monitoring the Dynamic Cloud
… your monitoring tools need to help you keep up
Your world is changing fast…

42. Change is speeding up
Traditional
Data Center
Dynamic Cloud enables better applications faster.
The rate of change is increasing…
Good

43. Change is speeding up
Traditional
Data Center
Cloud
Data Center
Dynamic Cloud enables better applications faster.
The rate of change is increasing…
Good Better

44. Change is speeding up
Traditional
Data Center
Cloud
Data Center
Dynamic
Cloud
Dynamic Cloud enables better applications faster.
The rate of change is increasing…
Good Better Best

45. Lessons learned
Government
can be nimble

46. Lessons learned
Government
can be nimble
Avoiding
“The Capitol Hill”
moment

47. Lessons learned
Government
can be nimble
Avoiding
“The Capitol Hill”
moment
Leverage
data driven
decision-making

48. Lessons learned
Government
can be nimble
Avoiding
“The Capitol Hill”
moment
Leverage
data driven
decision-making
More gains
possible via
Dynamic Cloud

49. Lessons learned
Government
can be nimble
Avoiding
“The Capitol Hill”
moment
Leverage
data driven
decision-making
More gains
possible via
Dynamic Cloud
Others
can help

50. Thank You!
Andy Wetzel, Director of Industry Solutions at New Relic, Inc.
Donald J. Patti, Program Manager & ScrumMaster, Telesis