You don't deploy a single microservice. The journey to microservice architecture involves more than how code is written or applications are packaged. It's about creating an interconnected ecosystem that keeps things running. Infrastructure and tools have only grown in importance as microservices have emerged as a dominant architecture pattern. Deploying, scaling, and monitoring are more important for microservices than they ever were before. Attendees will leave this session knowing the basic infrastructure and tooling needs for microservices to be successful.

1. No Microservice
Is An Island
Michele Titolo
Lead Software Engineer @ Capital One

2. Microservices Are A
Distributed System

5. Something
WILL PROBABLY
GO
WRONG

6. How Do You Figure
Out What’s Wrong?

7. How Do You Deploy
A Fix?

8. New Challenges

9. Microservices Are
More Than
Application Size

10. Failure Points++

11. Microservices Need
An Ecosystem

12. Apps Must
Contribute

13. Infrastructure Must
Contribute

14. Otherwise You’ll
Play Catch Up

15. Three Biggest
Challenges

16. Deployment

17. Scaling

18. Debugging

19. Create A Foundation

20. Deploying
Microservices

21. Characteristics

22. Small Releases

23. Frequent Releases

24. Consistent Releases

25. Limit Special Snowflakes

26. Top-Notch Tooling

27. Automate Every
Aspect

28. 🙅

29. ✋

30. Strategies

31. Staged Deployments

32. Canary

33. R
O
U
T
E
R

34. Blue/Green
Red/Black

35. R
O
U
T
E
R

36. R
O
U
T
E
R

37. R
O
U
T
E
R

38. Automatic Rollbacks

39. How Do You Know A
Deployment Is
Successful?

40. What Features Or
Tools Need To Exist?

41. App

42. - Tests Pass
- Health Check
- Dependency/Secret
Injection

43. Robust Tests

44. Unit And Integration

45. Frequent Deployments
➡
Frequent Testing

46. Standardized Health
Check

47. Consume Dependencies
And Secrets

48. Dependencies i.e.
URL to database

49. Secrets i.e.
username/password to
DB

50. - Tests Pass
- Health Check
- Dependency/Secret
Injection

51. System

52. Aggregate Health Checks

53. Show System State

54. Successful Deployment?

55. Define “Successful”

56. Proactive

57. Report

58. Bots

59. Recap

60. Once a Deployment
Is Finished,
Everything’s Done?

61. Distributed Systems
Are Constantly
Changing

62. Scaling Microservices

63. App Has More Or
Less Load

64. Health Tracked

65. - RAM
- CPU
- Latency
- Requests per second

66. Custom Metrics

67. How Do
Microservices Scale?

68. Automation

69. Smart System

70. Trigger Scaling

71. Scale Up
Scale Down

72. Scale Up
Scale Down

73. Deploy New Instances

74. Monitor Health Checks

75. Send Traffic To New
Instances

76. Scale Up
Scale Down

77. Instances Aren’t Being
Used

78. Stop Routing Traffic

79. Gracefully Shutdown

80. Disconnect

81. Report Progress

82. Ultimately Get Removed

83. Routing Traffic?

84. Infrastructure++

85. Load Balancer
Service Discovery

86. Load Balancer
Service Discovery

87. All Cloud Providers Have
One Built-In

88. Attach To Scaling Group

89. New Instances Registered

90. Conversely, Instances
Going Away Are Removed

91. Load Balancer
Service Discovery

92. Routes To Healthy
Instances

93. Apps Register

94. Route Via Convention

95. A B
Service
Discovery
GET /b/foo GET /foo

96. A 🤢
Service
Discovery
GET /b/foo
🚫

97. Knows Health, Can
Trigger Scaling
Automation

98. Scaling Only Solves
So Many Problems

99. Debugging
Microservices

100. First You Need To
Know There’s A
Problem

101. How?

102. Alerts

103. What Qualifies As A
Problem?

104. Varies Per App

105. RAM, CPU, Latency

106. Less Obvious, Too

107. Key Performance
Indicator

108. Alerts Are For Known
Issues

109. Dashboards

110. How Do You Debug
An Issue?

111. First: Check Problematic
App

112. Pretend Ssh Doesn’t Exist

113. Next Best Thing:
Logs

114. Log Collection And
Aggregation

115. Configurable Log Levels

116. Inspect Logs

117. Non-Trivial Issue
➡
Isolate

118. Avoid Cascading
Failures

119. How Do You Isolate
A Service?

120. Identify
Isolate

121. Identify
Isolate

122. Q: What Parts Of The
System Depend On
Another?

123. Request Tracing

125. Unique Request Ids

126. Best As Overlay

127. Opentracing

128. Add Header Manually

129. Identify
Isolate

130. Circuit Breaking

131. Let App Recover

132. App Or System

133. Fail Gracefully

134. Deploy A Fix

135. Slowly Start Sending
Traffic

136. Can Be Done
Automatically

137. Built-In Load Balancers
Service Discovery

138. Otherwise Update
Manually

139. Everything Returns
To Normal

140. What If The Problem
Isn’t One Of Your
Apps?

141. Do You Have An
External Dependency?

142. No Access To Logs

143. No Access To Source

144. You Can’t Deploy A Fix

145. Fewer Tools

146. Damage Control

147. How Limit Impact?

148. Status Page

149. Monitor The Status Page

150. If Status Page Shows No
Issue, Raise It

151. Keeping Your System
Healthy

152. What’s Inside Your
Control?

153. Can’t See Logs

154. Request Tracing

155. Circuit Breaking

156. Degrade Gracefully

157. Sometimes
Infrastructure Fails

158. Ex: S3 Outage In 2017

159. Circuit Breaking

160. To Recap

161. Debugging Internal
Apps

162. Log, Trace, Circuit Break

163. Debugging External
Apps

164. Trace And Circuit Break

165. One Thing All Of
These Have In
Common…

166. Metrics

167. Monitoring

168. Gain Insights Into
Distributed System

169. Not Free

170. Apps Updated

171. Health Checks

172. Infrastructure
Updated

173. Monitor Health
Checks

174. Gather Information

175. Create Confidence
That System Is
Healthy

176. Creating An
Ecosystem For
Microservices Is
Doable

177. Requires More
Metrics

178. Requires Smarter
Infrastructure

179. Infrastructure won’t evolve on its
own; people need to change their
behavior and fundamentally think
of what it takes to run an
application a different way.
Nova, Kris and Garrison, Justin (2017).
Cloud Native Infrastructure. O’Reilley.

180. Build Software
To
Manage Software

181. Build Software To
Support A Healthy
Ecosystem

182. Thank You!
@micheletitolo

183. • https://unsplash.com/photos/-lp8sTmF9HA
• https://unsplash.com/photos/Rfflri94rs8
• https://unsplash.com/photos/
B87VRCWmpqU
• https://unsplash.com/photos/aLDuLRjZknE
• https://unsplash.com/photos/xal6tVZRCGc
• https://unsplash.com/photos/
VRAxb_gNjQQ
Photo Credits