Owain will talk about the journey JustGiving.com have gone through to get to Continuous delivery on their Windows environment. He will talk about what they did, how they did it and lessons learned along the way Video: https://www.youtube.com/watch?v=MVXaR6oEK60 Join DevOps Exchange London here: http://www.meetup.com/DevOps-Exchange-London Follow DOXLON on twitter http://www.twitter.com/doxlon

1. Continuous Delivery of Windows
microservices in the cloud
@owainperry
http://www.slideshare.net/perryofpeek/continuous-delivery-of-windows-micro-services-in-the-cloud

2. What?
● The journey of the last 18 months,
Justgiving has taken to ship more in less
time
● How we have implemented supporting
infrastructure around micro services
● Where we went wrong

3. ● Raise Money for Charities
● Turnover ~£400 Million a year
● Process UK gift aid
● Text Giving
● Crowd funding
● ~1000 nodes

4. Stephen Sutton

5. Monolith legacy
● High developer pain
● Branching
● 2 week release cycles
● Long build times
● Long test times
● Massive git repository
● Legacy source code patterns (bin files)
● etc…...

6. Micro services

7. JustGiving micro service?
● It's a small service!
– Usually a component makes up part of a
business domain
– Has one purpose
– Is simple to reason about
– Reality - various levels of granularity

8. Where we went next
● New stuff:
– built using micro services
– Hosted in the cloud
– New pipelines for deployment
● Monolith stuff:
– Chop out choice areas, bite size chunks at a time
– Build a simpler pipeline
– Simpler branching strategy

9. Infrastructure principles
● “Very” Available
● Repeatable through automation
● Development define and build their
infrastructure
● Keep environments isolated
● Be pragmatic….

10. Train track automation
● Development - self service infrastructure
– Fast to spin up infrastructure
– Less Ops bottleneck
● Reduce the need for AWS keys
● Consistency benefits:
– Service architecture
– Names and name conventions
– Tags
– Security groups
– Scripting

11. Types of service
● 1. Windows service
Either:
– Self hosted nancyfx web server
● Application
● Health endpoint
– Message based service
● Health endpoint
● 2. IIS hosted application
– Health endpoint

12. Core tools
● AWS
● Artifactory – package repository
● Chef – open source v11
● GO CD
● Team City
● Chocolatey

13. Auto scale groupAuto scale group
Micro service hosting
architecture
● Varnish is the load balancer
Elastic Load BalancerElastic Load Balancer
Varnish
Node 1 Node 2 Node 1Node 2

14. CD of micro services
Each micro service has its own
● Git repository
● GO Pipeline
● Team City build
● Chef cookbook
● Tests
● AWS Infrastructure
● Operations run book
● Health check endpoint - /status/health
● Blue / Green Deployment

15. Steps for a new micro
service
● Think of a name
● Create a git repository
● Open up micro service template in visual studio
– Rename a few things
● Teamcity build (copy n paste)
● Go pipeline (copy n paste)
● Define Infrastructure via json

16. The pipeline
Teamcity
Git
Chef
(cookbook)
Go CD
Artifactory
(choco pkg)
Dev Staging Prod Sandbox

17. Versions
● Application chocolatey package
● Cookbook

18. AWS infrastructure
● Treat AWS as another resource
● Heavy Weight Resource Providers for AWS
● Light weight resource Providers to abstract the HWRP
● json data_bags based infrastructure
● Infrastructure builder node executes chef client for
infrastructure
● “Most” of our infrastructure is built this way, not just
business application services

19. LWRP – data bag - json
{
"id": "dev",
"type": "asg-lcf",
"region": "eu-west-1",
"availability_zones": ["eu-west-1a" ],
"vpc": "DEV-VPC",
"routetable": "RTB-DEV-PRIVATE",
"ami": "JG-UBUNTU12-04-V5",
"instance_type": "m3.medium",
"security_groups": ["********" ],
"iam_role": "instance_default",
"launch_configuration_name": "LCF-DEV-SENSU",
"aws_autoscale_group_name": "ASG-DEV-SENSU",
"health_check_type": "EC2",
"health_check_grace_period": 300,
"asg_min_size": 3,
"asg_max_size": 3,
"asg_desired_capacity": 3,
"asg_default_cooldown": 200,
"subnets": [
{
"name": "SUB-DEV-MONITORING-ZA",

20. Micro service GO pipeline

21. Micro service cookbook
● Chef is used to configure the node
● Chef is used to deploy the application
● “deployment” - is just version constraint in a
chef environment

22. Deployment
● Set the version constraint for the inactive
environment
● Chef search all nodes
● For each node
{ remote in an kick off chef client scheduled task}
● Switch active environment
● Poll for nodes to be on correct version.
● Post deployment tests

23. Consistency in
architecture
● Name conventions
● Build scripts
● Packages
● Supports
● Monitoring
● Refactoring
● Comprehension

24. Bootup
● Chef is prebaked into AMI (faster)
● Instance userdata script contains the bootup
script (inc runlist)
● Auto scale groups used to instantiate new
instances

25. Automated change
● Change log automated into Jira for releases
– Staging and production
● Change log automated for cookbook
releases
● Platform to include more over time.

26. Automated change log

27. Information

28. What we didn't get right...
– Zombies….
– Chocolatey (exe / bat)
– AWS reboot
– Cost control
– Abstraction of infrastructure json
– Chef search to configure load balancer
– Not automating our base AMI's from the outset

29. Future
● Rolling instance recycling – daily / weekly?
● Bake images in production
● Ohai for sensu subscriptions
● Service discovery rather than chef search
● Application secret security
● Code security scanning in the pipelines
● Infrastructure penetration testing
● Infrastructure security checks
● Packer to build base ami's

30. So….
– Micro services – has helped productivity
– AWS automation in hands of developers
– Simple json defined infrastructure
– Chef is core at the centre
– We have made mistakes
– But it's been a worthwhile journey so far.

31. Questions?
@owainperry
http://www.slideshare.net/perryofpeek/continuous-delivery-of-windows-micro-services-in-the-cloud

32. Additional

33. Chef resources
● Elastic Load Balancer
● Auto scale groups
● Security groups
● IAM roles
● Launch configurations
● DNS
● Subnets
● Ec2 instances
● Route tables

34. HWRP example
aws_subnet 'subtest2' do
accesskey accesskey
secretkey secretkey
region “eu-west-1”
vpc “DEV-VPC”
azone 'eu-west-1a'
cidr '10.20.30.0/24'
routetable 'RTB-STG-PRIVATE'
tags [ { "a" => "1"},{"c" => "2" }]
action :create
end

35. Pipelines for chef
cookbooks

36. Chocolatey
● Rebuild all packages for internal hosting.
– Don't want a dependency on 3rd party download
– Control versions of software in use
● Common pattern for git and builds
– Take ~10 mins to repackge
● ( This is a bit of a chore )