Empfohlen
Weitere ähnliche Inhalte
Was ist angesagt?
Was ist angesagt? (20)
Ähnlich wie The road to continuous deployment (DomCode September 2016)
Ähnlich wie The road to continuous deployment (DomCode September 2016) (20)
Mehr von Michiel Rook
Kürzlich hochgeladen
Kürzlich hochgeladen (20)
The road to continuous deployment (DomCode September 2016)
- 1. THE ROAD TO CONTINUOUS DEPLOYMENT - A CASE STUDY MICHIEL ROOK
- 2. ▸ Java, PHP & Scala developer ▸ Consultant, trainer, speaker ▸ make.io ▸ Dutch Web Alliance ▸ Maintainer of Phing ▸ @michieltcs
- 3. THIS TALK ▸ Background ▸ The approach ▸ Process / standards ▸ Build pipelines ▸ Results & lessons learned
- 5. THE SYSTEM - SAN DIEGO ▸ ... or the Big Ball Of Mud ▸ Large legacy monolith ▸ Generates significant income ▸ Slow & complex ▸ Technical debt
- 6. SAN DIEGO FRONTEND MYSQL DB SAN DIEGO BACKEND LOAD BALANCERS / VARNISH ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK SAN DIEGO FRONTEND SAN DIEGO FRONTEND SAN DIEGO FRONTEND SAN DIEGO BACKEND SAN DIEGO BACKEND SAN DIEGO BACKEND MEMCACHE FTP EXT. SERVICES SOLR
- 7. THE SYSTEM - SAN DIEGO ▸ Infrequent, manual releases ▸ Fragile tests ▸ Low velocity ▸ Frequent outages / bugs / issues ▸ Frustrated team ▸ Low confidence modifying existing code
- 8. GOALS ▸ Reduce issues ▸ Reduce cycle time ▸ Increase productivity ▸ Increase motivation
- 10. APPROACH ▸ Services per domain object (job, jobseeker, ...) ▸ Strangler pattern ▸ Proxy to switch between old/new ▸ Migrate individual pages
- 12. APPROACH ▸ Services behind load balancers ▸ Access legacy db’s ▸ Continuous deployment ▸ Docker containers ▸ Frontends are services
- 13. SAN DIEGO ELASTIC SEARCHLEGACY DB JOB SERVICE RMQ ITBANEN INTERMEDIAIR NATIONALEVACATUREBANK MONGO DB ITBANEN JOBSEEKER SERVICE NVBINTERMEDIAI
- 14. PROCESS
- 15. STARTING OFF ▸ Scrum, 1 week sprints ▸ TDD / BDD ▸ Definition of Done ▸ Team mindset / experience ▸ Focus on value ▸ Replace old features with new (legacy becomes obsolete)
- 17. DEV BUILD / TEST CONTINUOUS INTEGRATION
- 18. DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DELIVERY
- 19. DEV BUILD / TEST ACCEPTANCE PRODUCTION CONTINUOUS DEPLOYMENT
- 20. WHY CONTINUOUS DEPLOYMENT ▸ Small steps ▸ Early feedback ▸ Less overhead ▸ Reduce cycle time ▸ Reduce risk
- 21. ONLY COMMIT TO MASTER
- 22. NO BRANCHES
- 24. PAIR PROGRAMMING
- 25. BOY SCOUT RULE
- 26. QUALITY GATES
- 28. FEATURE TOGGLES, A/B TESTS
- 29. DASHBOARDS
- 30. BUILD PIPELINE
- 32. EVERY COMMIT GOES TO PRODUCTION
- 33. DEFENSE IN DEPTH UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS
- 34. DEFENSE IN DEPTH UNIT TESTS INTEGRATION TESTS ACCEPTANCE UI TESTS public function testJobCannotBeFound() { $jobRepository = $this->prophesize(JobRepository::class); $jobRepository->getById(EXPECTED_JOB_ID) ->shouldBeCalled() ->willReturn(false); $jobService = new JobService($jobRepository->reveal()); $this->assertFalse($jobService->getById(EXPECTED_JOB_ID)); }
- 35. DEFENSE IN DEPTH UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS public function testFindJob() { $expectedJob = $this->loadFixture('active_job.yml'); $actualJob = $this->repository->getById($expectedJob->getId()); self::assertInstanceOf(Job::class, $actualJob); self::assertEquals($expectedJob->getId(), $actualJob->getId()); }
- 36. DEFENSE IN DEPTH UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTSScenario: Link to related job Given a job exists And there are related jobs available When that job is viewed Then a list of related jobs is shown And each related job links to the detail page of the related job
- 37. DEFENSE IN DEPTH UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS
- 38. DEFENSE IN DEPTH MANUAL TESTING? UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI
- 39. CONTINUOUS TESTING UNIT TESTS INTEGRATION TESTS ACCEPTANCE TESTS UI TESTS SMOKE TESTS Cost Speed Exploratory testing Monitoring
- 40. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER
- 41. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER docker pull
- 42. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER docker run
- 43. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER wait_for: port=8080 delay=5 timeout=15
- 44. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER uri: url: http://localhost:8080/_health status_code: 200 timeout: 30
- 45. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg service: name=haproxy state=reloaded
- 46. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER template: src=haproxy.cfg.j2 dest=/etc/haproxy/haproxy.cfg service: name=haproxy state=reloaded
- 47. DEPLOYING PULL IMAGE START NEW CONTAINER WAIT FOR PORT SMOKE TESTS / HEALTH CHECKS ADD NEW CONTAINER TO LB REMOVE OLD CONTAINER FROM LB STOP OLD CONTAINER docker stop docker rm
- 48. PIPELINE AS CODE node { stage 'Run tests' sh "phpunit" sh "behat" stage 'Build docker image' sh "phing build" sh "docker build -t jobservice:${env.BUILD_NUMBER} ." sh "docker push jobservice:${env.BUILD_NUMBER}" stage 'Deploy acceptance' sh "ansible-playbook -e BUILD_NUMBER=${env.BUILD_NUMBER} -i acc deploy.yml" stage 'Deploy production' sh "ansible-playbook -e BUILD_NUMBER=${env.BUILD_NUMBER} -i prod deploy.yml" }
- 49. BUILD PIPELINE
- 50. FEEDBACK!
- 51. RESULTS
- 52. RESULTS ▸ Total build time per service < 10 minutes ▸ Significantly improved page load times ▸ Improved audience stats (time on page, pages per session, session duration, traffic, seo ranking, etc) ▸ Increased confidence and velocity ▸ Experimented with new tech/stacks (angular, jvm, event sourcing) ▸ More fun
- 53. LESSONS LEARNED ▸ Team acceptance ▸ Change is hard ▸ Overhead of weekly sprint; requires discipline ▸ Docker orchestration ▸ Issues with traffic between Amazon <-> on-premise datacenter ▸ Javascript testing
- 54. LESSONS LEARNED ▸ Experience with new tech ▸ Stability of build pipelines ▸ Management/leadership buy-in ▸ Business alignment ▸ Not enough focus on replacing legacy application
- 55. THANK YOU! @michieltcs / michiel@make.io