26. Continuous Deployment
â˘âŻ Deploy new software quickly
â˘âŻ At IMVU time from check-in to production = 20 minutes
â˘âŻ Tell a good change from a bad change (quickly)
â˘âŻ Revert a bad change quickly
â˘âŻ Work in small batches
â˘âŻ At IMVU, a large batch = 3 days worth of work
â˘âŻ Break large projects down into small batches
27. Cluster Immune System
What it looks like to ship one piece of code to production:
â˘âŻ Run tests locally (SimpleTest, Selenium)
Everyone has a complete sandbox
oâŻ
â˘âŻ Continuous Integration Server (BuildBot)
o⯠All tests must pass or âshut down the lineâ
Automatic feedback if the team is going too fast
oâŻ
â˘âŻ Incremental deployâŠ
Monitor cluster and business metrics in real-timeâŠ
oâŻ
Reject changes that move metrics out-of-bounds
oâŻ
â˘âŻ Alerting & Predictive monitoring (Nagios)âŠ
Monitor all metrics that stakeholders care aboutâŠ
oâŻ
If any metric goes out-of-bounds, wake somebody upâŠ
oâŻ
Use historical trends to predict acceptable bounds
oâŻ
WhenâŠcustomersâŠseeâŠaâŠfailure:âŠ
Fix the problem for customers
oâŻ
Improve your defenses at each levelâŠ
oâŻ