Deployment Pipeline for Magento Enterprise in the Cloud. The Talk covers the Amazon Cloud Infrastructure; Scaling and Autoscaling in the Cloud, the Deployment Pipeline used to do continuous deployments...
9. The Angry Birds Magento Universe
Magento Store DRM Server
Storefronts Reporting
Products + Mail Service
Ordermgmt
Warehouse Inventory
Giftcards
Shipwire
Manage Warehouse Fullfillment Braintree
Tuesday, April 24, 2012
10. Angrybirds Magento Universe
Sh ipw ire
Magento Store bal
DRM Server
ges glo
M ana
es &
Storefronts Reporting
wa reh ous
Products + Mail Service
Ordermgmt
Warehouse Invetory
inv ent ory
ay for
ind be st w
F ent Giftcards
rF ullf illm
Shipwire o rde
Manage Warehouse Fullfillment Braintree
Tuesday, April 24, 2012
14. Cloud Drawbacks?
Complexity Plan for EC2 Failover
Need to adjust
Plan Security
Application
Lack of Service
Tuesday, April 24, 2012
15. Cloud Drawbacks? ture Req uirem ents
Arch itec over
le Fail
ultip
ailab ility (M
H igh av cing)
balan
Complexity
and Load
Bac kend Plan for EC2 Failover
in
to Adm m Shop-Fro ntend
M agen
ende nt fro
indep depen dent
ient a nd in nage ment
Effic nd tas k ma
back grou Need to adjust
Plan Security ching
In tensi ve Ca Application
n the cloud
agem ent i
Asse t man
Lack of Service
Tuesday, April 24, 2012
19. Architecture
ELB R53 Cloud Front
DNS CDN for Skin
Cloud Front S3 Autoscaling
CDN for Assets Assets Storage Varnish Array
Frontend Array Backend Array Worker Array
Autoscaling
FE FE FE
RDS
mySQL
Elastic Cache S3
Cache Backend Deployment
Tuesday, April 24, 2012
21. Scaling and Tuning
Bottleneck Varnish Load
Varnish Varnish
Cloud
Array Balancing Front
Bandwidth
Bottleneck Frontend Array FE FE FE FE FE FE
FE FE FE FE FE FE FE
CPU Autoscaling FE FE FE FE FE FE
Bottleneck
Database RDS Elastic Cache
mySQL Cache Backend
Tuesday, April 24, 2012
22. Image Caching
Frontend Frontend Backend
Image
Cloud Front resized?
CDN for Assets
Read and save
resized images
Upload new original
S3 images to S3
Assets Storage
Tuesday, April 24, 2012
23. The 6 secrets of caching
1. achieve a high hit rate
2. don‘t flood your cache
3. choose your cache backends wisely
4. never loose all cache at once
5. clean up old cache entries
6. be able to clean/refresh on demand
Tuesday, April 24, 2012
24. Magento Performance Pointers
1. Asyncronous Cache Clearing Queue (Aoe_AsyncCache)
2. Take care of Cache Cleaning (Aoe_CacheCleaner)
3. Disable Magento Logs
4. Make page cacheable in Varnish (Aoe_Static)
5. Prefix bundeled JS and CSS with timestamp
6. and... Profile, Profile, Profile:
•xdebug + kcachegrind
•Newrelic
•xhprof
Tuesday, April 24, 2012
25. Magento Cloud-Adjustments
1. Compress Javascript + CSS on deploy time
2. Use Deployment Name as Cache prefix
3. Cache Warmup Script as part of Deployment
4. Health Check for Loadbalancer
5. Automatic Varnish Purging
6. Use E-Mail Service
7. Establish Asset handling
Tuesday, April 24, 2012
27. Lessons learned
1. 404s are too expensive
=> cache or redirect
2. Cloudfront with S3 backend
cannot handle gzip
=> deploy time bundeling/compression
3. Don‘t hit „Reports“ in the backend :-)
4.You cannot forsee everything
=> But you can be prepared to fix things quickly
=> continuous deployment
Tuesday, April 24, 2012
38. Angry Birds Deployment Pipeline
Backup
SVN
Storage
Commit Install on Selenium Performance
Unit Tests
and Build „latest“ Acceptance Tests Tests
Build Downstream and Test Strategy
Feedback to Developers
Tuesday, April 24, 2012
40. Angry Birds Deployment Pipeline
Install on Cloud Integration Cloud
Deployment Copy to S3 Deployment Tests deployment
System Staging AOE + Rovio production
Tuesday, April 24, 2012
41. Cloud Deployment
R53
Deployment „Version 88“ DNS Deployment „Version 89“
Varnish Array Start new Varnish Array by
release
ELB V V V ELB V V V
✓ cloning old deployment
✓ adjusting release number
Autoscaling Backend Worker Autoscaling Backend Worker
Frontend Array Array Array Frontend Array Array Array
FE FE FE BE BE BE W FE FE FE BE BE BE W
RDS
mySQL
S3
Tuesday, April 24, 2012