В продолжение темы непрерывной интеграции, Макс расскажет о своем подходе организации непрерывной интеграции и деплоймента в Symfony проектах. Рассказ включает следующие темы:
- Управления зависимостями
- Процесс и инструменты для сборки
- Сервера непрерывной интеграции и в частности Jenkins, плагины к нему, jobs
- Процесс разработки в git
- Процесс выгрузки релиза
- Миграция БД
- Откат релиза
4. Continuous integration (CI) is the practice, in
software engineering, of merging all
developer working copies with a shared
mainline several times a day.
http://en.wikipedia.org/wiki/Continuous_integr
ation
http://www.martinfowler.com/articles/continuo
usIntegration.html
What is Continuous Integration?
4
5. • Improve quality of application
• Understand current situation on project
• Automate your checklists & verifications
• Notify different people on different events
– Build broken? – Developers
– New build deployed to QA? – QA engineers
– New build deployed to UAT/Production? –
Customers
How could it help you?
5
6. Dependencies
• Composer
• Platform
packages
• Phar
• Pear
• Pecl
Build
• Phing
• phpqatools.org
Continuous
Integration
• Jenkins CI
• Additional PHP-
related & general
plugins
• Git
Deployment
• Package with
vendors
• Capifony
• Doctrine
Migrations
Our proposal for Symfony-based projects
6
9. Checklist
• stability
• lib-*, ext-*
• install vs update
• require
• version masks
• Keep composer.phar
under VCS
• Satis for private
packages
9
Composer
github.com/composer/satis
lib-*, ext-*: goo.gl/Z8dFal
10. Phar
• Only for CLI libraries used for development and build
• Composer, php-cs-fixer
Pear
• Build tools
• Phing, phpunit*, phpqatools
Pecl
• For PHP extensions missing in platform packages
Platform packages (apt, yum, ports)
• Use it for all purposes if you can (PHP, extensions etc.)
Other tools
10
11. What to update?
• Composer dependencies
• PHARs
• PEARs
• PECLs
• Platform packages
Keep in mind
• Update at the beginning of the
iteration
• Read changelogs
• Subscribe to news feeds
• Use * as Composer dependency
version to receive all updates
• Not for Symfony itself
Dev
QA
Staging
• Performance testing
• Check logs
Production
11
Don’t forget about updates!
13. Ant
• From Java world
• XML declarative config
• Can run tasks in parallel
• No PHP-specific tasks
• Could be extended (Java)
• IDE support
Phing
• Written in PHP
• Ant config syntax
• All tasks by default are running
in the single PHP process
• Could be extended (PHP)
• IDE support
Pake
• Written in PHP
• Not so popular
• PHP-based config
• github.com/indeyets/pake/wiki
Build tools
13
15. Target
• A set of tasks
• Could depend
on other targets
• phing
<target_name>
Task
• Action
executed as a
part of target
• Custom tasks
can be
implemented
Types
• Reference
• FileSet
• Property
• Other
• FileList
• Filters
Phing Terminology
15
16. build
• for dev
environment
• outputs to
the console
build-ci
• for CI
environment
• outputs to
XML files
package
• generates
artifacts for
deployment
• tar.gz
archive
• SQL
schema
dump
deploy
• deploys
tar.gz
archive to
the
environment
Main Phing Targets
16
20. Symfony-
specific
copy-configs-ci parameters.yml for CI
cache:warmup app/cache is removed
during build
twig:lint for Bundles and
app/Resources
assets:install
assetic:dump YUICompressor, Google
Closure Compressor,
lessphp
Phing Targets (4/6)
20
22. Package package-version version.txt
package-db-schema schema.sql
package-archive
Deployment deploy-maintenance- enable/disable
deploy-backup-db
deploy-clear-doctrine-cache Only if not in APC
deploy-cleanup-old-releases
deploy-artifact artifact name provided via CLI
argument
Phing Targets (6/6)
22
25. • Open-source
• Enterprise
• Lot of plugins
• PHP Template from Sebastian
Bergmann
• IDE Integration
• RESTful API
25
Jenkins CI overview
jenkins-ci.org
26. • No comments git
• green & red signs for buildsgreenballs
• parameterized builds
parameterized-
trigger
• git tag selector for builds
• disable internal tagging!
git-parameter
• copy artifact from one job to
anothercopyartifact
• sends emails on successful builds
• useful for deployment jobs
email-ext
26
Jenkins Plugins (common)
goo.gl/IZ2gKE
27. phing
• phpcs
• hphpacheckstyle
• phpcpddry
• php_dependjdepend
• phploc using CSV filesplot
• phpmdpmd
• aggregates info from phpcs, hphpa,
phpmd, phpcpdviolations
• phpunitxunit
• HTML artifactshtmlpublisher
27
Jenkins Plugins (PHP-specific)
jenkins-php.org
28. Master
• Bleeding edge
• For completed features
Feature
branch
• Merge to master on completion
• After code review
• Developer executes phing build at least before code review
Tag
• Stable version from master
Bugfix &
develop?
• Sprint branch
• Master branch
• Merge feature branches to sprint branch
• Jenkins job for sprint branch
Git workflow
28
29. • Builds specific branch (master)
• Features are verified after merge
build-<branchname>
• Parameterized
• Produces .tar.gz artifact from specific
Git tag
build-package-tag
Jenkins Jobs for build
29
30. • Produces .tar.gz artifact from latest revision of branch
• Deploys to QA
• Not recommended to have more than one such job
(issues with migrations)
• Workaround: multiple QA or complete DB purge on deploy
deploy-qa-<branchname>
• Deploys specific package from build-package-tag to
specified environment
• Ability to enable/disable maintenance mode and error
message
• Email after deployment
deploy-package-tag
Jenkins Jobs for deployment
30
31. Deployed .tar.gz archive
• <tag>.<build#>
• v0.0.3.26
• <branchname>-<build#>-<revision>-SNAPSHOT
• master-12-7c041a90366ce9d958760094c763d809ffbc629e-
SNAPSHOT
Previous deployed version
DB dump before deployment
Doctrine Migrations info
DB schema
Jenkins Jobs for deployment. Artifacts
31
33. Capistrano
• from Ruby world
Capifony
• Based on
Capistrano
Shell script
• VCS update
• Rsync
Phar
• Don’t use WebPhar
• Silex does not use
it anymore
Zend Server
package
• No support for
vanilla PHP
PaaS
• AWS Elastic
Beanstalk
• PagodaBox
Deployment tools
33
34. Based on Capistrano
• Supports Capistrano plugins
Symfony 1&2 deployment
Multistage
• qa, staging, uat, production
Maintenance mode
Clustered environments
34
Capifony
capifony.org
35. Directories
• releases/
• current/ (symlink)
• shared/
Shared
• app/config/parameters.yml
• app/cache
• app/logs
• web/uploads
Shared files & dirs are
symlink’ed
Shared files & dirs are
not shared across
nodes
Directory structure for Capistrano projects
35
37. All you need is package
• capistrano-deploy-strategy-archive
Github not available?
• Goodbye, vendors!
Decreases deployment time
• Lot of vendors
• Slow internet connection
Reproducible
• You know what you deploy
• You can deploy exactly the same package to multiple
environments
Deployment via SCM vs package
37
40. User
• Triggers job on Jenkins CI
• Choose options (env, maintenance mode)
Jenkins
• Executes single phing target with additional
command-line arguments
Phing
• Executes several Capifony commands
according to command-line arguments
Capifony
• Performs actions on remote machines
Capifony & Jenkins CI
40
41. Enables Maintenance mode
• maintenance.html in webroot
Backups DB
Clears Doctrine cache
• Not for APC cache
Downloads previous deploy metadata
• package version
• Doctrine Migrations info
Deploys tar.gz archive
Capifony & Phing (1/2)
41
42. Restarts services
• php-fpm or Apache
• Doctrine APC cache is cleared
Symfony-specific tasks
• copy app/config/parameters.yml
• doctrine:migrations:migrate
• cache:warmup
Disables Maintenance mode
Performs cleanup for old releases
Capifony & Phing (2/2)
42
43. Semi-
automatical
• from
deployment
server
• usually CI
server
Enable
Maintenance
• manually via
phing
Manually backup
for analysis
• DB
• current version
• Doctrine
Migrations
status
• codebase
Rollback (1/2)
43
44. Switch to
previous release
• Manually via
capifony CLI
• Recover DB
• Rollback
Doctrine
Migrations
• Restore DB
dump
• Custom
Additional tasks
• Restore web
server config
• Restart service
• cache:warmup
Disable
Maintenance
• Manually via
phing
Rollback (2/2)
44
45. Write rollback guide
• Step-by-step
• Verify it periodically
Enable monitoring on prod
• Zabbix
• Munin
• New Relic
• Pinba
• Graylog2
Rollback tips
45
47. • Currently not available online
• Planning to release in 1-2 months on
GitHub
Where to download?
47
48. Deployment
•Cronjob / CLI script handling
•Web server (Apache, nginx)
config manipulations
•Ext- and lib- dependency
verification via Composer
What is not implemented yet
48