Opening presentation from Solano Labs Co-Founder Jay Moorthi for Automated Testing San Francisco. This is an overview of Continuous Integration and Deployment best practices. Please let us know what you think!

1. Automated
Tes,ng,
Con,nuous
Integra,on
&
Con,nuous
Deployment
Jay
Moorthi
Co-­‐Founder,
Solano
Labs
@SolanoLabs
Automated
Tes,ng
San
Francisco

2. Some Facts & Figures*
High performing organizations:
• Deploy code 30 times more frequently.
– 8000 times faster than their peers
– Deploy multiple times a day, versus an average of once a month.
• Have 50 percent fewer failures.
• Restore service 12 times faster than their peers.
– Fewer failures and faster recovery mean less risk to the business when
changes are deployed.
*Source:
Puppet
Labs
2013
State
of
DevOps
Report

3. How
are
they
doing
this?
I
don’t
have
all
the
answers…
But
here’s
one:
Automa,on

4. Automated Testing Does More Than Find Bugs
Its Helps Teams Build a Better Product
Bug are Common, Difficult
to Eliminate & Very Costly
Automated Testing:
Using software to test software
• Reduces
cost
of
bugs
by
reducing
,me
between
introduc,on
and
discovery
• Frees
engineers
to
make
large
changes
fearlessly,
reducing
cycle
,me
• Reduces
complexity
and
risk
at
feature
integra,on

5. Automation: Reduces the Cognitive Feedback Loop
Automa,on
shortens
the
cogni,ve
loop
for
engineers
• Automa,on
delivers
results
for
a
build
quickly
• Shorter
turn-­‐around
,mes
change
how
engineers
work
• Test-­‐Driven
Development
(TDD)
is
one
logical
extreme
Waiting for tests to run
is no fun!
Incorporate
Feedback
Run
Tests
Write
Code
Get
Results

6. What Makes for a Good Automated Test?
• Eliminate repetitive manual tests
• Yield reproducible results
• Cover common and edge/error cases
• Ideally either pure unit or integration test
• Strive for clarity even when code is gnarly
• Test for business requirements

7. Why Not Automate Everything?
• Writing tests does have a cost
– May not be appropriate for experimental code
– May be fragile – UI/mobile automated testing is evolving
– Difficult or impossible to express design and human factors
• Thorough testing requires commitment and discipline
– Writing tests is a short-term pain for a specific individual
– Test suite as a whole is valuable to the team over time

8. What
do
you
do
with
all
those
automated
tests?

9. Continuous Integration: A Discipline for Testing Every
Change
I
Strong
Isola,on
Detailed
Instrumenta,on
Consistent
Run,me
Environment
Reproducible
Results
Components that Make for Effective CI:
Typical CI Setup:

10. Continuous Deployment: Automatically Release Validated
Changes
• Goal: make release a non-event
– Popularized by Web 2.0, but much more widely applicable
– Final decision to deploy may still be manual
• Valuable to business and engineering sides of the house:
– Selectively enabled features – feature flags
– Consistent, widespread measurement
• Key Ingredients:
– Small changes, continuously integrated
– Automated infrastructure (DevOps)
– Staging/test environment accurately models production

11. Further Reading
• Martin Fowler from ThoughtWorks: martinfowler.com
• Book: Continuous Delivery: Reliable Software (Jez Humble)
• IMVU: http://timothyfitz.com/2009/02/10/continuous-deployment-at-
imvu-doing-the-impossible-fifty-times-a-day/
• Etsy Blog: codeascraft.com
• Github: teach.github.com/articles/lesson-continuous-delivery/
• http://info.puppetlabs.com/2013-state-of-devops-report.html
• http://engineering.quora.com/Continuous-Deployment-at-Quora