1. quality @ANGIESLIST
5/28/2014

6. 1. DEVELOPMENT TESTING

7. DEVELOPMENT TESTING
CODE functions and performs as expected
$0.10 defects

8. DEVELOPMENT TESTING
unit | api | isolated functional | benchmark
+thousands run per commit
+pre merge
+assume function will break

9. DEVELOPMENT TESTING
"catch bugs on your laptop"
isolated | local db | small dataset | no external
dependencies

10. DEVELOPMENT TESTING
KEYS TO success
+sub-10 minute build
+enforced code coverage in build
+functional/benchmark test coverage
+TDD

11. 2. RELEASE TESTING

12. RELEASE TESTING
CODE integrates and performs as expected
$1,000 defects

13. RELEASE TESTING
function | api | behavior | benchmark
+managed test environment
+weekly production snapshot
+assume integrations and data will break

14. RELEASE TESTING
PRE-PROD e2e integration
+integration testing with real data
+prod db snapshot
+external dependencies on

15. RELEASE TESTING
PRE-PROD Performance
+established benchmarks
+production snapshot (scrubbed)
+test high impact, revenue critical functions

16. RELEASE TESTING
KEYS TO success
+no performance degradation under load
+feature validation and verification
+demos to stakeholders
+release speed (fix => production)

17. 3. PRODUCTION TESTING

18. DOGFOODING
+share release candidates with AL employees
+mobile and web

19. MONITORING
+notification on thresholds (for dev)

20. PRODUCTION TESTING
KEYS TO success
+no disruption for AL employees
+quick turnaround on monitoring failures/warnings

22. ANGIE'S LIST
Jeff Rogers
@jrojers
jrojers@gmail.com