Common Blindspots on the Journey to Production and Beyond- Presented at XP Conference India 2015 by Vijay Raghavan Aravamudhan

1. Common blind spots on the
Journey to Production
Vijay Raghavan Aravamudhan
Code/People Agitator @
ThoughtWorks Technologies (Chennai)
Email: vraravam@thoughtworks.com
Github: vraravam
Twitter: @avijayr1

2. Is it ….
CI/CD?
Delivery timelines?
Quality?
Maintainability?
Roadmap?

3. Customer / End User
Team – both dev team + ops
Architecture + Tech Stack
Delivery process
v2+

5. In the context of:
● A new product?
● For existing users?
● Personalization + Analytics
● A/B Testing [for product ideas]

7. ● Non-clustered architecture
● Multiple responsibilities packaged into one “app”
● Sticky sessions
● Blocking requests even for long-running steps
● Hard-coded IP addresses for endpoint URLs
● Hard-coded “linked server” IP addresses in
database (SQL Server)

9. Do you have the right…
• Skillsets?
• Composition?
• Motivation?
• Goal?

10. SCM: {VSS, TFS}, {SVN, CVS}, DVCS
CI: Jenkins/Hudson, TeamCity, GoCD, CircleCI,
TravisCI
CD: GoCD
Testing: QTP, SpecFlow, Cucumber, Selenium
Perf: LoadRunner, SilkTest, Locust.io, Gatling,
Apache Bench, Wrk
Deployment: Gradle, Maven, Ant, Psake, Rake

11. • Central vs Distributed
• Branch per feature
• Trunk/Master
• Short-lived POC-style branches

12. • Checkout
• Clean
• Compile
• Run unit tests
• Run js tests
• Code coverage
metrics
• Run integration
tests
• Package
• Deploy to
Functional Test
env
• Run Functional
Tests

13. ● Build time goes up as the codebase grows
● Time for feedback is longer
● More complex CI setup

14. ● Checkout, Clean, Compile, Run unit tests
(parallellize), Run js tests, Headless tests, Collect
Code coverage metrics, Package
● Deploy to Functional Test env, Run FT
o Split randomly or by functional vertical into a
build/test grid (ala Selenium Grid)

15. Core product is chugging along
Customer1 gets a forked version - 3 month release
cycle
Customer2 gets another fork - 4/5 month release
cycle
Each gets the cumulative feature-set only once both
a complete

16. Both teams diverge in tools and process
Domain knowledge gets siloed
Technology-based career growth might become
stunted
Feature-merge/Integration hell

18. ● Combine both teams at least for design stage so
that each understands what other’s client wants
o Rotate frequently among teams to cross-
pollinate knowledge (tools + techniques)
o “Software artisans”
● Use Dependency Injection (based on tech stack)
+ Feature Toggles
● Feature-branch based development
o Use feature branches @ SCM-level for
architectural changes while delivering BAU

19. Active development across branches
⇒
More CI pipelines

20. Think beyond “application deployment”
Think “environment deployment” or
“ecosystem deployment”
Use tools like Ansible, Vagrant, Puppet, Chef,
Docker*

21. • Db changes should be developed alongside the
story
• It should also be part of the commits into the
SCM
• Scripts should always be incremental in nature
• As part of the CI build, ensure both roll-forward
and roll-back works
Use tools like flywaydb, dbDeploy

22. ● Most teams assume that the app-layer will be
enough to ensure data integrity
● What happens if the app is replaced by a new
app - the db will live on, correct?
● Data validations should also be applied at the db
level - for eg foreign keys, unique constraints,
non-null checks, case-sensitive checks
● ACID Transactionality should be ensured whether
or not an ORM is used

23. • Use tools like active_sanity (rails gem)
• Obfuscated database snapshot from production
uploaded into non-prod env for testing on weekly
basis
• [Unfortunately] Yet another checkpoint before
pushing to production

24. • App should be clustered, and cluster-aware
• Deployments should not have any manual
intervention (including DB)
• DB changes should be backwards compatible
 (n+1)th release can cleanup temporary stuff
from nth release
• API-changes should be backwards compatible
 Dont have a “long tail” of multi-version support

25. ● SOX/PCI Compliance
● Data at rest
● Data in transit
● Threat Vectors
● Attack surface
OWASP Guidelines