Data Saturday #1 - Pordenone 27 February 2021 https://datasaturdays.com/events/datasaturday0001.html

1. #datasatpn
February 27th, 2021
Database deployment:
still hard after all these years
Giulio Vian
@giulio_vian

2. Avvertenze

3. In a Start-up far far away
DBA
Senior Dev
Scrum
Master
Junior
Dev

4. Free!
Troppo bello per
esser vero
• gratis
• open source
• tutti i DB!
• ed è pure usato
• e funziona!

5. Passano
I mesi
fino
ad un certo
venerdì

7. La sfiga
ci vede
benissimo

8. Hardware spec:
1 KB RAM
(16KB after upgrade)
4 KB ROM
(8KB after upgrade)
First computer Past Companies Communities
Giulio Vian Senior DevOps Engineer
@giulio_vian
giuliovdev@hotmail.com

9. Agenda
Paradigmi cambiamento
Modifiche ai dati
Test Automatizzati
Pipeline e Automazione
Tecniche per il Deploy

10. Paradigmi

11. Migration-based
Source: https://www.red-gate.com/hub/product-learning/sql-source-control/moving-from-application-automation-to-true-devops-by-including-the-database

12. Baseline (aka v0.0.0)

13. Script di migrazione independentI

14. Il Limite di DbUp
ma anche di FlyWay, Liquibase, e simili

15. Script di migrazione independentI
conflict

16. State-based
Source: https://www.red-gate.com/hub/product-learning/sql-source-control/moving-from-application-automation-to-true-devops-by-including-the-database

17. Diff script from schema-based tool
uh oh

18. Neanche dacpac puo’ salvare

19. Migration based
v1  v2
Non rileva conflitti
L’ordine è cruciale
State based
v1  v2
Non riconosce i rename
Azzera eventuali
differenze
I due paradigmi

20. Migration based
v1  v2
State based
v1  v2
I due paradigmi

21. Are the two mutually exclusive?
No
Dev can use a Schema diff tool like SSDT to generate
migration scripts
Migration script can be augmented stating the baseline
version
State-based tools have pre- and post-deploy phases for data
migration
We can add dependency check between migration scripts
To warn about scripts interaction and dependency
MS T-SQL syntax parser available for .Net

22. Powershell basta per un
controllo della baseline

23. Modifiche ai dati
e gestione di sviluppi paralleli

24. Quale DB per gli Sviluppatori?
Privato Condiviso

25. Dati realistici
Dimensioni
Requisiti di riservatezza
Pre-trattamento
Spazio occupato

26. Conseguenze
Problemi tecnici
Durata del Deployment
Performance delle query
Problemi commerciali e legali
Abuso di informazioni

27. Test Automatizzati

28. Unit testing
tSQLt
Open source database Unit Testing
framework for SQL Server
https://tsqlt.org/
DbUnit
JUnit extension
http://www.dbunit.org/
NDbUnit2
.NET library for managing database state
during unit testing
https://github.com/savornicesei/NDbUnit
2
DbFit
Fitness style (Java+Jdbc)
http://dbfit.github.io/dbfit/
Respawn
Intelligent database cleaner for
integration tests
https://github.com/jbogard/Respawn

29. Oltre lo schema
Containers
Macchine virtuali
Snapshot di volumi

30. Pipeline e Automazione
Packaging

31. Elementi Chiave
In input solo sorgenti
Pacchetti (es. dacpac) vanno in un Artifact Store
e devono essere il risultato di una build
Istanze di database per la build
Ad ogni Branch la sua Istanza
Gestione delle credenziali

32. Tecniche per il Deploy

33. Database Refactoring Cycle
Initial state Final state
Implement the refactoring
Test on non-production
Refactoring completed
deploy new schema,
migrate data
deploy new application version

34. Database Refactoring Cycle
Initial state Final state
Implement the refactoring
Test on non-production
Refactoring completed
deploy new schema,
migrate data
deploy new application version

35. Database Refactoring Cycle
Initial state Final state
Transitional state
Implement the refactoring
Test on non-production
Refactoring completed
All systems use the new schema
Coexistence of
old and new schema
Expand:
deploy new schema,
migrate data,
add scaffolding code
Contract:
Drop old schema objects
and scaffolding code

36. Riassumendo

37. applicazioni
DATI

38. La morale
Nessun tool
ti può salvare
Studiare è
l’unico modo
di prevenire
problemi
Disciplina,
tanta
disciplina

39. References
Tools
SQL Server Data Tools (SSDT)
RedGate SQL Compare
devart
Flyway
dbup
dbdeploy.net
Articles
https://en.wikipedia.org/wiki/Sc
hema_migration
https://martinfowler.com/article
s/evodb.html
https://michaeljswart.com/2018
/01/100-online-deployments
http://blog.dixo.net/2015/02/bl
ue-turquoise-green-deployment
Books
Refactoring Databases:
Evolutionary Database Design
http://www.agiledata.org/

40. Grazie e alla prossima