This presentation is about managing database scripts, why we need to do it from theoretical and practice perspective, how it improves continues integration and delivery process on real projects.
2. Who am I?
Lead Software Engineer at EPAM Systems
More than 7 years in commercial java development
2 project from scratch to production
Speaker experience at Epam events, Logik Night,
UADEVCLUB.
3. Why me?
Working with liquibase since 2014
Have used liquibase on 3 projects
Implemented liquibase on 1 project
Liquibase is a part of my dream application
4. Agenda
How do we work with DB? What do we really need?
Liquibase artifacts
Liquibase commands
Liquibase with maven
Liquibase with Spring Boot
5. First DB - first decisions
DB script management
DAO testing
Deployment
6. Types of DB scripts
Scripts that change existing data or structure in DB
Scripts that change existing logic(procedures, views...)
7. DB management
Create one file with all sql scripts
Create separate files named 1.sql, 2.sql….
Use spring.jpa.hibernate.ddl-auto
Custom tool
DBA will find a solution
8. DAO tests
Run scripts on test DB
Use existing database, populate it with test data before
test running and clean it after tests running
Use embeded db with predefined test data
Insert test data
Remove test data after test
17. Liquibase is just a jar file
liquibase [options] [command] [command parameters]
Example:
java -jar liquibase.jar
--driver=oracle.jdbc.OracleDriver
--classpath=pathtoclasses:jdbcdriver.jar
--changeLogFile=com/example/db.changelog.xml
--url="jdbc:oracle:thin:@localhost:1521:oracle"
--username=scott
--password=tiger
update
18. Liquibase.properties file
1.Create liquibase.properties file near liquibase.jar
2.Add all connection data to this file
driver: org.mariadb.jdbc.Driver
url: jdbc:mariadb://localhost:3306/shop_db
username: shop
password: qwerty
classpath: /home/illcko/liquibase/dbdrivers/mariadb-java-client-1.4.6.jar
24. Bundled Changes
ADD AUTO INCREMENT
ADD COLUMN
ADD DEFAULT VALUE
ADD FOREIGN KEY CONSTRAINT
ADD LOOKUP TABLE
ADD NOT NULL CONSTRAINT
ADD PRIMARY KEY
ADD UNIQUE CONSTRAINT
ALTER SEQUENCE
CREATE INDEX
CREATE PROCEDURE
CREATE SEQUENCE
CREATE TABLE
CREATE VIEW
CUSTOM CHANGE
DELETE
DROP ALL FOREIGN KEY
CONSTRAINTS
DROP COLUMN
DROP DEFAULT VALUE
DROP FOREIGN KEY
CONSTRAINT
DROP INDEX
DROP NOT NULL CONSTRAINT
DROP PRIMARY KEY
DROP PROCEDURE
DROP SEQUENCE
DROP TABLE
DROP UNIQUE CONSTRAINT
DROP VIEW
EMPTY
EXECUTE COMMAND
INSERT
LOAD DATA
LOAD UPDATE DATA
MERGE COLUMNS
MODIFY DATA TYPE
RENAME COLUMN
RENAME TABLE
RENAME VIEW
SQL
SQL FILE
STOP
TAG DATABASE
UPDATE
29. SQL as part of changeset
<changeSet id="drop_storage_with_index_data" author="Illia_Seleznov" dbms="oracle">
<sql splitStatements="false">
<![CDATA[
DECLARE
filter_count number;
BEGIN
select count(*) into filter_count FROM CTXSYS.CTX_PREFERENCES WHERE PRE_NAME
= '<MNG_STORAGE>' AND PRE_OWNER in (select user from dual);
IF filter_count > 0 THEN
ctx_ddl.drop_preference( ''<MNG_STORAGE>');
END IF;
END;
]]>
</sql>
</changeSet>
35. Preconditions
<changeSet id="1" author="bob">
<preConditions onError="MARK_RAN">
<tableExists tableName="angry_devops"/>
</preConditions>
<comment>Comments should go after preCondition. If they are before then liquibase usually gives
error.</comment>
<createTable tableName="angry_devops">
<column name="angry" type="int"/>
</createTable>
</changeSet>
39. Update commands
update - updates database to current version
updateCount <value> - applies the next <value> change sets
updateSQL - writes SQL to update database to current version to STDOUT
updateCountSQL <value> - writes SQL to apply the next <value> change sets
to STDOUT
40. Diff commands
diff [diff parameters] - writes description of differences to standard out
diffChangeLog [diff parameters] - writes Change Log XML to update the base database to the target
database to standard out
Include:
● Version Differences
● Missing/unexpected tables
● Missing/unexpected views
● Missing/unexpected columns
● Missing/unexpected primary keys
● Missing/unexpected unique constraints
● Missing/unexpected foreign Keys
● Missing/unexpected sequences
● Missing/unexpected indexes
● Column definition differences (data type,
auto-increment, etc.)
● View definition differences
● Data differences (limited), not checked by
default
Exclude:
● Non-foreign key constraints (check, etc)
● Stored Procedures
● Data type length
41. Documentation commands
dbDoc <outputDirectory> - generates Javadoc-like
documentation based on current database and change log.
java -jar liquibase.jar --changeLogFile=changelogs/xml/master.xml dbDoc ../doc
42. Maintenance commands
tag <tag> - "tags" the current database state for future rollback.
tagExists <tag> - Checks whether the given tag is already existing.
status - Outputs count (list if --verbose) of unrun change sets.
validate - checks the changelog for errors.
changelogSync - mark all changes as executed in the database.
changelogSyncSQL - writes SQL to mark all changes as executed in the database to STDOUT.
markNextChangeSetRan - mark the next change set as executed in the database.
listLocks - lists who currently has locks on the database changelog.
releaseLocks - Releases all locks on the database changelog.
dropAll - Drops all database objects owned by the user. Note that functions, procedures and packages are not dropped
(limitation in 1.8.1).
clearCheckSums - Removes current checksums from database. On next run checksums will be recomputed.
generateChangeLog - generateChangeLog of the database to standard out. v1.8 requires the dataDir parameter
currently.
43. Rollback commands
rollback <tag> - rolls back the database to the state it was in when the tag was applied.
rollbackToDate <date/time> - rolls back the database to the state it was in at the given date/time.
rollbackCount <value> - rolls back the last <value> change sets.
rollbackSQL <tag> - writes SQL to roll back the database to the state it was in when the tag was applied
to STDOUT.
rollbackToDateSQL <date/time> - writes SQL to roll back the database to the state it was in at the
given date/time version to STDOUT.
rollbackCountSQL <value> - writes SQL to roll back the last <value> change sets to STDOUT.
futureRollbackSQL - writes SQL to roll back the database to the current state after the changes in the
changeslog have been applied.
updateTestingRollback - updates the database, then rolls back changes before updating again.
