In this session Max Mether, VP of Product Management at MariaDB, provides an introduction to MariaDB Platform X3 and the new features in MariaDB Server 10.3 and MariaDB MaxScale 2.3. He then turns his focus to what’s coming in MariaDB Server 10.4, including instant DROP COLUMN, the INTERVAL data type and advanced security features like account locking.

1. MARIADB
What’s New
Max Mether
VP MariaDB Server PM
MariaDB Corporation

2. MariaDB Platform X3

3. MariaDB TX
MariaDB Server
MariaDB MaxScale
InnoDB/MyRocks
MariaDB AX
MariaDB Server
MariaDB MaxScale
ColumnStore
App/dev
MariaDB
TX
BI/data science
MariaDB
AX

4. MariaDB TX 3.0
MariaDB Server 10.3
MariaDB MaxScale 2.2
InnoDB/MyRocks
MariaDB AX 2.0
MariaDB Server 10.2
MariaDB MaxScale 2.2
ColumnStore 1.2
MariaDB Platform X3
MariaDB MaxScale 2.3
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3

5. Applications
Containers
MariaDB Platform X3
MariaDB MaxScale 2.3
CDC
MariaDB Server 10.3
InnoDB/MyRocks
MariaDB Server 10.3
ColumnStore 1.3
Transactional Analytical
Kubernetes (Helm) Docker (Compose)
C JDBC ODBC Node.js
Ingest streaming data
Kafka connector
Administration
SQL Diagnostic
Manager
SQLyog
MariaDB Backup
MariaDB Flashback
Import bulk data
Spark connector
C/Java/Python API

6. MariaDB Server
(Spider)
Application
Connection 1
MariaDB
Server 1
Row storage
MariaDB
Server 2
Row storage
MariaDB
Server n
Row storage
MariaDB Server
(ColumnStore)
CS node 1
Columnar
storage
CS node 2
Columnar
storage
CS node n
Columnar
storage
Sharding Distributed storage
MariaDB
MaxScale
Transactional Analytical

7. MariaDB Server 10.4

8. MariaDB Server 10.4
Performance Security Temporal High Availability

9. WHAT WE’RE FOCUSING ON
1. New Instant operations to speed up altering tables
for InnoDB
2. Enhancing the security features in regards to
a. User Accounts
b. Data-at-Rest
3. Implement the next level for working with temporal
tables
4. Setting the ground for providing more features for
MariaDB Cluster
5. Better Backup

10. Instant Schema
Changes
For InnoDB
Performance Security Temporal
High
AvailabilityPerformance

11. Overview
● Instant operations for altering tables
○ Speed up operations
○ Lower storage space requirements for ALTER TABLE
operations
● Adding columns including reordering
● Drop columns
● reorder columns
○ FIRST, LAST, AFTER
● Instant charset and collation changes
● Increasing maximum column length for VARCHAR

12. Instant ALTER TABLE
set alter_algorithm=instant;
| ALGORITHM = INSTANT
● Can also added directly to the query
● The algorithm INSTANT results in an error, if
○ The table would need a full copy
○ Data would be partially copied
○ Data would need to be read
Assure an Instant Operation

13. Instant ALTER TABLE
Adding Columns
| ADD [COLUMN] [IF NOT EXISTS] col_name column_definition
[FIRST | AFTER col_name ]
| ADD [COLUMN] [IF NOT EXISTS] (col_name column_definition,...)
● In MariaDB Server 10.3 a column needs to be added as the last column
MariaDB [10.3]> alter table t1 add column mychar3 char(2) FIRST;
ERROR 1845 (0A000): ALGORITHM=INSTANT is not supported for this operation. Try
ALGORITHM=INPLACE

14. Instant ALTER TABLE
Changing Column Order
| CHANGE [COLUMN] [IF EXISTS] old_col_name new_col_name column_definition
[FIRST|AFTER col_name]
● With MariaDB 10.4 FIRST / AFTER are supported as instant operation
○ Data type and size cannot be changed

15. Instant ALTER TABLE
Changing Column Order
| MODIFY [COLUMN] [IF EXISTS] col_name column_definition
[FIRST | AFTER col_name]
● Instant if
○ Data type and size does not change

16. Instant ALTER TABLE
Dropping a Column
| DROP [COLUMN] [IF EXISTS] col_name [RESTRICT|CASCADE]
● Instant removal of a column

17. INSTANT ALTER TABLE
Increasing column length for VARCHAR
ALTER TABLE t1 CHANGE f1 f1 VARCHAR(256), alter_algorithm=instant;
ALTER TABLE t1 MODIFY f1 VARCHAT(256), alter_algorithm=instant;
● Depends on the used InnoDB row format
○ Default is DYNAMIC
● Any extension of VARCHAR from ≤127 bytes
● Unlimited if ROW_RORMAT=REDUNDANT

18. Optimizer
Performance Security Temporal
High
AvailabilityPerformance

19. Overview
● New Optimizer defaults
● Optimizer Trace
○ JSON object recording the execution path through the
optimizer
● Condition push down
○ into materialized IN subqueries
○ From HAVING into WHERE
● Histograms based on random row samples
● In-memory primary key / rowid filters

20. New Optimizer Defaults
● Make use of histogram information and engine independent table statistics by
default
○ Condition Selectivity
■ Use selectivity of all range predicates estimated with histogram
○ Use engine independent table statistics (EITS)
○ Build histograms when collecting EITS
● Auto-size for the join buffer
● Use index statistics (cardinality) instead of records_in_range for large IN-lists

21. Optimizer Trace
● JSON object recording execution
path
○ Understand why the optimizer is
choosing a path
mariadb> set optimizer_trace=1;
mariadb> <query>;
mariadb> select * from
information_schema.optimizer_trace;
{
"steps": [
{
"join_preparation": {
"select#": 1,
"steps": [
{
"expanded_query": "/* select#1 */ select
`t1`.`col1` AS `col1`,`t1`.`col2` AS `col2` from `t1`
where (`t1`.`col1` < 4)"
}
]
}
},
{
"join_optimization": {
"select#": 1,
"steps": [
{
"condition_processing": {
"condition": "WHERE",
"original_condition": "(`t1`.`col1` < 4)",
...

22. Security
Performance Security Temporal
High
Availability
Security

23. Overview
● Enhanced Authentication and Privilege System
● Improvements for Data-at-Rest
● Enhancements to Security Maintenance
For MariaDB Enterprise Server
● Audit Plugin Enhancements*
● Galera Data-at-Rest encryption for Gcache*

24. Enhanced Authentication / Privilege System
● Password Expiration
○ To fulfill enhanced security requirements
● Disable User Accounts via SQL
○ Allows to temporarily lock an account
● SET PASSWORD statement
○ Simplifies changing a password
● Enable authentication via unix_socket by default
○ mysql_secure_installation will ask for setup unix_socket based authentication
● Multiple Authentication Methods per User
○ Allows to verify a user internally if an external authentication is currently not available
● Block user accounts after a number of failed login attempts

25. Improvements for Data-at-Rest
● Key rotation now also for the InnoDB Redo Log
○ It was a limitation for the redo log that key rotation was not possible
● Full Data-at-Rest Encryption also for MariaDB Cluster (Enterprise Feature)
○ The Galera Cluster Gcache (IST Transaction Log) is now encrypted
○ Encryption enabled by using binary log Data-at-Rest encryption
● Spatial Index for InnoDB is now providing Data-at-Rest encryption

26. Enhancements to Security Maintenance
● Dynamic loading of SSL certificates
○ SSL certificates can be reloaded without a server restart
○ Certificate revocation list (CRL) reload
● Server log includes information about ignored passwords
○ Passwords are ignored if external authentication is used
● New format for the user table for future enhancements

27. Audit Plugin Enhancements
● Enhanced internal API
○ When writing a plugin, more information is available
● Enterprise Audit Plugin
○ Flexible User based filter definition for auditing
■ Auditing requirements for human users and application users can be different
○ Filters and templates defined in system tables
○ Auditing configuration options and changes can be logged
○ Connections can be filtered

28. Temporal
Application-Time Period
Tables
Performance Security Temporal
High
Availability
Temporal

29. Overview
● MariaDB Server 10.3 introduced System Versioned
Tables
○ Tracking data changes based on “system time of
change”
○ Special requirement for auditing
● NEW: Application-Time Period Tables
○ Tracking data changes based on the time, a change is
valid for
○ Application controlled timestamps
○ Limitation: The option WITHOUT OVERLAPS is not
yet supported

30. Compare Type of Temporal
● Time of change by system time
● Forensic analysis & legal
requirements
○ Store data for N years.
● Data analytics (retrospective,
trends etc.)
● Point-in-time recovery - recover
as of particular point in time
System Versioned
● Valid time period defined
by application
● Insurance Applications
● Banking (transfer limit, ...)
● whenever data has to be valid
for a given start and end date
“period of interest”
Application-Time Period Tables

31. Bitemporal
Application-Time Period + System Versioned
Tables
combined in one Table

32. Application-Time Period Tables
● With Application-Time Period Tables the timestamps of the period are defined
by the user/application
○ Period of validity defined and changed by INSERT, UPDATE
acc_num Start End maxTrans
1234 2010-01-01 2011-11-12 5000
acc_num Start End maxTrans
1234 2010-01-01 2011-11-12 5000
1234 2011-02-03 2011-09-10 10000
1234 2011-09-10 2011-11-12 5000
UPDATE Emp
FOR PORTION OF accPeriod
FROM DATE '2011-02-03'
TO DATE '2011-09-10'
SET maxTrans = 10000
WHERE acc_num = 1234;

33. High Availability
Enhancements to
MariaDB Cluster
Performance Security Temporal
High
AvailabilityHigh
Availability

34. Overview
● MariaDB Cluster based on Galera Version 4
○ New Galera 4 based API in MariaDB Server will allow
further extensions
○ New Galara 4 Library
● Support of MariaDB group commits
● Streaming Replication for huge transactions
● Support of Instant Alter for InnoDB
● Rolling Upgrade from MariaDB Cluster 10.3 to 10.4

35. Support of MariaDB group commits
● MariaDB Server group commits allow parallel replication
○ Galera now also can gain advantage of the grouped transactions
● A group of transactions can be committed as a block
● Performance advantages where non-conflicting transactions exist

36. Streaming Replication
● Size of a transaction limited in MariaDB Cluster 10.3
● No limitation with streaming replication
○ Recommendation: Huge transactions still should be an exception
● Transaction replicated in fragments
Huge Transaction Support

37. Optimized Backup

38. Overview
● Optimized “FLUSH TABLES WITH READ LOCK” (FTWRL)
○ Instant FTWRL for not used tables
○ Second FTWRL for the rest of the tables when not used anymore
● Implementation of backup stages
○ Engine aware locking methods
● BACKUP LOCK statement
○ Meta lock on a table to block DDL
○ Allows consistent copy of dependent files
● Enterprise Backup tool to create consistent backups while minimizing needed
locks
○ DDL allowed to execute in parallel
○ Locking minimized for all engine

39. And even more ...

40. Overview
● Crash safe system tables by using Aria
● Validation for field type JSON
○ The JSON_VALID constraint will be added by default
● Unique indexes are now available also for blobs
● Support of brackets (parentheses) in
UNION/EXCEPT/INTERSECT

41. THANK YOU!