Wussten Sie, dass das rein relationale Dogma von SQL bereits 1999 aufgegeben wurde? SQL-92 war der letzte Standard, der auf die relationale Idee beschränkt war. Ab 1999 wurde SQL um nicht-relationale Operationen und Datenstrukturen erweitert. Obwohl dieser Schritt damals viel diskutiert wurde, dauerte es Jahrzehnte, bis die Datenbankhersteller dieses Paradigmenwechsel verdaut hatten. Viele Entwickler haben bis heute nichts davon gehört. Dieser Vortrag gibt einen Überblick über die Evolution von SQL seit 1999 und stellt einige der neuen Funktionen anhand häufiger Anforderungen vor. Dabei wird auch aufgezeigt, wann diverse Hersteller begonnen haben, nicht-relationales SQL zu unterstützen. Letztendlich zeigt der Vortrag, dass sich SQL in den letzten Jahrzehnten genauso weiterentwickelt hat, wie die Anforderungen mit denen man in der Entwicklung zu tun hat.

1. Die Mutter aller Abfragesprachen:
SQL im 21. Jahrhundert
@MarkusWinand • @ModernSQL
http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

2. Die Mutter aller Abfragesprachen:
SQL im 21. Jahrhundert
@MarkusWinand • @ModernSQL
http://www.almaden.ibm.com/cs/people/chamberlin/sequel-1974.pdf

3. 1974 1992

4. SQL-92 — Tied to the Relational Idea

5. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)

6. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png

7. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
A B C
Atom image: https://commons.wikimedia.org/wiki/File:Stylised_atom_with_three_Bohr_model_orbits_and_stylised_nucleus.png

8. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
A B C

9. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
‣Schema independent of
processing purposes
‣“Normalization”
A B C

10. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
‣Schema independent of
processing purposes
‣“Normalization”
A B C C D B E

11. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
‣Schema independent of
processing purposes
‣“Normalization”
Relational Operations
‣Transform data for
each particular
processing purposes
‣JOIN, UNION, nesting, …
A B C C D B E A B C D E

12. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
‣Schema independent of
processing purposes
‣“Normalization”
Relational Operations
‣Transform data for
each particular
processing purposes
‣JOIN, UNION, nesting, …
A B C C D B E
A B C D E
A B E

13. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
‣Schema independent of
processing purposes
‣“Normalization”
Relational Operations
‣Transform data for
each particular
processing purposes
‣JOIN, UNION, nesting, …
A B C C D B E
A B C D E
A B E
C D E

14. SQL-92 — Tied to the Relational Idea
Relational Data Model
‣“Atomic” types (domain)
‣Schema independent of
processing purposes
‣“Normalization”
Relational Operations
‣Transform data for
each particular
processing purposes
‣JOIN, UNION, nesting, …
A B C C D B E
A B C D E
A B E
C D E

15. 1992 1999

16. https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

17. SQL:1999 — Escaping the Relational Cage
To say that these SQL:1999 extensions are mere
“extended interpretations” of the relational data model
is like saying that an intercontinental ballistic missile is
merely an “extended interpretation” of a spear.
https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

18. SQL:1999 — Escaping the Relational Cage
To say that these SQL:1999 extensions are mere
“extended interpretations” of the relational data model
is like saying that an intercontinental ballistic missile is
merely an “extended interpretation” of a spear.
With SQL/99 you can get the best of both worlds and
of course, you can get the worst of both worlds.
It’s up to the database practitioners to do the right thing.
https://www.wiscorp.com/DBMS_-_GreatNews-TheRelationalModelIsDead_-_paper_-_sam.pdf

19. SQL:1999 — Escaping the Relational Cage

20. Relational Model?
SQL:1999 — Escaping the Relational Cage

21. Relational Model?
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
I was as confused as anyone else

22. Relational Model?
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
?I was as confused as anyone else

23. Relational Model?
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light

24. Relational Model?
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

25. Relational Model?
‣Introduced rich types
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

26. Relational Model?
‣Introduced rich types
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
A
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

27. Relational Model?
‣Introduced rich types
‣arrays
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
A B
[ , ]
[ ]
[]
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

28. Relational Model?
‣Introduced rich types
‣arrays
‣Nested tables (multiset)
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
A B
[ , ]
[ ]
[]
C
C D
C D
C D
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

29. Relational Model?
‣Introduced rich types
‣arrays
‣Nested tables (multiset)
‣composite types (objects)
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
A B C D
[ , ] {x: ,
y: }
[ ] {x: ,
y: }
[] {x: ,
y: }
C D
C D
C D
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

30. Relational Model?
‣Introduced rich types
‣arrays
‣Nested tables (multiset)
‣composite types (objects)
Non-Relational Operations
‣Introduced recursive
queries that process
their own output
‣Transitive closure
Chris DateDate on Database: Writings 2000-2006
SQL:1999 — Escaping the Relational Cage
?I was as confused as anyone else
By the early 1990s, however,
I’d seen the light
Domains Can Contain Anything!

31. SQL:1999 — Recursion CREATE TABLE t (
id INTEGER,
parent INTEGER,
)

32. SQL:1999 — Recursion CREATE TABLE t (
id INTEGER,
parent INTEGER,
)

33. SQL:1999 — Recursion CREATE TABLE t (
id INTEGER,
parent INTEGER,
)

34. SQL:1999 — Recursion

35. SQL:1999 — Recursion

36. SQL:1999 — Recursion
SELECT t.id, t.parent
FROM t
WHERE t.id = ?

37. SQL:1999 — Recursion
SELECT t.id, t.parent
FROM t
WHERE t.id = ?
UNION ALL
SELECT t.id, t.parent
FROM t
WHERE t.parent = ?

38. SQL:1999 — Recursion
SELECT t.id, t.parent
FROM t
WHERE t.id = ?
UNION ALL
SELECT t.id, t.parent
FROM t
WHERE t.parent = ?

39. SQL:1999 — Recursion
SELECT t.id, t.parent
FROM t
WHERE t.id = ?
UNION ALL
SELECT t.id, t.parent
FROM t
WHERE t.parent = ?

40. SQL:1999 — Recursion
WITH RECURSIVE prev (id, parent) AS (
)
SELECT t.id, t.parent
FROM t
WHERE t.id = ?
UNION ALL
SELECT t.id, t.parent
FROM t
JOIN prev ON t.parent = prev.id
SELECT * FROM prev

41. SQL:1999 — Recursion
WITH RECURSIVE prev (id, parent) AS (
)
SELECT t.id, t.parent
FROM t
WHERE t.id = ?
UNION ALL
SELECT t.id, t.parent
FROM t
JOIN prev ON t.parent = prev.id
SELECT * FROM prev

42. SQL:1999 — Recursion
WITH RECURSIVE prev (id, parent) AS (
)
SELECT t.id, t.parent
FROM t
WHERE t.id = ?
UNION ALL
SELECT t.id, t.parent
FROM t
JOIN prev ON t.parent = prev.id
SELECT * FROM prev

43. SQL:1999 — Recursion
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 10.2 MariaDB
8.0 MySQL
8.4 PostgreSQL
3.8.3[0]
SQLite
7.0 DB2 LUW
11gR2 Oracle
2005 SQL Server
[0]
Only for top-level SELECT statements

45. LATERAL

46. SQL:1999 — LATERAL
[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar
Select-list sub-queries must be scalar[0]:
SELECT …
, (SELECT column_1
FROM t1
WHERE t1.x = t2.y
) AS c
FROM t2
…
(an atomic quantity that can hold only one value at a time[1])

47. SQL:1999 — LATERAL
[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar
Select-list sub-queries must be scalar[0]:
SELECT …
, (SELECT column_1
FROM t1
WHERE t1.x = t2.y
) AS c
FROM t2
…
(an atomic quantity that can hold only one value at a time[1])
✗, column_2
More than
one column?
Syntax error

48. SQL:1999 — LATERAL
[0] Neglecting row values and other workarounds here; [1] https://en.wikipedia.org/wiki/Scalar
Select-list sub-queries must be scalar[0]:
SELECT …
, (SELECT column_1
FROM t1
WHERE t1.x = t2.y
) AS c
FROM t2
…
(an atomic quantity that can hold only one value at a time[1])
✗, column_2
More than
one column?
Syntax error
}
More than
one row?
Runtime error!

49. SQL:1999 — LATERAL
Lateral derived tables lift both limitations and can be correlated:
SELECT …
, ldt.*
FROM t2
CROSS JOIN LATERAL (SELECT column_1, column_2
FROM t1
WHERE t1.x = t2.y
) AS ldt
…

50. SQL:1999 — LATERAL
Lateral derived tables lift both limitations and can be correlated:
SELECT …
, ldt.*
FROM t2
CROSS JOIN LATERAL (SELECT column_1, column_2
FROM t1
WHERE t1.x = t2.y
) AS ldt
…
“Derived table” means
it’s in the
FROM/JOIN clause

51. SQL:1999 — LATERAL
Lateral derived tables lift both limitations and can be correlated:
SELECT …
, ldt.*
FROM t2
CROSS JOIN LATERAL (SELECT column_1, column_2
FROM t1
WHERE t1.x = t2.y
) AS ldt
…
“Derived table” means
it’s in the
FROM/JOIN clause
Still
“correlated”

52. SQL:1999 — LATERAL
Lateral derived tables lift both limitations and can be correlated:
SELECT …
, ldt.*
FROM t2
CROSS JOIN LATERAL (SELECT column_1, column_2
FROM t1
WHERE t1.x = t2.y
) AS ldt
…
“Derived table” means
it’s in the
FROM/JOIN clause
Still
“correlated”
Regular join
semantics

53. SQL:1999 — LATERAL

54. SQL:1999 — LATERAL
‣ Top-N per group
inside a lateral derived table
FETCH FIRST (or LIMIT, TOP)
applies per row from left tables.

55. SQL:1999 — LATERAL
‣ Top-N per group
inside a lateral derived table
FETCH FIRST (or LIMIT, TOP)
applies per row from left tables.
FROM t
JOIN LATERAL (SELECT …
FROM …
WHERE t.c=…
ORDER BY …
LIMIT 10
) derived_table

56. SQL:1999 — LATERAL
‣ Top-N per group
inside a lateral derived table
FETCH FIRST (or LIMIT, TOP)
applies per row from left tables.
Add proper index
for Top-N query
https://use-the-index-luke.com/sql/partial-results/top-n-queries
FROM t
JOIN LATERAL (SELECT …
FROM …
WHERE t.c=…
ORDER BY …
LIMIT 10
) derived_table

57. SQL:1999 — LATERAL
‣ Top-N per group
inside a lateral derived table
FETCH FIRST (or LIMIT, TOP)
applies per row from left tables.
‣ Also useful to find most recent
news from several subscribed
topics (“multi-source top-N”).
Add proper index
for Top-N query
https://use-the-index-luke.com/sql/partial-results/top-n-queries
FROM t
JOIN LATERAL (SELECT …
FROM …
WHERE t.c=…
ORDER BY …
LIMIT 10
) derived_table

58. SQL:1999 — LATERAL
‣ Top-N per group
inside a lateral derived table
FETCH FIRST (or LIMIT, TOP)
applies per row from left tables.
‣ Also useful to find most recent
news from several subscribed
topics (“multi-source top-N”).
‣ Table function arguments
(TABLE often implies LATERAL)
Add proper index
for Top-N query
https://use-the-index-luke.com/sql/partial-results/top-n-queries
FROM t
JOIN TABLE (your_func(t.c))
FROM t
JOIN LATERAL (SELECT …
FROM …
WHERE t.c=…
ORDER BY …
LIMIT 10
) derived_table

59. SQL:1999 — LATERAL
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB
8.0.14 MySQL
9.3 PostgreSQL
SQLite
9.1 DB2 LUW
11gR1[0]
12cR1 Oracle
2005[1]
SQL Server
[0]
Undocumented. Requires setting trace event 22829.
[1]
LATERAL is not supported as of SQL Server 2016 but [CROSS|OUTER] APPLY can be used for the same effect.

60. 1999 2003

61. http://www.acm.org:80/sigmod/record/issues/0206/standard.pdf (via Wayback machine)

62. SQL:2003 — Schemaless & Analytical
Schemaless
‣Introduced XML
‣Non-uniform
documents in
a single column

63. SQL:2003 — Schemaless & Analytical
Schemaless
‣Introduced XML
‣Non-uniform
documents in
a single column
Later:
‣ JSON added with SQL:2016
‣ Proprietary JSON support:
‣ 2012: PostgreSQL
‣ 2014: Oracle
‣ 2015: MySQL
‣ 2016: SQL Server

64. SQL:2003 — Schemaless & Analytical
Analytical
‣Introduced
window functions
‣Accessing other rows
of the current result
Schemaless
‣Introduced XML
‣Non-uniform
documents in
a single column
Later:
‣ JSON added with SQL:2016
‣ Proprietary JSON support:
‣ 2012: PostgreSQL
‣ 2014: Oracle
‣ 2015: MySQL
‣ 2016: SQL Server

65. SQL:2003 — Schemaless & Analytical
Analytical
‣Introduced
window functions
‣Accessing other rows
of the current result
Schemaless
‣Introduced XML
‣Non-uniform
documents in
a single column
Later:
‣ Extended in SQL:2011
‣ Popular among “New SQLs”
‣ 2013: BigQuery, Hive
‣ 2014: Impala
‣ 2015: Spark SQL
‣ 2016: NuoDB, MemSQL,
Cockroach DB, VoltDB
Later:
‣ JSON added with SQL:2016
‣ Proprietary JSON support:
‣ 2012: PostgreSQL
‣ 2014: Oracle
‣ 2015: MySQL
‣ 2016: SQL Server

66. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
SQL:2003 — Analytical

67. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical

68. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
+10

69. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
+10
+30
+20
+70
+40
+20

70. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

71. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

72. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

73. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

74. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

75. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

76. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

77. SELECT id, value
FROM t
id
1
2
3
4
5
6
value
+10
+20
-10
+50
-30
-20
bal
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

78. SELECT id, value
FROM t
id value bal
1 +10
2 +20
3 -10
4 +50
5 -30
6 -20
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

79. SELECT id, value
FROM t
id value bal
1 +10
2 +20
3 -10
4 +50
5 -30
6 -20
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

80. SELECT id, value
FROM t
id value bal
1 +10
2 +20
3 -10
4 +50
5 -30
6 -20
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

81. SELECT id, value
FROM t
id value bal
1 +10
2 +20
3 -10
4 +50
5 -30
6 -20
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

82. SELECT id, value
FROM t
id value bal
1 +10
2 +20
3 -10
4 +50
5 -30
6 -20
SQL:2003 — Analytical
ORDER BY id
ROWS BETWEEN
UNBOUNDED PRECEDING
AND CURRENT ROW
, SUM(value)
OVER (
) bal
+10
+30
+20
+70
+40
+20

83. SQL:2003 — Analytical
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 10.2 MariaDB
8.0 MySQL
8.4 PostgreSQL
3.25.0 SQLite
7.0 DB2 LUW
8i Oracle
2005[0]
2012 SQL Server
[0]
No framing

84. SQL:2003 — Analytical (Median)

85. SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
SQL:2003 — Analytical (Median)

86. SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
Number rows
SQL:2003 — Analytical (Median)

87. SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
Number rows
Pick middle one
SQL:2003 — Analytical (Median)

88. SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
Number rows
Pick middle one
SQL:2003 — Analytical (Median)

89. SELECT d1.val
FROM data d1
JOIN data d2
ON (d1.val < d2.val
OR (d1.val=d2.val AND d1.id<d2.id))
GROUP BY d1.val
HAVING count(*) =
(SELECT FLOOR(COUNT(*)/2)
FROM data d3)
Number rows
Pick middle one
SQL:2003 — Analytical (Median)

90. SQL:2003 — Analytical (Median)

91. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
SQL:2003 — Analytical (Median)

92. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Median
SQL:2003 — Analytical (Median)

93. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Median
Which value?
SQL:2003 — Analytical (Median)

94. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
SQL:2003 — Analytical (Median)

95. SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

96. 1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

97. 1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

98. 1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

99. 1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC(0.5)
SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

100. 1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC(0.5)
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_CONT
PERCENTILE_DISC(0.5)
SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

101. 1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_DISC(0.5)
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_CONT
PERCENTILE_DISC(0.5)
0 0.25 0.5 0.75 1
1
2
3
4
PERCENTILE_CONT(0.5)
PERCENTILE_DISC(0.5)
SELECT PERCENTILE_DISC(0.5) WITHIN GROUP (ORDER BY val)
FROM data
Two variants:
‣for discrete values
(categories)
‣for continuous values
(linear interpolation)
SQL:2003 — Analytical (Median)

102. SQL:2003 — Analytical (Median)
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 10.3.7[0]
MariaDB
MySQL
9.4 PostgreSQL
SQLite
DB2 LUW
9iR1 Oracle
2012[0]
SQL Server
[0]
Only as window function (OVER required).

103. 2003 2016

104. SQL:2016 — JSON
http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip

105. SQL:2016 — JSON
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip

106. SQL:2016 — JSON
id a1
42 foo
43 bar
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
http://standards.iso.org/ittf/PubliclyAvailableStandards/c067367_ISO_IEC_TR_19075-6_2017.zip

107. SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r
SQL:2016 — JSON_TABLE
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
id a1
42 foo
43 bar

108. SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r
SQL:2016 — JSON_TABLE
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
id a1
42 foo
43 bar
Bind
Parameter

109. SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r
SQL:2016 — JSON_TABLE
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
id a1
42 foo
43 bar
SQL/JSON Path
‣ Query language to
select elements from
a JSON document
‣ Defined in the
SQL standard
Bind
Parameter

110. SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r
SQL:2016 — JSON_TABLE
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
id a1
42 foo
43 bar
SQL/JSON Path
‣ Query language to
select elements from
a JSON document
‣ Defined in the
SQL standard
Bind
Parameter

111. SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r
SQL:2016 — JSON_TABLE
[
{
"id": 42,
"a1": "foo"
},
{
"id": 43,
"a1": "bar"
}
]
id a1
42 foo
43 bar
SQL/JSON Path
‣ Query language to
select elements from
a JSON document
‣ Defined in the
SQL standard
Bind
Parameter

112. SQL:2016 — JSON_TABLE — Use Case
SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r

113. SQL:2016 — JSON_TABLE — Use Case
SELECT *
FROM JSON_TABLE
( ?
, '$[*]'
COLUMNS
( id INT PATH '$.id'
, a1 VARCHAR(…) PATH '$.a1'
)
) r
INSERT INTO target_table

114. SQL:2016 — JSON_TABLE
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB
8.0 MySQL
PostgreSQL
SQLite
DB2 LUW
12cR1 Oracle
SQL Server

116. MATCH_RECOGNIZE
(Row Pattern Recognition)

117. Time
30 minutes
Example: Logfile
SQL:2016 — Pattern Matching

118. Example: Logfile
Time
30 minutes
Session 1 Session 2
Session 3
Session 4
SQL:2016 — Pattern Matching

119. Time
30 minutes
SQL:2016 — Pattern Matching

120. SELECT count(*) sessions, avg(duration) avg_duration
FROM (SELECT MAX(ts) - MIN(ts) duration
FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no
FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' )
OVER( ORDER BY ts )
+ INTERVAL '30' minute
THEN 1
END grp_start
FROM log
) tagged
) numbered
GROUP BY session_no
) grouped
Time
30 minutes
SQL:2016 — Pattern Matching

121. SELECT count(*) sessions, avg(duration) avg_duration
FROM (SELECT MAX(ts) - MIN(ts) duration
FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no
FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' )
OVER( ORDER BY ts )
+ INTERVAL '30' minute
THEN 1
END grp_start
FROM log
) tagged
) numbered
GROUP BY session_no
) grouped
Time
30 minutes
Start-of-group
tags
SQL:2016 — Pattern Matching

122. SELECT count(*) sessions, avg(duration) avg_duration
FROM (SELECT MAX(ts) - MIN(ts) duration
FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no
FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' )
OVER( ORDER BY ts )
+ INTERVAL '30' minute
THEN 1
END grp_start
FROM log
) tagged
) numbered
GROUP BY session_no
) grouped
Time
30 minutes
Start-of-group
tags
SQL:2016 — Pattern Matching

123. SELECT count(*) sessions, avg(duration) avg_duration
FROM (SELECT MAX(ts) - MIN(ts) duration
FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no
FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' )
OVER( ORDER BY ts )
+ INTERVAL '30' minute
THEN 1
END grp_start
FROM log
) tagged
) numbered
GROUP BY session_no
) grouped
Time
30 minutes
number
sessions
SQL:2016 — Pattern Matching

124. SELECT count(*) sessions, avg(duration) avg_duration
FROM (SELECT MAX(ts) - MIN(ts) duration
FROM (SELECT ts, SUM(grp_start) OVER(ORDER BY ts) session_no
FROM (SELECT ts, CASE WHEN ts >= LAG( ts, 1, DATE'1900-01-01' )
OVER( ORDER BY ts )
+ INTERVAL '30' minute
THEN 1
END grp_start
FROM log
) tagged
) numbered
GROUP BY session_no
) grouped
Time
30 minutes
number
sessions
2222 2 33 3 44 42 3 4
1
SQL:2016 — Pattern Matching

125. .S*
SQL:2016 — Pattern Matching

126. .S*
Regular Expression
SQL:2016 — Pattern Matching

127. any character non-white space
.S*
{
Rail track diagram by regexper.com
SQL:2016 — Pattern Matching

128. any character non-white space
.S*
{
{ Rail track diagram by regexper.com
SQL:2016 — Pattern Matching

129. any character non-white spaceany character non-white space
.S*
{
{{
Rail track diagram by regexper.com
SQL:2016 — Pattern Matching

130. SELECT COUNT(*) sessions, AVG(duration) avg_duration
FROM log
MATCH_RECOGNIZE(
ORDER BY ts
MEASURES
LAST(ts) - FIRST(ts) AS duration
ONE ROW PER MATCH
PATTERN ( any cont* )
DEFINE cont AS ts < PREV(ts)
+ INTERVAL '30' MINUTE
) t
Time
30 minutes
Oracle doesn’t support avg on intervals — query doesn’t work as shown
SQL:2016 — Pattern Matching

131. SELECT COUNT(*) sessions, AVG(duration) avg_duration
FROM log
MATCH_RECOGNIZE(
ORDER BY ts
MEASURES
LAST(ts) - FIRST(ts) AS duration
ONE ROW PER MATCH
PATTERN ( any cont* )
DEFINE cont AS ts < PREV(ts)
+ INTERVAL '30' MINUTE
) t
Time
30 minutes
define
continued
Oracle doesn’t support avg on intervals — query doesn’t work as shown
SQL:2016 — Pattern Matching

132. SELECT COUNT(*) sessions, AVG(duration) avg_duration
FROM log
MATCH_RECOGNIZE(
ORDER BY ts
MEASURES
LAST(ts) - FIRST(ts) AS duration
ONE ROW PER MATCH
PATTERN ( any cont* )
DEFINE cont AS ts < PREV(ts)
+ INTERVAL '30' MINUTE
) t
Time
30 minutes
any number
of “cont”
rows
Oracle doesn’t support avg on intervals — query doesn’t work as shown
SQL:2016 — Pattern Matching

133. SELECT COUNT(*) sessions, AVG(duration) avg_duration
FROM log
MATCH_RECOGNIZE(
ORDER BY ts
MEASURES
LAST(ts) - FIRST(ts) AS duration
ONE ROW PER MATCH
PATTERN ( any cont* )
DEFINE cont AS ts < PREV(ts)
+ INTERVAL '30' MINUTE
) t
Time
30 minutes
Very much
like GROUP BY
Oracle doesn’t support avg on intervals — query doesn’t work as shown
SQL:2016 — Pattern Matching

134. SELECT COUNT(*) sessions, AVG(duration) avg_duration
FROM log
MATCH_RECOGNIZE(
ORDER BY ts
MEASURES
LAST(ts) - FIRST(ts) AS duration
ONE ROW PER MATCH
PATTERN ( any cont* )
DEFINE cont AS ts < PREV(ts)
+ INTERVAL '30' MINUTE
) t
Time
30 minutes
Very much
like SELECT
Oracle doesn’t support avg on intervals — query doesn’t work as shown
SQL:2016 — Pattern Matching

135. SQL:2016 — Pattern Matching
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
MariaDB
MySQL
PostgreSQL
SQLite
DB2 LUW
12cR1 Oracle
SQL Server

136. 2011 2016

137. SQL:2011 — Time Travelling
http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf

138. SQL:2011 — Time Travelling
Application Versioning
‣Dedicated syntax added
‣When did something
happen in the
real world?
http://cs.ulb.ac.be/public/_media/teaching/infoh415/tempfeaturessql2011.pdf

139. SQL:2011 — Time Travelling
Application Versioning
‣Dedicated syntax added
‣When did something
happen in the
real world?
New syntax (excerpt)
‣ FOR PORTION OF in
UPDATE and DELETE
‣ WITHOUT OVERLAPS in UNIQUE
constraints & PRIMARY KEYS
‣ [IMMEDIATELY] PRECEDES,
OVERLAPS in WHERE,HAVING,…

140. SQL:2011 — Time Travelling
System Versioning
‣Fully automatic and
(almost) transparent
‣When did we learn
about something
Application Versioning
‣Dedicated syntax added
‣When did something
happen in the
real world?
New syntax (excerpt)
‣ FOR PORTION OF in
UPDATE and DELETE
‣ WITHOUT OVERLAPS in UNIQUE
constraints & PRIMARY KEYS
‣ [IMMEDIATELY] PRECEDES,
OVERLAPS in WHERE,HAVING,…

141. SQL:2011 — Time Travelling
System Versioning
‣Fully automatic and
(almost) transparent
‣When did we learn
about something
Application Versioning
‣Dedicated syntax added
‣When did something
happen in the
real world?
New syntax (excerpt)
‣ FOR PORTION OF in
UPDATE and DELETE
‣ WITHOUT OVERLAPS in UNIQUE
constraints & PRIMARY KEYS
‣ [IMMEDIATELY] PRECEDES,
OVERLAPS in WHERE,HAVING,…
Transparent changes,
new syntax for queries
‣ INSERT, UPDATE & DELETE
use the system time
automatically
‣ SELECT can use FOR
SYSTEM_TIME AS OF

142. SQL:2011 — Application Time Periods

143. SQL:2011 — Application Time Periods
CREATE TABLE t (
...
, from TIMESTAMP(9)
, till TIMESTAMP(9)
, PERIOD FOR a (from, till)
)

144. SQL:2011 — Application Time Periods
ID Data From Till
1 X 10:00:00 12:00:00
INSERT t (id, data, from. , till. )
VALUES ( 1, 'X', '10:00:00', '12:00:00')

145. SQL:2011 — Application Time Periods
UPDATE t
FOR PORTION OF a FROM '10:30:00' TO '11:30:00'
SET DATA = 'Y'
ID Data From Till
1 X 10:00:00 10:30:00
1 Y 10:30:00 11:30:00
1 X 11:30:00 12:00:00
ID Data From Till
1 X 10:00:00 12:00:00
INSERT t (id, data, from. , till. )
VALUES ( 1, 'X', '10:00:00', '12:00:00')

146. SQL:2011 — Application Time Periods
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB
MySQL
PostgreSQL
SQLite
10.5 DB2 LUW
Oracle
SQL Server

147. SQL:2011 — Application Time Periods
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 MariaDB
MySQL
PostgreSQL
SQLite
10.5 DB2 LUW
Oracle
SQL Server

148. SQL:2011 — System Versioning

149. SQL:2011 — System Versioning
CREATE TABLE t (
...
, from TIMESTAMP(9) GENERATED ALWAYS
AS ROW START
, till TIMESTAMP(9) GENERATED ALWAYS
AS ROW END
, PERIOD FOR SYSTEM_TIME (from, till)
) WITH SYSTEM VERSIONING

150. SQL:2011 — System Versioning

151. SQL:2011 — System Versioning
INSERT INTO t (id, data)
VALUES (1 , 'X' )
id data from till
1 X 10:00

152. SQL:2011 — System Versioning
INSERT INTO t (id, data)
VALUES (1 , 'X' )
id data from till
1 X 10:00
UPDATE t
SET data = 'Y'
WHERE id = 1
id data from till
1 X 10:00 11:00
1 Y 11:00

153. SQL:2011 — System Versioning
INSERT INTO t (id, data)
VALUES (1 , 'X' )
id data from till
1 X 10:00
UPDATE t
SET data = 'Y'
WHERE id = 1
id data from till
1 X 10:00 11:00
1 Y 11:00
DELETE FROM t
WHERE id = 1
id data from till
1 X 10:00 11:00
1 Y 11:00 12:00

154. SQL:2011 — System Versioning
id data from till
1 X 10:00 11:00
1 Y 11:00 12:00

155. SQL:2011 — System Versioning
id data from till
1 X 10:00 11:00
1 Y 11:00 12:00
SELECT *
FROM t
id data from till

156. SQL:2011 — System Versioning
id data from till
1 X 10:00 11:00
1 Y 11:00 12:00
SELECT *
FROM t
id data from till
SELECT *
FROM t
FOR SYSTEM_TIME AS OF
TIMESTAMP'…10:30:00'
id data from till
1 X 10:00 11:00

157. SQL:2011 — System Versioning
1999
2001
2003
2005
2007
2009
2011
2013
2015
2017
5.1 10.3 MariaDB
MySQL
PostgreSQL
SQLite
10.1 DB2 LUW
10gR1[0]
11gR1[1]
Oracle
2016 SQL Server
[0]
Short term using Flashback.
[1]
Flashback Archive. Proprietery syntax.

158. https://webstore.iec.ch/publication/59685

159. A lot has
happened
since SQL-92

160. SQL has evolved
beyond
the relational idea
A lot has
happened
since SQL-92

161. SQL has evolved
beyond
the relational idea
If you use SQL for
CRUD operations only,
you are doing it wrong
A lot has
happened
since SQL-92

162. SQL has evolved
beyond
the relational idea
If you use SQL for
CRUD operations only,
you are doing it wrong
A lot has
happened
since SQL-92
https://modern-sql.com
@ModernSQL by @MarkusWinand
Training:
https://winand.at/