Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Functional Database Strategies at Scala Bay

944 Aufrufe

Veröffentlicht am

As developers using a functional programming language, we embrace immutable collections and pure functions in our code. However, we often have separate rules for how we work with our persistent collections in databases. In this talk we'll look at why & how to take a more functional approach to persistence with databases, from managing the connections, using immutable table rows, and considering event-source persistence

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

Functional Database Strategies at Scala Bay

  1. 1. Functional Database Strategies Scala Bay • March 20, 2017
  2. 2. Hello Scala Bay Hello Scala Bay
  3. 3. YesQL
  4. 4. by Jason Swartz @swartzrock
  5. 5. Functional Database Strategies
  6. 6. Step One Buy A Functional Database
  7. 7. Silly! Databases Are mutable, not functional!
  8. 8. Well? Aren’tthey?
  9. 9. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  10. 10. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  11. 11. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  12. 12. Remember That Your Database is mutable.
  13. 13. Avoid Sharing Your State
  14. 14. ● Avoid shared mutable SESSIONS ● Avoid shared mutable CURSORS ● Mutating state? Cool! But MODEL IT FIRST ● Execute state changes at THE EDGE Safe Database Interactions
  15. 15. Doobie A Typelevel Project
  16. 16. Are these steps? Or a Monad?
  17. 17. val xa: Transactor[Task] = DriverManagerTransactor[Task]( "org.postgresql.Driver", "jdbc:postgresql://localhost/issues", "u", "p" )
  18. 18. case class Issue(id: Int, issueId: Int, title: String, assignee: Int, status: String) val issue: Issue = sql""" select id, issue_id, title, assignee, status from issues where issue_id = ${req.issue_id} order by id desc limit 1 """ .query[Issue] .unique .transact(xa) .unsafePerformSync
  19. 19. ● ResultSet to Case Class Conversion ● Async IO modeling with Task (scalaz version) ● Think in SQL, not an ORM ● Plain SQL with sql"" interpolation ● SQL functions supported Why Doobie?
  20. 20. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  21. 21. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  22. 22. Let’s Talk About Immutable Tables
  23. 23. Create-Read- Update-Delete
  24. 24. GET /issues GET /issues/{id} POST /issues PUT /issues/{id} Issue Endpoints
  25. 25. How Do These Events Affect The Database?
  26. 26. 1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
  27. 27. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
  28. 28. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  29. 29. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Not Bad.
  30. 30. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  31. 31. +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  32. 32. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Do You Know How We Got Here?
  33. 33. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+ Why is ‘assignee’ NULL?
  34. 34. Mutable Table Rows Lose History
  35. 35. Immutable Table Rows Keep Their History
  36. 36. Let’s Try To Lock Down Our State
  37. 37. 1. POST /issues title=’Config ELB’ +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+
  38. 38. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+
  39. 39. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  40. 40. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  41. 41. 1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  42. 42. Tables Are Mutable, But Table Rows Should Be Immutable
  43. 43. In Other Words, Tables Should Be Append-Only
  44. 44. How Do You Make An Append-Only Table?
  45. 45. One: Don’t Let Your DB User Make Changes
  46. 46. Grant select, insert on issues to my-db-user; -- tested on Postgresql
  47. 47. Thank You! Goodbye!
  48. 48. Two: Pick The Right Columns
  49. 49. 1. GET /issues/1 +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:19 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | NULL | true | +------+-------------+------------+----------+-------+
  50. 50. create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false )
  51. 51. 1. GET /issues/1 +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ +------+-------------+------------+------------+----------+-------+ | 4 | 09-18 18:24 | 1 | Config ELB | NULL | true | +------+-------------+------------+------------+----------+-------+
  52. 52. select * from issues where issue_id = :issue_id order by id desc limit 1;
  53. 53. What Defines An Issue, The Version or the Entity?
  54. 54. ISSUES +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+
  55. 55. create table issues ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), title text, assignee int, done boolean default false ) create table notifications ( id serial, created timestamp default now(), issue_row_id int references issues )
  56. 56. ISSUES +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ NOTIFICATIONS +------+-------------+--------------+ | id | created | issue_row_id | +------+-------------+--------------+ | 1 | 09-18 19:13 | 2 | +------+-------------+--------------+
  57. 57. ISSUES +------+-------------+------------+------------+----------+-------+ | id | created | issue_id | title | assignee | done | +------+-------------+------------+------------+----------+-------+ | 1 | 09-18 18:13 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ | 2 | 09-18 18:16 | 1 | Config ELB | 10 | false | +------+-------------+------------+------------+----------+-------+ | 3 | 09-18 18:19 | 1 | Config ELB | NULL | false | +------+-------------+------------+------------+----------+-------+ NOTIFICATIONS +------+-------------+--------------+ | id | created | issue_row_id | +------+-------------+--------------+ | 1 | 09-18 19:13 | 2 | +------+-------------+--------------+
  58. 58. Issue Notifications Are Immutable In The Database
  59. 59. That’s The Basics Of Immutability In Table Rows
  60. 60. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  61. 61. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  62. 62. SQL:2003 expands Groups into Windows
  63. 63. Works Great In Postgresql
  64. 64. ● Aggregation functions, eg sum(), rank(), avg() ● Window definitions with over() ● Grouping with partition by, order Window Functions
  65. 65. +------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
  66. 66. +------+------------+------------+----------+-------+ | id | issue_id | title | assignee | done | +------+------------+------------+----------+-------+ | 1 | 1 | Config ELB | NULL | false | +------+------------+------------+----------+-------+ | 2 | 1 | Config ELB | 10 | false | +------+------------+------------+----------+-------+ | 3 | 2 | Bug to fix | 11 | false | +------+------------+------------+----------+-------+ | 4 | 2 | Bug to fix | 11 | true | +------+------------+------------+----------+-------+
  67. 67. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  68. 68. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  69. 69. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  70. 70. with latest_issues as ( select *, row_number() over ( partition by issue_id order by id desc ) from issues where created > now() - interval '7 day' ) select * from latest_issues where row_number = 1
  71. 71. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  72. 72. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  73. 73. +------+------------+------------+----------+-------+------------+ | id | issue_id | title | assignee | done | row_number | +------+------------+------------+----------+-------+------------+ | 1 | 1 | Config ELB | NULL | false | 2 | +------+------------+------------+----------+-------+------------+ | 2 | 1 | Config ELB | 10 | false | 1 | +------+------------+------------+----------+-------+------------+ | 3 | 2 | Bug to fix | 11 | false | 2 | +------+------------+------------+----------+-------+------------+ | 4 | 2 | Bug to fix | 11 | true | 1 | +------+------------+------------+----------+-------+------------+
  74. 74. That Was Window Functions
  75. 75. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  76. 76. 1. Functional DB’s In Scala a. Doobie 2. Functional DB’s In The Database a. Immutable Rows b. Window Functions c. Events, Not State Agenda
  77. 77. You Know How To Maintain State
  78. 78. Do We Still Need State?
  79. 79. Let’s Talk About Event-Sourcing
  80. 80. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+
  81. 81. 1. POST /issues title=’Config ELB’ 2. PUT /issues/1 assignee=10 3. PUT /issues/1 done=true +------+-------------+------------+----------+-------+ | id | updated | title | assignee | done | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:13 | Config ELB | NULL | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:16 | Config ELB | 10 | false | +------+-------------+------------+----------+-------+ | 1 | 09-18 18:24 | Config ELB | 10 | true | +------+-------------+------------+----------+-------+ Events States
  82. 82. Now We’re Storing Events, Not States
  83. 83. create table issue_events ( id serial, created timestamp default now(), issue_id int default nextval(‘iseq’), originator text, payload text )
  84. 84. 1. POST /issue/1/event ‘Originator: 4a48239-8a..’ payload=’<Update val=”done=true”>’ +----+-------------+----------+------------+---------+ | id | created | issue_id | originator | payload | +----+-------------+----------+------------+---------+ | 14 | 09-18 18:50 | 1 | 4a482... | <...> | +----+-------------+----------+------------+---------+
  85. 85. Create Events And Simulate The State
  86. 86. 1. Create-Issue Issue(“Config ELB”, null, false); Real Events Virtual States
  87. 87. 1. Create-Issue 2. Assign-Issue Issue(“Config ELB”, 10, false); Real Events Virtual States
  88. 88. 1. Create-Issue 2. Assign-Issue 3. Complete-Issue Issue(“Config ELB”, 10, true); Real Events Virtual States
  89. 89. So Why Use Event-Sourcing?
  90. 90. 1. High Write Performance 2. Potential for Command/Query Separation 3. Auditable 4. Replayable 5. Undo-able 6. Monitorable Reasons For Event-Sourcing
  91. 91. It’s Like Having Control Over The Versions Of Your State Changes
  92. 92. It’s Like Having Control Over The Versions Of Your Data
  93. 93. It’s Like Git For Your Data
  94. 94. 1. Frankly, It’s Weird 2. Requires Events. No Events, No Event-Sourcing. 3. As Of March 2017, It’s Still Non-Standard Reasons Against Event-Sourcing
  95. 95. Wait! We’re Scala developers!
  96. 96. Who Cares About Being Non-Standard?
  97. 97. That About Sums Up Event Sourcing
  98. 98. That About Sums Up Database Interactions
  99. 99. Okay, Actually That’s The Entire Talk Unless There’s More Time
  100. 100. Functional Database Strategies Scala Bay • March 20, 2017
  101. 101. by Jason Swartz @swartzrock
  102. 102. Thank You For Attending
  103. 103. Fin
  104. 104. THIS SPACE LEFT BLANK

×