Kevin Grittner Concurrent database transactions can interact in surprising ways. Users currently have various methods to deal with these issues within PostgreSQL, through: - the choice between two transaction isolation levels, - table locks programmed at the application level, - the use of additional programmed updates, - automatic retry of transactions rolled back due to conflicts, - externally programmed locking techniques, or - (most often) some combination of the above. The underlying issues as well as current techniques will be covered, and a discussion of current efforts to add a third transaction isolation level which, when used consistently, would eliminate the need for any of the above except for the transaction retries. The new mode would be of particular interest to those in shops with many programmers working with a large number of queries, particularly if ad hoc queries or generated queries are possible. (Common sources of generated queries are ORMs or other frameworks which generate queries dynamically based on user input.) In such environments it can currently be extremely difficult to ensure that no anomalies from concurrent transactions cause violations of complex data integrity rules enforced through triggers or other application programming, or that no read-only queries see views of data which are inconsistent with some sequence of transaction commits.

5. Failure of multi-row integrity checks

7. Limit concurrency with connection pool

9. SELECT FOR UPDATE

12. Integrity (business rules enforced)

13. Shared (concurrent) access

14. Security (by user and stored object)

15. Flexible access to data (using relational algebra and relational calculus)

16. Speed (through cost-based optimization of queries)

17. Scalability

18. Recoverability

19. Availability Why use a database, rather than a spreadsheet, a word processing document, or a shoe box full of scraps of paper?

22. Consistency : A transaction can only commit if it leaves the database in a consistent state. Database constraints, triggers, and client code may all participate in enforcing these business rules.

23. Isolation : A transaction can be written as though it will be run by itself, before or after every other transaction.

24. Durability : A guarantee that once a transaction commits, its work is visible to every subsequent transaction, until overwritten by another transaction. System failures after a successful commit will not cause data loss.

25. Isolation Levels, Generally The problem with truly serializable transactions is that enforcing this behavior at run time can be expensive. Not only is there the direct cost of tracking information needed to prevent anomalies, but any practical technique involves rolling back transactions which have already completed some of their work – requiring these transactions to start over from the beginning. Most practical techniques also result in some level of blocking, where one transaction must stop and wait until another transaction completes. Because of these costs, various other (less strict) isolation levels have been developed and used. In all cases they reduce the integrity guarantees automatically provided by the database in exchange for faster performance, with less blocking and fewer rollbacks. This doesn't mean that correctness is lost; only that the database engine itself no longer guarantees it, and the onus is on the application developers to recognize vulnerabilities and explicitly add protection.

27. Non-repeatable read: prohibited at REPEATABLE READ or stricter.

28. Phantom rows: prohibited at SERIALIZABLE. Transaction isolation levels are defined, in part, by certain phenomena which are not allowed to occur. It is not required that any of these phenomena do occur at a less strict level than that at which they are prohibited.

29. SQL Standard SERIALIZABLE Although the very first version of the SQL standard defined the SERIALIZABLE transaction isolation level in terms of phenomena which must not occur at that level, in the 1999 version and later the definition is: “ The execution of concurrent SQL-transactions at isolation level SERIALIZABLE is guaranteed to be serializable. A serializable execution is defined to be an execution of the operations of concurrently executing SQL-transactions that produces the same effect as some serial execution of those same SQL-transactions. A serial execution is one in which each SQL-transaction executes to completion before the next SQL-transaction begins.” There is also a note which points out that as a consequence of this definition, it is not possible for any of the phenomena which define other transaction isolation levels to occur at the SERIALIZABLE isolation level.

30. Specific Isolation Levels Below are the transaction isolation levels defined by the SQL standard, and those implemented in PostgreSQL, in descending order of strictness. SQL Standard PostgreSQL Serializable Serializable (actually uses Snapshot) Repeatable Read Read Committed Read Committed (uses mixed snapshots after blocking on update conflict) Read Uncommitted The standard makes all levels below serializable optional – as long as a request for a given level actually provides the requested level or something more strict.

32. Block conflicting access – read locks block writes; write locks block both reads and writes.

33. Reads locks must be ”predicate locks” – to conflict with writes of what they would have read, had it existed at the time of the read.

34. Circular blocking causes ”deadlock” – a form of serialization failure.

36. Write locks only block other writes; reads neither block nor are blocked.

37. The commit of a transaction with a write lock rolls back transactions blocked by it – a form of serialization failure.

38. A transaction which reads data which is modified by a concurrent transaction is the one which appears to have committed first; circular read-write dependencies silently produce incorrect results.

42. No writes are allowed.

43. Work done so far by other transactions is visible, even though it might not ever get committed.

44. Indexes can't be used unless they are unique, so that an index scan can find its position if the page it's on is modified.

46. Write locks are held until transaction commit.

47. No read locks are automatically taken, except at the table level.

48. When blocking on a write lock, if the other transaction commits, roll back with a serialization failure.

50. Write locks are held until transaction commit.

51. No read locks are automatically taken, except at the table level.

52. When blocking on a write lock, if the other transaction commits, find the new version of the row, see if it still matches our search criteria, and use it or ignore it based on the result. This results in using some data from outside the snapshot otherwise used for the statement.

53. Serialization Anomalies in Snapshot Isolation T0 and TN both overlap T1, and may be the same transaction. T0 tries to read data matching what T1 writes, but can't see T1's writes because they are concurrent. T1 tries to read data matching what TN writes, but can't see TN's writes because they are concurrent. TN commits before T1 does. If T0 and TN are separate transactions, one must depend on the other in some way. The above conditions must all be met for a set of transactions running at snapshot isolation level to cause an anomaly. T1 (pivot) TN T0

54. Simple Write Skew Two concurrent transactions can generate a result which could not have occurred if either committed before the start of the other. This is known as ”write skew.” Transaction 1 Transaction 2 UPDATE fruit SET name = 'Chinese gooseberry' WHERE name = 'kiwifruit'; UPDATE fruit SET name = 'kiwifruit' WHERE name = 'Chinese gooseberry'; COMMIT; COMMIT; If either had run first, there could not be both 'kiwi fruit' and 'Chinese gooseberry' in the database; under snapshot isolation, these values can be swapped by concurrent transactions without generating an error.

55. Multi-Row Integrity Constraints Integrity checks involving mupliple rows can't be reliably enforced under snapshot isolation without programming to introduce additional conflicts. Transaction 1 – adding an offense Transaction 2 – deleting a statute Read the statute table to see if the statue cite exists with an effective date on or before the offense date and without an expiration date before the offense date. Read the offense table to see if there are any offenses with a matching statute cite and an offense date covered by the statute's effective and expiration dates. If not exists, throw error; else insert the offense record. If there is, throw an error; else delete the statute record. Commit. Commit. This leaves us with an offense for which the related statute is missing.

57. At least one doctor is on call for each shift.

58. An offense matches some statute which was in effect on the offense date.

59. Warrant status on person is removed when last warrant is cancelled. There's a very wide variety of integrity constraints which can fail to be enforced during concurrent updates, even when implemented by database triggers in ”serializable” transactions. These may work properly a high percentage of the time, but silently fail to enforce integrity or fail to properly update summary data on an occasional basis. Basically, any validation which requires a transaction to look at rows beyond those which it has itself written can't be trusted to do the right thing under snapshot isolation without explicit help from the programmer. Conditional updates of derived data like status codes can misfire under the same conditions. Some examples:

60. ” Consistent With” versus Actual Order The SQL standard requires that the behavior of serialiable transactions be consistent with some serial execution of the transactions. There is no requirement that the order match transaction start or commit order (or anything else). Transaction 1 Transaction 2 Update the deposit date in the control table. Read deposit date from control table and use it to insert a receipt. Commit. Commit. With snapshot isolation, even though Transaction 2 committed first, Transaction 1 appears to have executed first, since its receipt is inserted with a date based on the state of the control table prior to the actions of Transaction 2.

61. Seeing Things Snapshot anomalies can cause results to be returned to the client which are not consistent with any serial execution of the transactions. Transaction 1 Transaction 2 Transaction 3 Update the deposit date in the control table. Read deposit date from control table and use it to insert a receipt. Commit. Select receipts for the deposit date just closed. Commit. Commit. Transaction 1's receipt won't show in the deposit report. Adding a read-only transaction to a previously safe mix has caused a serialization anomaly!

62. DON'T PANIC!

66. Update summary data, or some proxy table, to create write conflicts for failure cases. This is called materializing a conflict.

67. SELECT FOR UPDATE can provide row-level locks. This is called promoting a conflict.

68. Explicit table locks can be taken at the beginning of the transaction.

73. Reaction: When data is found which violates the rules the triggers are attempting to enforce, an effort can be made to ”hunt down” the offending patterns of transactions based on the data involved.

74. Static analysis: Either manually or using software, search the application code for transactions and build a graph of all dangerous interactions among them. This is too resource-intensive to do on a frequent basis, but if code is relatively static, it can be done periodically, even in environments with a large and complex code base. It does tend to generate a large proportion of false positives, so all flagged combinations must be evaluated, and mitigation explicitly programmed where needed.

76. Materialize the conflict: Rather than just selecting the sum of the account balances and checking that the total is OK, save it to a total_balance column in a table related to the person. Conflicting updates are now recognized.

77. Promote the conflict: SELECT FOR UPDATE will create row level locks much like writes as you read data. This will block updates and will cause serialization failure if necessary to avoid anomalies.

78. Lock tables: You can lock entire tables explicitly, although this must be done at the very beginning of the transaction to be useful in most cases. Locks are held until the end of the transactions. Blocking and deadlocks (a form of serialization failure) are possible. Having identified the combinations of transactions which, when run concurrently, can cause anomalies, there are several ways to control the problem.

79. Introducing Serializable Snapshot Isolation (SSI) In recent years there has been much study of the precise conditions under which serialization anomalies can occur in transactions run using snapshot isolation. This culminated, in 2008 and 2009, in papers on a new technique which provides truly serializable transactions in an MVCC database based on snapshot isolation. This technique aims to eliminate the need for explicit coding to prevent anomalies; rather it automatically prevents them, simplifying the development effort and supporting ad hoc queries. Basically, it works by allowing snapshot isolation to run as usual, with writes not blocking reads and reads not blocking anything, and monitors the read-write dependencies among transactions to spot dangerous structures which could cause serialization anomalies. When a dangerous structure is found, a transaction is forced to roll back with a serialization error.

80. Work In Progress The Wisconsin State Courts system has many databases, the most complex of which has 375 tables in a well-normalized schema. There are over 20 programmers, all of whom work on that schema from time to time, and many of them are working on it at any given time. There are many queries generated by framework and ORM tools, with many possible variations on selection criteria, and there are often ad hoc queries. Static analysis of conflicting transactions would be prohibitively expensive, and out-of-date at the moment of completion. While it is hard to ever be sure that a data integrity problem is the result of snapshot serialization anomalies, there is sufficient evidence that it is a material factor that work on implementation of SSI has been authorized. A review and some experimental work began in January, and serious development began in February. Work is progressing quickly; there's already a working prototype.

82. Passes regression tests.

84. Performance testing has not yet been done.

85. Several promising techniques for reducing false positives have not been implemented.

86. There haven't been enough tests or eyes on the code to believe that all corner cases have yet been covered.

88. Generally avoids table level locks for multi-row constraints.

89. No extra disk writes (like SELECT FOR UPDATE would generate).

91. Cause of serialization failure may not be obvious.

92. Serialization failures will sometimes happen on read-only transactions.

93. Rate of serialization failure is higher than other techniques.

95. Ad hoc queries are run against the database.

96. Some queries are generated by a framework or ORM.

97. Multi-row integrity rules are enforced by triggers or application code (not counting foreign key definitions). This is particularly an issue if it's not practical to acquire table locks at the very start of every transaction which will modify data related to such integrity rules.

98. Data violating business rules has been found in the database which has no explanation or which has been attributed to snapshot isolation anomalies. Serializable Snapshot Isolation will not be the best choice for everyone. Here are factors which tend to suggest it will be helpful in an environment.

99. If You're Interested We in the Wisconsin Court System hope that others will be interested in this feature. If it is something you might like to use, please let me know: Kevin.Grittner@wicourts.gov To learn more about this effort, visit the Wiki page: http://wiki.postgresql.org/wiki/Serializable If you want to help, the most critical need at this point is a good set of application mixes for performance benchmarking as we move through the performance tuning phase. Any suggestions for queries or combinations of concurrent queries which might be challenging are also welcome, so that we can have a thorough set of regression tests. PLEASE NOTE: If you are in a position to move the PostgreSQL 9.0 release forward, please don't take any time away from that effort to help this one. The sooner the 9.0 release is ready, the sooner we can start to work with the community on possible integration of this work into the PostgreSQL core.