From dc4afd93c8c1cceb92f50fa6574acac3930019bf Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Alberto=20Leiva=20Obando?= <56046999+jleiva-gap@users.noreply.github.com> Date: Tue, 17 Oct 2023 11:45:35 -0600 Subject: [PATCH] [RND-643] Serializable Snapshot Isolation (SSI) in PostgreSQL (#310) * [RND-643] Serializable Snapshot Isolation (SSI) in PostgreSQL Add documentation about PostgreSQL Serializable Snapshot Isolation (SSI) performance testing Add a note to locking\delete.test.ts * Update Delete.test.ts * Update postgres-SSI-performance-testing.md --- .../test/integration/locking/Delete.test.ts | 4 ++ .../postgres-SSI-performance-testing.md | 64 +++++++++++++++++++ 2 files changed, 68 insertions(+) create mode 100644 docs/performance-testing/postgres-SSI-performance-testing.md diff --git a/Meadowlark-js/backends/meadowlark-postgresql-backend/test/integration/locking/Delete.test.ts b/Meadowlark-js/backends/meadowlark-postgresql-backend/test/integration/locking/Delete.test.ts index c6adf7ac..fb3bf1bf 100644 --- a/Meadowlark-js/backends/meadowlark-postgresql-backend/test/integration/locking/Delete.test.ts +++ b/Meadowlark-js/backends/meadowlark-postgresql-backend/test/integration/locking/Delete.test.ts @@ -3,6 +3,10 @@ // The Ed-Fi Alliance licenses this file to you under the Apache License, Version 2.0. // See the LICENSE and NOTICES files in the project root for more information. +/** + * NOTE: If Postgres query transactions are modified to use SSI and the use of SELECT...FOR UPDATE/SHARE is removed, + * this test case should be updated to adjust its logic and avoid deadlocks at the database level and some test will fail. + */ import { DocumentInfo, NoDocumentInfo, diff --git a/docs/performance-testing/postgres-SSI-performance-testing.md b/docs/performance-testing/postgres-SSI-performance-testing.md new file mode 100644 index 00000000..46617111 --- /dev/null +++ b/docs/performance-testing/postgres-SSI-performance-testing.md @@ -0,0 +1,64 @@ +# RND-643: PostgreSQL Serializable Snapshot Isolation (SSI) performance testing + +## Goal + +Run Postgresql performance to compare the performance of the SSI implementation vs the SELECT...FOR UPDATE/SHARE implementation. + +## Description + +To test performance we use Autocannon tool. + +Changed transaction usage to use SSI. Additionally, retry logic was added with this change. With this logic, if a 40001 error occurs, a retry is executed. All functions that use transactions have been modified. + +As a side effect, the crash test\delete.test.ts failed. This test has transactions to create a lock and has no retry mechanism, so the test creates a deadlock and returns a timeout error. + +To capture database statistics we are reading data from pg_stat_database. A process runs each 5 seconds and inserts a row with the current statistics. With all the rows inserted, we can get an average of some indicators. From pg_stat_database we can get: + +- xact_commit: number of transactions in the database that have been completed successfully. +- xact_rollback: number of transactions that have been cancelled or rolled back. +- blks_read: number of disk blocks that have been read directly from the disk +- blks_hit: This is the number of times that the needed data was found in the cache or buffer. +- tup_returned: number of rows returned by queries in the database +- tup_fetched: number of rows fetched by queries in the database +- tup_inserted: number of rows inserted +- tup_updated: number of rows updated +- tup_deleted: number of rows deleted + +## Postgres Comparison + +For testing, Autocannon was run in the existing version in main and in the modified version. For each test, 3 iterations were executed and the comparison was made with the data obtained. + +First, we tested Autocannon with the original version and the modified version. + +||2xxx|Non-2xxx|Latency|Request|Throughput +| :--- | ---: | ---: | ---: | ---: | ---: | +**Non-SSI**|7075.67|0|564.35|44.51|15532.14 +**SSI**|5723.33|8659.66666666667|277.82|90.28|28020.23 +**Differences**|**-19.11%**|**100.00%**|**-50.77%**|**102.83%**|**80.40%** + +For this case we can see: + +- The SSI version has less 2xxx responses and more Non-2xxx. Now, we this change we are not locking the database, so that behavior is expected. +- In the same case, the SSI version has less latency and more requests and throughput compared to the original version. + +With this results we can see some improvement in the performance. + +Now, if we compare database statistics, we have the following results: + +|Pool Size | Avg commit | Avg rollback | Avg disk blocks read | Avg blocks hit cache | Avg tuples returned | Avg tuples fetched | Avg tuples_inserted | Avg tuples updated | Avg tuples deleted | +| :--- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | +Non-SSI|328.20|0|30.52|7662.52|1432.53|788.24|327.96|73.20|281.24 +SSI|149.51|98.88|17.03|5955.38|4748.24|445.19|151.52|46.46|153.13 +|**Differences**|**-54.00%**|**100.00%**|**-44.00%**|**-22.00%**|**231.00%**|**-44.00%**|**-54.00%**|**-37.00%**|**-46.00%** + +These results are consistent with the Autocannon results: + +- SSI version has less commits than previous version, and more rollbacks than previous version, that makes sense because we are not locking the database and we are executing a rollback to retry. +- Also, SSI version returned more tuples but affected tuples are less compared to previous version. That behavior is consistent with a retry mechanism, because when we need to retry, we need to restart the process. + +## Conclusion +- The version with SSI reduces latency but also generates a large amount of rollback, so the number of altered records is smaller. + +- Additionally, in cases like locking\delete.test.ts, removing the SELECT...FOR UPDATE/SHARE can cause errors and deadlocks. + +- For SSI to be effective, retry logic must be implemented, otherwise many transactions would fail without altering any records. \ No newline at end of file