{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":105944401,"defaultBranch":"master","name":"yugabyte-db","ownerLogin":"yugabyte","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2017-10-05T21:56:00.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/17074854?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1717833578.0","currentOid":""},"activityList":{"items":[{"before":"863ffadc92cd0d30a9f5d817864522c230068400","after":"599a763ae8a4f734c30115713d6c065e58fdabcb","ref":"refs/heads/posriniv/update-dbs","pushedAt":"2024-06-08T19:53:23.000Z","pushType":"force_push","commitsCount":0,"pusher":{"login":"posriniv","name":"Srinivas Pothuraju","path":"/posriniv","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/48309285?s=80&v=4"},"commit":{"message":"address final review comments","shortMessageHtmlLink":"address final review comments"}},{"before":"ac59ff5bb829a4912b2d68730b9031ec6de78cae","after":"25940f09ef9548723b8dfd2757ccf2b1f9f551d0","ref":"refs/heads/master","pushedAt":"2024-06-08T15:43:26.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ddhodge","name":"Dwight Hodge","path":"/ddhodge","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79169168?s=80&v=4"},"commit":{"message":"[doc][yba] Fix for manual on prem (#22639)\n\n* Fix for manual on prem\r\n\r\n* revert\r\n\r\n* stable\r\n\r\n* edits for style\r\n\r\n* minor fixes to RN\r\n\r\n* minor edit\r\n\r\n* format\r\n\r\n* minor edit","shortMessageHtmlLink":"[doc][yba] Fix for manual on prem (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2330095453\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22639\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/22639/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/22639\">#22639</a>)"}},{"before":"e9e806b2f49123eb5334d8d8b43f4f42037c019d","after":"ac59ff5bb829a4912b2d68730b9031ec6de78cae","ref":"refs/heads/master","pushedAt":"2024-06-08T04:37:07.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"d-uspenskiy","name":"Dmitry Uspenskiy","path":"/d-uspenskiy","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47734295?s=80&v=4"},"commit":{"message":"[#20797] YSQL: Fix SameTempTableCreationWithResponseCache/ResponseCacheInvalidationOnConnectionWithTempTableClosure unit tests\n\nSummary:\nThe `SameTempTableCreationWithResponseCache` and `ResponseCacheInvalidationOnConnectionWithTempTableClosure` unit tests require client connection to be fully closed by postgres (i.e. process started for connection processing must be finished). Both tests use sleep for this purpose. But there is no guaranty that process will be finished within specified time.\n\nCurrent diff substitutes the sleep with using of the `ClientConnectionsCountFetcher` helper class. This class is able to wait for client connection process termination by using info from the postgres' webserver proxy.\n\n**Note**: The diff moves some utility functions for json processing into `json_util.h` file. Local versions of same functions in several files are substituted with these new functions.\n\nTest Plan:\nJenkins\n\n```\n./yb_build.sh --gtest_filter PgCatalogPerfTest.SameTempTableCreationWithResponseCache -n 20\n./yb_build.sh --gtest_filter PgCatalogPerfTest.ResponseCacheInvalidationOnConnectionWithTempTableClosure -n 20\n```\n\nReviewers: fizaa, myang, kfranz\n\nReviewed By: myang\n\nSubscribers: ybase, yql\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35565","shortMessageHtmlLink":"[<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2102526714\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/20797\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/20797/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/20797\">#20797</a>] YSQL: Fix SameTempTableCreationWithResponseCache/ResponseCac…"}},{"before":"1f0aec226abcd76eb13ba4243f4a57857db05f4e","after":"e9e806b2f49123eb5334d8d8b43f4f42037c019d","ref":"refs/heads/master","pushedAt":"2024-06-07T21:51:44.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"pao214","name":"Bvsk Patnaik","path":"/pao214","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/16426043?s=80&v=4"},"commit":{"message":"[#22615] YSQL: Incorrect seek key in prefix-based backward scan\n\nSummary:\n### Context\n\nHybridScanChoices adds a kGroupEnd marker at the end of the DocKey. ScanChoices adds this marker at the end of the prefix in a prefix-based scan. This is acceptable in a forward scan since we wish to seek to the smallest key >= the scan target. However, in a backward scan, we wish to seek to the largest possible key <= the scan target. kGroupEnd marker is fighting against us in this case. We append kHighest (representing +Inf) to seek to the largest key but that happens after the kGroupEnd marker is appended. This is undesirable.\n\n### Fix\n\nRemove the kGroupEnd marker during the backward prefix-based seek scan (from a copy of current scan target).\n\n### Impact\n\n1. Uses a strict prefix-based scan.\n\nFull key scans are exempt from this issue since the kGroupEnd is not added on a key column.\n\n2. Is a backward scan.\n\nAs explained in the context section, kGroupEnd is fine when scanning forward since it is a small value.\n\n3. Has IN clauses.\n\nWithout multiple options, we may not be changing the scan target based on the user provided input more than once. Seeks still happen, however, we do not append a kGroupEnd marker on new scan targets obtained directly from scanning the database.\n\n### Scan Target Debugging\n\n1. Scan Target in HybridScanChoices is not always followed by a kGroupEnd marker. This marker is used to end hash keys group and range keys group. When converting a DocKey to a human-readable string, we need this end of the keys marker. This change appends this marker so we do not get corruption errors when debugging the doc keys.\n\n2. Propagate the allow_special flag when decoding keys so that special keys such as +Inf can be debug dumped appropriately.\nJira: DB-11524\n\nTest Plan:\nJenkins\n\n./yb_build.sh --java-test TestPgRegressDistinctPushdown\n\nAdded a test case that\n\n1. Uses a prefix-based scan.\n2. Is descending.\n3. Has multiple Option Ranges (IN clause).\n\nBackport-through: 2.18\n\nReviewers: tnayak, telgersma, smishra\n\nReviewed By: tnayak\n\nSubscribers: yql, ybase\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35497","shortMessageHtmlLink":"[<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2326018896\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22615\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22615/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22615\">#22615</a>] YSQL: Incorrect seek key in prefix-based backward scan"}},{"before":"a80a224b40011a8e87928a3f21b66c375d768c7f","after":"1f0aec226abcd76eb13ba4243f4a57857db05f4e","ref":"refs/heads/master","pushedAt":"2024-06-07T21:46:27.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"aishwarya24","name":"Aishwarya Chakravarthy ","path":"/aishwarya24","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5341399?s=80&v=4"},"commit":{"message":"[docs] sorted releases table by date (#22777)\n\n* sorted releases\r\n\r\n* some improvements","shortMessageHtmlLink":"[docs] sorted releases table by date (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2341191388\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22777\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/22777/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/22777\">#22777</a>)"}},{"before":"959513677c481a3f29dbc33bb28ab9bd23ed4cf2","after":"a80a224b40011a8e87928a3f21b66c375d768c7f","ref":"refs/heads/master","pushedAt":"2024-06-07T20:27:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"rajmaddy89","name":"Rajagopalan Madhavan","path":"/rajmaddy89","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10095310?s=80&v=4"},"commit":{"message":"[PLAT-14284]: Modify API Url in common components and integrate with YBA\n\nSummary:\n1. Modify API Url query param from customerUuid to customer_uuid in common components and integrate with YBA\n2. In triubleshoot-universe.tsx page, pass apiUrl as a param to the API in common component\n3. Split 1 component into 2 for better readability\n\nTest Plan: Not changes in behavior, same as previous time, just minor improvements\n\nReviewers: jmak, amalyshev\n\nReviewed By: jmak\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35703","shortMessageHtmlLink":"[PLAT-14284]: Modify API Url in common components and integrate with YBA"}},{"before":"2ce3383230c44995a1d82e2349e0bf0d3efba681","after":"959513677c481a3f29dbc33bb28ab9bd23ed4cf2","ref":"refs/heads/master","pushedAt":"2024-06-07T20:21:03.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ddhodge","name":"Dwight Hodge","path":"/ddhodge","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79169168?s=80&v=4"},"commit":{"message":"[DOC-388] updated CBO flags (#22765)\n\n* updated flags\r\n\r\n* fixed links\r\n\r\n* Enhanced Postgres Compatibility Mode\r\n\r\n* more packed row caveats\r\n\r\n* Apply suggestions from code review\r\n\r\n* Edit\r\n\r\n* v2.21 fixes\r\n\r\n* fix Versions menu\r\n\r\n* review comment\r\n\r\n---------\r\n\r\nCo-authored-by: Dwight Hodge <ghodge@yugabyte.com>\r\nCo-authored-by: Dwight Hodge <79169168+ddhodge@users.noreply.github.com>","shortMessageHtmlLink":"[DOC-388] updated CBO flags (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2338989930\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22765\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/22765/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/22765\">#22765</a>)"}},{"before":"fedd72540fbfee6ea0ea95453717bbf4506fe4cb","after":"2ce3383230c44995a1d82e2349e0bf0d3efba681","ref":"refs/heads/master","pushedAt":"2024-06-07T20:12:22.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"ddhodge","name":"Dwight Hodge","path":"/ddhodge","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/79169168?s=80&v=4"},"commit":{"message":"[docs] OS support pages update for v2.20, v2024.1, v2.21. (#22774)\n\n* some fixes\r\n\r\n* added an entry\r\n\r\n* added unsupported entries\r\n\r\n* edits for clarity\r\n\r\n* minor edits\r\n\r\n* minor edit\r\n\r\n* minor edit\r\n\r\n* version fix\r\n\r\n---------\r\n\r\nCo-authored-by: Dwight Hodge <ghodge@yugabyte.com>","shortMessageHtmlLink":"[docs] OS support pages update for v2.20, v2024.1, v2.21. (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2340593904\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22774\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/yugabyte/yugabyte-db/pull/22774/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/pull/22774\">#22774</a>)"}},{"before":"387775641c90a9fbc00b3d16ac89b9befecba2e0","after":"fedd72540fbfee6ea0ea95453717bbf4506fe4cb","ref":"refs/heads/master","pushedAt":"2024-06-07T19:06:24.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"artem-mindrov","name":"Artem","path":"/artem-mindrov","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/1575475?s=80&v=4"},"commit":{"message":"[PLAT-9735][xCluster] show errors exposed by GetReplicationStatus\n\nSummary: expose short descriptions of GetReplicationStatus errors via XClusterTableConfig in the GET XClusterConfig response\n\nTest Plan:\nTest with manually populated errors in the RPC response in `XClusterConfigTaskBase#setReplicationStatus`:\n\n```\n              List<ReplicationStatusErrorPB> replicationErrors =\n                  new ArrayList<>(replicationStatus.getErrorsList());\n              replicationErrors.add(\n                  ReplicationStatusErrorPB.newBuilder()\n                      .setError(REPLICATION_AUTO_FLAG_CONFIG_VERSION_MISMATCH)\n                      .setErrorDetail(\"test\")\n                      .build());\n              replicationErrors.add(\n                  ReplicationStatusErrorPB.newBuilder()\n                      .setError(REPLICATION_SCHEMA_MISMATCH)\n                      .setErrorDetail(\"test1\")\n                      .build());\n              replicationErrors.add(\n                  ReplicationStatusErrorPB.newBuilder()\n                      .setError(REPLICATION_MISSING_TABLE)\n                      .setErrorDetail(\"test2\")\n                      .build());\n```\n\nXCluster config overview page shows:\n{F251165}\nXCluster config details page:\n{F251167}\nRestart replication modal:\n{F251168}\n\nReviewers: hzare, cwang, vbansal, spothuraju, jmak, #yba-api-review\n\nReviewed By: hzare, cwang, jmak, #yba-api-review\n\nSubscribers: sanketh, yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35571","shortMessageHtmlLink":"[PLAT-9735][xCluster] show errors exposed by GetReplicationStatus"}},{"before":"9bb032f1f64e6314bac675b7a236ae0597956712","after":"387775641c90a9fbc00b3d16ac89b9befecba2e0","ref":"refs/heads/master","pushedAt":"2024-06-07T18:16:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Deepti-yb","name":"Deepti Kumar","path":"/Deepti-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/108267959?s=80&v=4"},"commit":{"message":"[PLAT-14195]Add GFlag groups field in Specific GFlags for create universe workflow\n\nSummary:\nAdding `gflagGroups` to `SpecificGFlags` that'll allow users to set groups of GFlags managed\nby YBA\n\nAddition to `UserIntent`\n```\n\"specificGFlags\": {\n        \"inheritFromPrimary\": false,\n        \"gflagGroups\": [\n            \"ENHANCED_POSTGRES_COMPATIBILITY\"\n        ],\n        \"perProcessFlags\": {\n            \"value\": {\n                \"MASTER\": {},\n                \"TSERVER\": {}\n            }\n        },\n        \"perAZ\": {}\n    }\n```\n\nIf a user tries to manually edit the gflags when the groups are added, the entries are overridden with values present in YBA.\n\nTest Plan:\n  # Universe creation with correct group name\n\n  # Universe creation with incorrect group name\n\n  # GFlag groups with audit logs (Both have entries to `ysql_pg_conf_csv`) - Needs to be tested\n\n  # User entries to `ysql_pg_conf_csv` a) on the same fields as the group b) different fields\n\n  # Apply the group to both primary and read replica clusters if present in primary cluster user intent regardless of `inheritFromPrimary` field\n\nReviewers: #yba-api-review, sanketh, sneelakantan\n\nReviewed By: #yba-api-review, sneelakantan\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35659","shortMessageHtmlLink":"[PLAT-14195]Add GFlag groups field in Specific GFlags for create univ…"}},{"before":"909d1799b075274d871a1910700a9e02bc83e5b3","after":"9bb032f1f64e6314bac675b7a236ae0597956712","ref":"refs/heads/master","pushedAt":"2024-06-07T16:55:05.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"mdbridge","name":"Mark","path":"/mdbridge","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/15827795?s=80&v=4"},"commit":{"message":"[#22737] DocDB: improve logging when we flush MemTables to conserve memory\n\nSummary:\nAs title.  As it stands, it's very hard to figure out whether the\nflushing is actually keeping us below the real limit or not.\nJira: DB-11640\n\nTest Plan:\nRun the following test and look at the logs:\n```\nybd --cxx-test flush-test --gtest_filter '*.TestFlushPicksOldestInactiveTabletAfterCompaction'\n```\nSample output:\n```\n   6200:[ts-1] I0605 12:20:18.997514 18650 tablet_memory_manager.cc:375] RocksDB reported write buffers size of 2044800 bytes now under Memstore global limit\n   6231:[ts-1] I0605 12:20:19.096494 18650 tablet_memory_manager.cc:340] Memstore global limit of 2097152 bytes reached (RocksDB reports write buffers using 2110792 bytes, over by 13640 bytes)\n   6232:[ts-1] I0605 12:20:19.096729 18650 tablet_memory_manager.cc:359] T a50a27279b7746db8e3d575d2f08f033 P 745c3828bf744e9bb2a2b01f8f337a4f : Flushing tablet a50a27279b7746db8e3d575d2f08f033, which has oldest memstore write time of { days: 19879 time: 12:20:13.965751 }\n   6235:[ts-1] I0605 12:20:19.097038 18650 tablet_memory_manager.cc:375] RocksDB reported write buffers size of 2053024 bytes now under Memstore global limit\n   6243:[ts-1] I0605 12:20:19.176455 18650 tablet_memory_manager.cc:340] Memstore global limit of 2097152 bytes reached (RocksDB reports write buffers using 2102528 bytes, over by 5376 bytes)\n   6244:[ts-1] I0605 12:20:19.176646 18650 tablet_memory_manager.cc:359] T ff7a9832571843ea9912419383c90802 P 745c3828bf744e9bb2a2b01f8f337a4f : Flushing tablet ff7a9832571843ea9912419383c90802, which has oldest memstore write time of { days: 19879 time: 12:20:14.065843 }\n   6246:[ts-1] I0605 12:20:19.176882 18650 tablet_memory_manager.cc:375] RocksDB reported write buffers size of 2094280 bytes now under Memstore global limit\n```\n\n```\nybd --cxx-test ts_tablet_manager-test --gtest_filter '*.TestProperBackgroundFlushOnStartup' --test-args '' >& /tmp/generic.mdl.log\n```\n\nReviewers: asrivastava, mhaddad, zdrudi\n\nReviewed By: zdrudi\n\nSubscribers: zdrudi, ybase\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35636","shortMessageHtmlLink":"[<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2336700832\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22737\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22737/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22737\">#22737</a>] DocDB: improve logging when we flush MemTables to conserve m…"}},{"before":"b2a0133a0c2098e0cd42e0febb47729b44c77217","after":"5abefa0c77a52e57ed1a31149c5045d1e2c54dbf","ref":"refs/heads/2.20","pushedAt":"2024-06-07T16:50:04.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"basavaraj29","name":"Basava Kolagani","path":"/basavaraj29","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/17742021?s=80&v=4"},"commit":{"message":"[BACKPORT 2.20][#22630] DocDB: Don't set key bounds for intent iterator used during conflict resolution\n\nSummary:\nOriginal commit: fb9e8f8e18e56a73dff45526713bf6f09700e1d1 / D35543\nAs part of conflict resolution, we create an intent iterator respecting the tablet's key bounds. This could lead to undetected conflicts and violation of serializable isolation guarantees.\n\nFor instance, consider a serializable transaction that did `select * from table`. It would lead to the following strong intent for empty doc key on all tablets of the table\n```\nSubDocKey(DocKey([], []), []) [kStrongRead] <ht> -> <txn>\n```\n\nDuring conflict resolution, an intent iterator is created using\n```\n      intent_iter_ = CreateIntentsIteratorWithHybridTimeFilter(\n          doc_db_.intents, &status_manager(), doc_db_.key_bounds, &intent_key_upperbound_);\n```\n\nSo essentially, when another transaction is trying to do an update, it requests weak read/write locks on the empty doc key `DocKey([], [])`. While checking conflicts for this record, we have the following\n```\n  Status ReadIntentConflicts(IntentTypeSet type, KeyBytes* intent_key_prefix) {\n    ...\n    // the below seeks to the tablet's lower key bound when the given target < tablet's lower key bound\n    intent_iter_.Seek(intent_key_prefix->AsSlice());\n    // and then we exit checking conflicts if the seeked intent record doesn't have prefix <intent_key_prefix>\n```\nwhich leads to undetected conflicts for empty doc key - `DocKey([], [])`. And the update/insert that should ideally have been blocked on the serializable txn instead goes through.\n\nThis issue is observed only when all of the following hold true\n1. the tablet has key bounds set, which seems to happen only for tablets created through split.\n2. there is a serializable transaction involved and other transaction are trying to perform a conflicting operation.\n\n**Solution**\nTo address the problem, we create the intent iterator with `kNoBounds` thus allowing it to see all intents of the parent tablet. There should be no significant overhead for this approach, because as part of conflict resolution, the intent iterator seeks to the given entry (by the current transaction) and calls `Next` until the found key doesn't have the supplied key as a prefix. Since post split, the new requests at the child tablet correspond to the keys hosted by the child, it should not result in unnecessary additional `Seek`/`Next` calls. Additionally, compaction on the intentsdb doesn't use key bounds, so the intent record(s) on empty key are preserved through compactions, which is necessary for correctness.\n\nIf there is any perf impact, we should revisit this issue. Here are a couple of other relevant observations-\n1. It is just the empty key that we had issues with in detecting conflicts. If serializable transaction are taken out of the picture, then we shouldn't have had hit such issues with just RC/RR transactions (since the conflict would have been at the key hash or higher, which falls within the tablet's key bounds).\n2. If necessary, we can create the intent iterator with no lower bound, but the upper bound set to the tablet's upper bound.\nJira: DB-11537\n\nTest Plan:\n./yb_build.sh --cxx-test='TEST_F(PgTabletSplitTest, TestConflictResolutionChecksConflictsAgainstEmptyKey)'\n\nThe test fails consistently without the fix.\n\nReviewers: rthallam, timur, arybochkin, sergei\n\nReviewed By: arybochkin\n\nSubscribers: ybase\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35590","shortMessageHtmlLink":"[BACKPORT 2.20][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2327666303\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22630\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22630/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22630\">#22630</a>] DocDB: Don't set key bounds for intent iterat…"}},{"before":"6c03b738ae6efe7ef5942d14048da0ff5f695bb6","after":"a5b57b60a4549edebf4e596409792ffe802552fd","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T16:48:16.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"basavaraj29","name":"Basava Kolagani","path":"/basavaraj29","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/17742021?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][#22630] DocDB: Don't set key bounds for intent iterator used during conflict resolution\n\nSummary:\nOriginal commit: fb9e8f8e18e56a73dff45526713bf6f09700e1d1 / D35543\nAs part of conflict resolution, we create an intent iterator respecting the tablet's key bounds. This could lead to undetected conflicts and violation of serializable isolation guarantees.\n\nFor instance, consider a serializable transaction that did `select * from table`. It would lead to the following strong intent for empty doc key on all tablets of the table\n```\nSubDocKey(DocKey([], []), []) [kStrongRead] <ht> -> <txn>\n```\n\nDuring conflict resolution, an intent iterator is created using\n```\n      intent_iter_ = CreateIntentsIteratorWithHybridTimeFilter(\n          doc_db_.intents, &status_manager(), doc_db_.key_bounds, &intent_key_upperbound_);\n```\n\nSo essentially, when another transaction is trying to do an update, it requests weak read/write locks on the empty doc key `DocKey([], [])`. While checking conflicts for this record, we have the following\n```\n  Status ReadIntentConflicts(IntentTypeSet type, KeyBytes* intent_key_prefix) {\n    ...\n    // the below seeks to the tablet's lower key bound when the given target < tablet's lower key bound\n    intent_iter_.Seek(intent_key_prefix->AsSlice());\n    // and then we exit checking conflicts if the seeked intent record doesn't have prefix <intent_key_prefix>\n```\nwhich leads to undetected conflicts for empty doc key - `DocKey([], [])`. And the update/insert that should ideally have been blocked on the serializable txn instead goes through.\n\nThis issue is observed only when all of the following hold true\n1. the tablet has key bounds set, which seems to happen only for tablets created through split.\n2. there is a serializable transaction involved and other transaction are trying to perform a conflicting operation.\n\n**Solution**\nTo address the problem, we create the intent iterator with `kNoBounds` thus allowing it to see all intents of the parent tablet. There should be no significant overhead for this approach, because as part of conflict resolution, the intent iterator seeks to the given entry (by the current transaction) and calls `Next` until the found key doesn't have the supplied key as a prefix. Since post split, the new requests at the child tablet correspond to the keys hosted by the child, it should not result in unnecessary additional `Seek`/`Next` calls. Additionally, compaction on the intentsdb doesn't use key bounds, so the intent record(s) on empty key are preserved through compactions, which is necessary for correctness.\n\nIf there is any perf impact, we should revisit this issue. Here are a couple of other relevant observations-\n1. It is just the empty key that we had issues with in detecting conflicts. If serializable transaction are taken out of the picture, then we shouldn't have had hit such issues with just RC/RR transactions (since the conflict would have been at the key hash or higher, which falls within the tablet's key bounds).\n2. If necessary, we can create the intent iterator with no lower bound, but the upper bound set to the tablet's upper bound.\nJira: DB-11537\n\nTest Plan:\n./yb_build.sh --cxx-test='TEST_F(PgTabletSplitTest, TestConflictResolutionChecksConflictsAgainstEmptyKey)'\n\nThe test fails consistently without the fix.\n\nReviewers: rthallam, timur, arybochkin, sergei\n\nReviewed By: arybochkin\n\nSubscribers: ybase, yql\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35587","shortMessageHtmlLink":"[BACKPORT 2024.1][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2327666303\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/22630\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/22630/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/22630\">#22630</a>] DocDB: Don't set key bounds for intent iter…"}},{"before":"13273c8b8f2e3fd0ff81bda67921deb4711cecff","after":"909d1799b075274d871a1910700a9e02bc83e5b3","ref":"refs/heads/master","pushedAt":"2024-06-07T16:13:00.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Vars-07","name":"Shubham Varshney","path":"/Vars-07","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/19248012?s=80&v=4"},"commit":{"message":"[PLAT-14275] skip setup node_exporter systemd for centos:7 machines\n\nSummary: skip setup node_exporter systemd for centos:7 machines as we already do that in provision phase.\n\nTest Plan: Tested centos:7 based universe creation using cron.\n\nReviewers: amalyshev, muthu, anijhawan, nbhatia\n\nReviewed By: nbhatia\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35695","shortMessageHtmlLink":"[PLAT-14275] skip setup node_exporter systemd for centos:7 machines"}},{"before":"caf6345bf09cf2c1c5e22e8c9b990cf6f084ee54","after":"6c03b738ae6efe7ef5942d14048da0ff5f695bb6","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T16:10:36.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"subramanian-neelakantan","name":"Subramanian Neelakantan","path":"/subramanian-neelakantan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5960548?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-14276][PLAT-14267] Fixing the openapi-format to 1.17.1 ver\n\nSummary:\nOriginal commit: e9fa07e6c3adee3f864b7e0c17bda7bca99cccde / D35690\nAs part of https://phorge.dev.yugabyte.com/D35315, I had fixed our builds to always use openapi-format@1.17.1 version. Always pulling and using the latest version of this tool is not good, as it might break our CI builds if there is a node version bump in the latest version of this tool. This was done in anticipation of such a scenario, but we unfortunately hit exactly this same scenario today. The change does ensure to use the exact version of 1.17.1 in a local build, but due to a bug, did not ensure that in the CI pipeline that uses npx.\n\nThis was exposed today as the upstream openapi-format tool released a new version (https://github.com/thim81/openapi-format/releases/tag/v1.18.0) that bumped the min nodejs version to 18.x (we use 12.22.11 and 16.10.0 on CI pipelines).\n\nThis change fixes the version of openapi-format tool used via npx command as well.\n\nTest Plan: Phab pipeline\n\nReviewers: skurapati, dshubin\n\nReviewed By: skurapati\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35697","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-14276][PLAT-14267] Fixing the openapi-format t…"}},{"before":"d5b05035208077aede2d8ab99d10c99c13ce317b","after":"13273c8b8f2e3fd0ff81bda67921deb4711cecff","ref":"refs/heads/master","pushedAt":"2024-06-07T15:33:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"yorq","name":"Yuriy Shchetinin","path":"/yorq","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90446433?s=80&v=4"},"commit":{"message":"[PLAT-14215] Cannot edit universe from ephemeral to regular instance type via edit universe\n\nSummary: Fixed smart resize check that failed if current storageType is null\n\nTest Plan:\nsbt test\nCreate universe with i3 instance type\nEdit universe and change instance type to c5.large\nVerify [FULL_MOVE] updateOptions in response\n\nReviewers: cwang\n\nReviewed By: cwang\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35624","shortMessageHtmlLink":"[PLAT-14215] Cannot edit universe from ephemeral to regular instance …"}},{"before":"eb20d876c75630239f03c52534317107e346afb3","after":"d5b05035208077aede2d8ab99d10c99c13ce317b","ref":"refs/heads/master","pushedAt":"2024-06-07T13:10:15.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"yorq","name":"Yuriy Shchetinin","path":"/yorq","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/90446433?s=80&v=4"},"commit":{"message":"[PLAT-13467] Change DB min placement to (1,1) for 2 AZ RF3\n\nSummary:\nFor this special case (RF3 2 AZs) it is better not to put the excessive replica to the specific AZ\n(so that DB will make that decision for each tablet based on random) to have more uniform distribution of tablet copies.\n\nTest Plan:\nsbt test\n\n1) Create RF3 2 AZ universe\n2) Run payload on the universe\n3) Run upgrade\n4) Verify everything is ok\n\nReviewers: sanketh, nsingh\n\nReviewed By: sanketh\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35450","shortMessageHtmlLink":"[PLAT-13467] Change DB min placement to (1,1) for 2 AZ RF3"}},{"before":"d47c1c3a9517d87ba3e3535238aac909e5be11b9","after":"caf6345bf09cf2c1c5e22e8c9b990cf6f084ee54","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T08:41:34.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vipul-yb","name":"Vipul Bansal","path":"/vipul-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/86227026?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-14250]: Do not promote autoflags on universe while deleting k8 universe\n\nSummary:\nEarlier, we used to promote autoflags while deleting k8s universe if it had xcluster, but this is not required after rollback.\nWe fixed it for VM universes earlier here: https://phorge.dev.yugabyte.com/D31469 but we forget to add it for k8s.\n\nOriginal diff/commit: D35630\n\nTest Plan: Tested manually by establishing a xcluster between k8s universe and verified that while deleting the universe autoflags are not promoted when both version are greater than rollback support version.\n\nReviewers: cwang, sanketh\n\nReviewed By: cwang\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35685","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-14250]: Do not promote autoflags on universe w…"}},{"before":"cec59ebdeb70845570807921a21c4d633f6814d8","after":"b2a0133a0c2098e0cd42e0febb47729b44c77217","ref":"refs/heads/2.20","pushedAt":"2024-06-07T08:39:24.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vipul-yb","name":"Vipul Bansal","path":"/vipul-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/86227026?s=80&v=4"},"commit":{"message":"[BACKPORT 2.20][PLAT-14250]: Do not promote autoflags on universe while deleting k8 universe\n\nSummary:\nEarlier, we used to promote autoflags while deleting k8s universe if it had xcluster, but this is not required after rollback.\nWe fixed it for VM universes earlier here: https://phorge.dev.yugabyte.com/D31469 but we forget to add it for k8s.\n\nOriginal diff/commit: D35630\n\nTest Plan: Tested manually by establishing a xcluster between k8s universe and verified that while deleting the universe autoflags are not promoted when both version are greater than rollback support version.\n\nReviewers: cwang, sanketh\n\nReviewed By: cwang\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35686","shortMessageHtmlLink":"[BACKPORT 2.20][PLAT-14250]: Do not promote autoflags on universe whi…"}},{"before":"e9fa07e6c3adee3f864b7e0c17bda7bca99cccde","after":"eb20d876c75630239f03c52534317107e346afb3","ref":"refs/heads/master","pushedAt":"2024-06-07T08:38:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"vipul-yb","name":"Vipul Bansal","path":"/vipul-yb","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/86227026?s=80&v=4"},"commit":{"message":"[PLAT-14250]: Do not promote autoflags on universe while deleting k8 universe\n\nSummary:\nEarlier, we used to promote autoflags while deleting k8s universe if it had xcluster, but this is not required after rollback.\nWe fixed it for VM universes earlier here: https://phorge.dev.yugabyte.com/D31469 but we forget to add it for k8s.\n\nTest Plan: Tested manually by establishing a xcluster between k8s universe and verified that while deleting the universe autoflags are not promoted when both version are greater than rollback support version.\n\nReviewers: cwang, sanketh\n\nReviewed By: cwang\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35630","shortMessageHtmlLink":"[PLAT-14250]: Do not promote autoflags on universe while deleting k8 …"}},{"before":"fddcc0e53c2fcb8427d7aedaec0b290784922e82","after":"d47c1c3a9517d87ba3e3535238aac909e5be11b9","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T05:32:37.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"iSignal","name":"Sanketh I","path":"/iSignal","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8207010?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1] Enable java frame pointer for potential perf debugging\n\nSummary:\nThis allows the linux perf tool to show Java call stacks in cmds like \"perf top\"\n\nOriginal diff: https://phorge.dev.yugabyte.com/D35546\n\nTest Plan: Try out the setting on portal, verify it works\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35644","shortMessageHtmlLink":"[BACKPORT 2024.1] Enable java frame pointer for potential perf debugging"}},{"before":"7cbea2dcb53cbbe668eee95041a76efcee3026dc","after":"d1bc2d8c41df7fa862ee33f0044801c9eecdf1e9","ref":"refs/heads/pg15","pushedAt":"2024-06-07T05:09:12.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"swapshivam3","name":null,"path":"/swapshivam3","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/72791604?s=80&v=4"},"commit":{"message":"[pg15] test: port TestPgRegressProfile, yb_index_selectivity, yb_pg_insert_conflict\n\nSummary:\nThis revision fixes test failures as below:\n - yb_profile: \\h command now links to the postgres documentation (27f3dea64833d68c1fa08c1e5d26176a579f69c8). Documentation added for \"ALTER USER is now an alias for ALTER ROLE\" (See gram.y AlterRoleStmt in master/PG15, is same in both) (45b9805706fdc726906fc9187c9a7b38c559755c). OID is not a hidden attribute (578b229718e8f15fa779e20f086c4b6bb3776106).\n - yb_role_profile: OID is not a hidden attribute anymore. rolprflockeduntil does not have a NOT NULL constraint (after D34852).\n - yb_pg_insert_conflict: Ported test from PG15. Some parts of partitioned tables with ON CONFLICT does not work (on master as well) so it remains commented.\n - yb_index_selectivity: Some queries now use incremental sort. Modified expected output to match the plans. (d2d8a229bc58a2014dce1c7a4fcdb6c5ab9fb8da)\n - yb_dml_pushdown: Disable query identifiers in EXPLAIN VERBOSE output\n\nTest Plan:\nJenkins: rebase: pg15\n```\n./pg15_tests/test_lint.sh\nfor _ in {1..50}; do grep TestPgRegressDml pg15_tests/passing_tests.tsv; done | pg15_tests/run_tests.sh\nfor _ in {1..50}; do grep TestPgRegressHashIndex pg15_tests/passing_tests.tsv; done | pg15_tests/run_tests.sh\nfor _ in {1..50}; do grep TestPgRegressProfile pg15_tests/passing_tests.tsv; done | pg15_tests/run_tests.sh\n```\n\nReviewers: aagrawal, jason\n\nReviewed By: jason\n\nSubscribers: aagrawal, yql\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D34990","shortMessageHtmlLink":"[pg15] test: port TestPgRegressProfile, yb_index_selectivity, yb_pg_i…"}},{"before":"9879bb0b1819c40d2cec65973ad9be556ed2d9ad","after":"fddcc0e53c2fcb8427d7aedaec0b290784922e82","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T04:39:11.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"d-uspenskiy","name":"Dmitry Uspenskiy","path":"/d-uspenskiy","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/47734295?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][#21623] YSQL: Simplify used_read_time management in PgClientSession\n\nSummary:\nThe `used_read_time` field in the PgClientSession holds the information about read time used by the previous success operation on the `kPlain` session. This field is altered in async manner without locking PgClientSession mutex. On performing new operation on `kPlain` session `PgClientSession` object checks the `used_read_time` field and set read time from it on a session if necessary. This action is safe because `PgClientSession` mutex is locked at this point.\n\nNowadays `PgClientSession` checks the `used_read_time` only in some special conditions and not on performing any operation for `kPlain` session. Updating read time on each performing new operation simplifies the code much and makes the purpose of `used_read_time` much more obvious.\n\n**Note:** Also this diff fixes theoretical possibility of applying used read time from obsolete operations.\nSuppose the following scenario:\n- new operation is performed, PgClientSession resets read time and send RPC to remote t-server without read time\n- for some reason operations takes too much time and PgClientSession receives request to start new transaction and run new operation without waiting for response for previous operation\n- after some time PgClientSession receives the response for first (obsolete) operation and apply user read time from this operation to session\n\nTo solve this issue integer value is used as a signature. On each reset of read time `PgClientSession` increments signature counter and provide current value to the operation. Prior to update used read time field completed operation compares its own signature with current signature on `used_read_time` field. `used_read_time` value will be changed only in case signature matches.\n\nOriginal commit: 4b0a251887c3dcd0c210e331c6863165e6a2c1bb / D33423\n\nJira: DB-10518\n\nTest Plan: Jenkins\n\nReviewers: pjain, sergei, amartsinchyk\n\nReviewed By: pjain\n\nSubscribers: bogdan, ybase, yql, smishra\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35438","shortMessageHtmlLink":"[BACKPORT 2024.1][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2200399794\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/21623\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/21623/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/21623\">#21623</a>] YSQL: Simplify used_read_time management in…"}},{"before":"3e97962ac212d987464978ab3dd2e6bb96a6de63","after":"9879bb0b1819c40d2cec65973ad9be556ed2d9ad","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T04:25:05.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-14255] Mask telemetry provider creds in the logs when request failed\n\nSummary:\nMask telemetry provider creds in the logs when request failed.\nOriginal commit: c31bafa86310a6048ff9e53ff8bc0e4fee369bd9 / D35651\n\nTest Plan:\nManually tested.\nRun UTs.\nRun itests.\n\nReviewers: amalyshev\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35662","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-14255] Mask telemetry provider creds in the lo…"}},{"before":"17ad71e2a439dce3d2dd6c4a0697b3113a29ad77","after":"3e97962ac212d987464978ab3dd2e6bb96a6de63","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T04:23:06.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Sahith02","name":"Sahith Kurapati","path":"/Sahith02","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/52966918?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1][PLAT-13740] Add preflight checks when creating telemetry provider for DD and GCP\n\nSummary:\nAdded preflight validations when creating telemetry provider for DD and GCP.\nOriginal commit: bf180c34fb2afb2147533eebb6d08dbddb727083 / D35439\n\nTest Plan:\nManually tested both Datadog and GCP, positive and negative test scenarios\nUpdate UTs.\nRun UTs.\nRun itests.\n\nReviewers: amalyshev, sneelakantan\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35661","shortMessageHtmlLink":"[BACKPORT 2024.1][PLAT-13740] Add preflight checks when creating tele…"}},{"before":"c9fa85ae1186368d7abb17b5ef5a8ac9d304280e","after":"e9fa07e6c3adee3f864b7e0c17bda7bca99cccde","ref":"refs/heads/master","pushedAt":"2024-06-07T04:19:07.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"subramanian-neelakantan","name":"Subramanian Neelakantan","path":"/subramanian-neelakantan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/5960548?s=80&v=4"},"commit":{"message":"[PLAT-14267] Fixing the openapi-format to 1.17.1 ver\n\nSummary:\nAs part of https://phorge.dev.yugabyte.com/D35315, I had fixed our builds to always use openapi-format@1.17.1 version. Always pulling and using the latest version of this tool is not good, as it might break our CI builds if there is a node version bump in the latest version of this tool. This was done in anticipation of such a scenario, but we unfortunately hit exactly this same scenario today. The change does ensure to use the exact version of 1.17.1 in a local build, but due to a bug, did not ensure that in the CI pipeline that uses npx.\n\nThis was exposed today as the upstream openapi-format tool released a new version (https://github.com/thim81/openapi-format/releases/tag/v1.18.0) that bumped the min nodejs version to 18.x (we use 12.22.11 and 16.10.0 on CI pipelines).\n\nThis change fixes the version of openapi-format tool used via npx command as well.\n\nTest Plan: Phab pipeline\n\nReviewers: skurapati, dshubin\n\nReviewed By: skurapati\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35690","shortMessageHtmlLink":"[PLAT-14267] Fixing the openapi-format to 1.17.1 ver"}},{"before":"9ea1c7487e3c88c3ea66c4627bdf3a800be12070","after":"cec59ebdeb70845570807921a21c4d633f6814d8","ref":"refs/heads/2.20","pushedAt":"2024-06-07T04:11:41.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"iSignal","name":"Sanketh I","path":"/iSignal","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8207010?s=80&v=4"},"commit":{"message":"[BACKPORT 2.20] [PLAT-14254] Add follower_lag_ms metric to the metrics dashboard\n\nSummary:\n1. Add follower_lag_ms metric to the dashboard\n2. Also remove some gflags from the most used flags list because these are set by YBA and in its \"forbidden to override\" list\n\nOriginal diff: https://phorge.dev.yugabyte.com/D35643\n\nTest Plan: Create a universe, verify dashboards, verify gflags UI\n\nReviewers: amalyshev, vbansal\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35646","shortMessageHtmlLink":"[BACKPORT 2.20] [PLAT-14254] Add follower_lag_ms metric to the metric…"}},{"before":"8f5ccdeda948151dd323cef1098bda3872345482","after":"17ad71e2a439dce3d2dd6c4a0697b3113a29ad77","ref":"refs/heads/2024.1","pushedAt":"2024-06-07T04:06:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"iSignal","name":"Sanketh I","path":"/iSignal","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/8207010?s=80&v=4"},"commit":{"message":"[BACKPORT 2024.1] [PLAT-14254] Add follower_lag_ms metric to the metrics dashboard\n\nSummary:\n1. Add follower_lag_ms metric to the dashboard\n2. Also remove some gflags from the most used flags list because these are set by YBA and in its \"forbidden to override\" list\n\nOriginal diff: https://phorge.dev.yugabyte.com/D35643\n\nTest Plan: Create a universe, verify dashboards, verify gflags UI\n\nReviewers: amalyshev, vbansal\n\nReviewed By: amalyshev\n\nSubscribers: yugaware\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35645","shortMessageHtmlLink":"[BACKPORT 2024.1] [PLAT-14254] Add follower_lag_ms metric to the metr…"}},{"before":"d06e0e997c4654d9bc18133b2dd5d4ff28c3d6d3","after":"c9fa85ae1186368d7abb17b5ef5a8ac9d304280e","ref":"refs/heads/master","pushedAt":"2024-06-06T22:11:29.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"rajmaddy89","name":"Rajagopalan Madhavan","path":"/rajmaddy89","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/10095310?s=80&v=4"},"commit":{"message":"[PLAT-14240]: Let troubleshoot-ui common component handle api token\n\nSummary: Let troubleshoot-ui common component handle api token\n\nTest Plan:\nPlease refer to screenshot\n{F252857}\n\nReviewers: amalyshev, jmak\n\nReviewed By: amalyshev, jmak\n\nSubscribers: yugaware\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35681","shortMessageHtmlLink":"[PLAT-14240]: Let troubleshoot-ui common component handle api token"}},{"before":"fcd6f4520467e809791b7e5421d89a60dff3c2d8","after":"9ea1c7487e3c88c3ea66c4627bdf3a800be12070","ref":"refs/heads/2.20","pushedAt":"2024-06-06T21:44:07.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"karthik-ramanathan-3006","name":"Karthik Ramanathan","path":"/karthik-ramanathan-3006","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/32414766?s=80&v=4"},"commit":{"message":"[BACKPORT 2.20][#18948] YSQL: Explicitly destruct YSQL webserver on SIGTERM\n\nSummary:\nOriginal commit: 5862233cc0847a73d13a8f4c4c26c9e129071e61 / D35116\nThe YSQL webserver has occasionally produced coredumps of the following form upon receiving a termination signal from postmaster.\n```\n                #0  0x00007fbac35a9ae3 _ZNKSt3__112__hash_tableINS_17__hash_value_typeINS_12basic_string <snip>\n                #1  0x00007fbac005485d _ZNKSt3__113unordered_mapINS_12basic_string <snip> (libyb_util.so)\n                #2  0x00007fbac0053180 _ZN2yb16PrometheusWriter16WriteSingleEntryERKNSt3__113unordered_mapINS1_12basic_string <snip>\n                #3  0x00007fbab21ff1eb _ZN2yb6pggateL26PgPrometheusMetricsHandlerERKNS_19WebCallbackRegistry10WebRequestEPNS1_11WebResponseE (libyb_pggate_webserver.so)\n                ....\n                ....\n```\n\nThe coredump indicates corruption of a namespace-scoped variable of type unordered_map while attempting to serve a request after a termination signal has been received.\nThe current code causes the webserver (postgres background worker) to call postgres' `proc_exit()` which consequently calls `exit()`.\n\nAccording to the [[ https://en.cppreference.com/w/cpp/utility/program/exit | C++ standard ]], a limited amount of cleanup is performed on exit():\n - Notably destructors of variables with automatic storage duration are not invoked. This implies that the webserver's destructor is not called, and therefore the server is not stopped.\n - Namespace-scoped variables have [[ https://en.cppreference.com/w/cpp/language/storage_duration | static storage duration ]].\n - Objects with static storage duration are destroyed.\n - This leads to a possibility of undefined behavior where the webserver may continue running for a short duration of time, while the static variables used to serve requests may have been GC'ed.\n\nThis revision explicitly stops the webserver upon receiving a termination signal, by calling its destructor.\nIt also adds logic to the handlers to return a `503 SERVICE_UNAVAILABLE` once termination has been initiated.\nJira: DB-7796\n\nTest Plan:\nTo test this manually, use a HTTP load generation tool like locust to bombard the YSQL Webserver with requests to an endpoint like `<address>:13000/prometheus-metrics`.\nOn a standard devserver, I configured locust to use 30 simultaneous users (30 requests per second) to reproduce the issue.\n\nThe following bash script can be used to detect the coredumps:\n```\n#/bin/bash\nITERATIONS=50\nYBDB_PATH=/path/to/code/yugabyte-db\n\n# Count the number of dump files to avoid having to use `sudo coredumpctl`\nidumps=$(ls /var/lib/systemd/coredump/ | wc -l)\nfor ((i = 0 ; i < $ITERATIONS ; i++ ))\ndo\n        echo \"Iteration: $(($i + 1))\";\n        $YBDB_PATH/bin/yb-ctl restart > /dev/null\n\n        nservers=$(netstat -nlpt 2> /dev/null | grep 13000 | wc -l)\n        if (( nservers != 1)); then\n                echo \"Web server has not come up. Exiting\"\n                exit 1;\n        fi\n\n        sleep 5s\n\n        # Kill the webserver\n        pkill -TERM -f 'YSQL webserver'\n\n        # Count the number of coredumps\n        # Please validate that the coredump produced is that of postgres/webserver\n        ndumps=$(ls /var/lib/systemd/coredump/ | wc -l)\n        if (( ndumps > idumps  )); then\n                echo \"Core dumps: $(($ndumps - $idumps))\"\n        else\n                echo \"No new core dumps found\"\n        fi\ndone\n```\n\nRun the script with the load generation tool running against the webserver in the background.\n - Without the fix in this revision, the above script produced 8 postgres/webserver core dumps in 50 iterations.\n - With the fix, no coredumps were observed.\n\nReviewers: telgersma, fizaa\n\nReviewed By: telgersma\n\nSubscribers: ybase, smishra, yql\n\nTags: #jenkins-ready\n\nDifferential Revision: https://phorge.dev.yugabyte.com/D35170","shortMessageHtmlLink":"[BACKPORT 2.20][<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"1877033179\" data-permission-text=\"Title is private\" data-url=\"https://github.com/yugabyte/yugabyte-db/issues/18948\" data-hovercard-type=\"issue\" data-hovercard-url=\"/yugabyte/yugabyte-db/issues/18948/hovercard\" href=\"https://github.com/yugabyte/yugabyte-db/issues/18948\">#18948</a>] YSQL: Explicitly destruct YSQL webserver on S…"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEYBBqwQA","startCursor":null,"endCursor":null}},"title":"Activity · yugabyte/yugabyte-db"}