v1.31.0

[romange]

Dragonfly v1.31.0

This release is mostly focused on stability features and bug fixes.

Important stability fixes and changes:

1. Introduce partial sync feature that avoids full replica reload during master/replica short disconnects chore: bring back partial sync #5182
2. Add support for GAT/GATS commands Implementation of memcache GAT and GATS commands #5257
3. Introduce experimental huffman encoding for keys feat: add flags support for huffman encoding #5179

What's Changed

• chore: support fetching a huffman encoded string from CompactObj by @romange in chore: support fetching a huffman encoded string from CompactObj #5086
• fix(hset_family): Fix val being overwritten by TTL by @abhijat in fix(hset_family): Fix val being overwritten by TTL #5094
• fix: memcached authentication handling by @vyavdoshenko in fix: memcached authentication handling #5098
• fix: memcached port has been already used crash by @vyavdoshenko in fix: memcached port has been already used crash #5097
• chore: Add suffix and infix search syntax support in query parser by @vyavdoshenko in chore: Add suffix and infix search syntax support in query parser #5091
• chore(server): cleanup unused noop opcode in journal by @adiholden in chore(server): cleanup unused noop opcode in journal #5101
• chore: add arch type to dfly bench release binary by @kostasrim in chore: add arch type to dfly bench release binary #5090
• feat: add HELP SCAN and MINMSZ filter by @romange in feat: add HELP SCAN and MINMSZ filter #5099
• fix test: rdb test keep loading expired keys by @adiholden in fix test: rdb test keep loading expired keys #5105
• refactor: provide CommandContext into InvokeCmd by @BorysTheDev in refactor: provide CommandContext into InvokeCmd #5100
• dfly_bench: Send auth if required before cluster setup by @abhijat in dfly_bench: Send auth if required before cluster setup #5107
• chore: tls improvements by @romange in chore: tls improvements #5116
• fix: XTRIM command crash by @vyavdoshenko in fix: XTRIM command crash #5121
• chore: update module versions by @romange in chore: update module versions #5126
• chore: provide metrics for huffman encoding by @romange in chore: provide metrics for huffman encoding #5117
• fix: make search keywords case insensitive by @romange in fix: make search keywords case insensitive #5129
• chore: replica clean ups by @romange in chore: replica clean ups #5127
• fix: collapsing logic in CapturingReplyBuilder by @romange in fix: collapsing logic in CapturingReplyBuilder #5133
• fix: improved socket error logging for connection diagnostics by @vyavdoshenko in fix: improved socket error logging for connection diagnostics #5062
• fix: out of bounds SIMD access by @romange in fix: out of bounds SIMD access #5140
• chore: improve logs around the parser error by @romange in chore: improve logs around the parser error #5144
• feat: introduce resp validator by @romange in feat: introduce resp validator #5147
• chore: minor fixes in geo_family by @romange in chore: minor fixes in geo_family #5149
• refactor: geosearch cmd with CmdArgParser by @BorysTheDev in refactor: geosearch cmd with CmdArgParser #5153
• feat: add Georadius command by @BorysTheDev in feat: add Georadius command #5158
• pytest: Add sleep to let master stop propagate to replica by @abhijat in pytest: Add sleep to let master stop propagate to replica #5157
• chore: enable simd optimizations for aarch64 by @romange in chore: enable simd optimizations for aarch64 #5150
• fix: prevent crash in GEORADIUS argument parsing by @vyavdoshenko in fix: prevent crash in GEORADIUS argument parsing #5166
• chore: more tests around dash table by @romange in chore: more tests around dash table #5168
• chore: Add option to ignore parsing errors in replay tool by @vyavdoshenko in chore: Add option to ignore parsing errors in replay tool #5167
• fix: Prevent crash in PFCOUNT when using keys of different types by @vyavdoshenko in fix: Prevent crash in PFCOUNT when using keys of different types #5161
• chore: lower log level for harmless "Invalid Json path" warnings by @vyavdoshenko in chore: lower log level for harmless "Invalid Json path" warnings #5163
• chore: remove global current_reply_size by @romange in chore: remove global current_reply_size #5057
• replication: Ensure snapshot fiber joined in all code paths by @abhijat in replication: Ensure snapshot fiber joined in all code paths #5171
• chore: refactorings in dash code. by @romange in chore: refactorings in dash code. #5178
• chore(dash_table): pass memory resource to segment by @romange in chore(dash_table): pass memory resource to segment #5176
• fix: show proactor kind on test failures by @kostasrim in fix: show proactor kind on test failures #5170
• fix(parser): Parse correctly split simple response or error msg in client mode by @mkaruza in fix(parser): Parse correctly split simple response or error msg in client mode #5177
• chore: introduce azure snapshot storage skeleton by @romange in chore: introduce azure snapshot storage skeleton #5104
• fix: improve parsing of client facing responses by @romange in fix: improve parsing of client facing responses #5181
• server(fix) : SCAN command reduce latency by @vyavdoshenko in server(fix) : SCAN command reduce latency #5151
• chore(json_family): Clean up the memory tracking code for JSON mutate operations. FOURTH PR by @BagritsevichStepan in chore(json_family): Clean up the memory tracking code for JSON mutate operations. FOURTH PR #5070
• feat(search_family): Ignore unsupported options in the FT.CREATE command by @BagritsevichStepan in feat(search_family): Ignore unsupported options in the FT.CREATE command #5188
• fix (server): fix replication partial sync flow by @adiholden in fix (server): fix replication partial sync flow #5189
• feat: add flags support for huffman encoding by @romange in feat: add flags support for huffman encoding #5179
• fix server: expire if need check valid condition fix by @adiholden in fix server: expire if need check valid condition fix #5193
• chore: minor dash fixes by @romange in chore: minor dash fixes #5191
• fix(json_family): Fix memory tracking for the JSON.DEL command. FIFTH PR by @BagritsevichStepan in fix(json_family): Fix memory tracking for the JSON.DEL command. FIFTH PR #5075
• fix: prevent crash on KEYLOCK_ACQUIRED check for NO_KEY_TRANSACTIONAL commands by @vyavdoshenko in fix: prevent crash on KEYLOCK_ACQUIRED check for NO_KEY_TRANSACTIONAL commands #5185
• fix(cluster_family): Cancel slot migration from incoming node on OOM by @mkaruza in fix(cluster_family): Cancel slot migration from incoming node on OOM #5000
• fix: timeout for test_seeder_key_target by @BorysTheDev in fix: timeout for test_seeder_key_target #5198
• fix: crash in HLL PFMERGE with corrupted data by @vyavdoshenko in fix: crash in HLL PFMERGE with corrupted data #5201
• fix: ZCOUNT crash when min > max by @vyavdoshenko in fix: ZCOUNT crash when min > max #5205
• fix: SearchStats race condition crash during concurrent FT operations by @vyavdoshenko in fix: SearchStats race condition crash during concurrent FT operations #5207
• fix(pytest): Sleep in sporadically failing test_slot_migration_oom pytest by @mkaruza in fix(pytest): Sleep in sporadically failing test_slot_migration_oom pytest #5214
• fix(transaction): DCHECK fail in non-atomic transactions by @romange in fix(transaction): DCHECK fail in non-atomic transactions #5217
• fix: seeder hset commands generation by @BorysTheDev in fix: seeder hset commands generation #5220
• fix(http_api): Return valid JSON string by @mkaruza in fix(http_api): Return valid JSON string #5210
• feat: add Lua GC flags by @BorysTheDev in feat: add Lua GC flags #5194
• fix: Unsupported status for Copy/Move/Rename operations by @vyavdoshenko in fix: Unsupported status for Copy/Move/Rename operations #5223
• (string_family): Add priliminary support for GAT by @abhijat in (string_family): Add priliminary support for GAT #5229
• chore: reduce leaky interface of DashCursor by @romange in chore: reduce leaky interface of DashCursor #5230
• chore: minor changes in dash code by @romange in chore: minor changes in dash code #5224
• test: add migration_restart_test with another slots set by @BorysTheDev in test: add migration_restart_test with another slots set #5231
• chore: bring back partial sync by @kostasrim in chore: bring back partial sync #5182
• fix: macos build by @vyavdoshenko in fix: macos build #5234
• chore: maintain meta information around continous transactions by @romange in chore: maintain meta information around continous transactions #5232
• fix: dcheck trigger in JournalFlushGuard constructor by @kostasrim in fix: dcheck trigger in JournalFlushGuard constructor #5241
• fix: typo in AclFamily by @vyavdoshenko in fix: typo in AclFamily #5244
• fix(pubsub): Skip stale pubsub messages by @dranikpg in fix(pubsub): Skip stale pubsub messages #5246
• chore: improve partial sync and test by @kostasrim in chore: improve partial sync and test #5195
• chore(search): Add self time and resp3 format to ft.profile by @dranikpg in chore(search): Add self time and resp3 format to ft.profile #5250
• fix: test_pubsub_unsubscribe test timeout increased by @vyavdoshenko in fix: test_pubsub_unsubscribe test timeout increased #5253
• chore: add corruption check for lpSetTotalBytes by @romange in chore: add corruption check for lpSetTotalBytes #5252
• chore: Pull latest helio by @mkaruza in chore: Pull latest helio #5254
• fix: BITFIELD: overflow handling, type validation, and return values by @vyavdoshenko in fix: BITFIELD: overflow handling, type validation, and return values #5242
• fix: prevent writing zero size fields into log file by @BorysTheDev in fix: prevent writing zero size fields into log file #5256
• fix(transaction): Fix non-transactional multi/exec transaction access by @dranikpg in fix(transaction): Fix non-transactional multi/exec transaction access #5260
• fix(search_family): Fix empty key bug by @BagritsevichStepan in fix(search_family): Fix empty key bug #5264
• chore: remove old ZIPLIST encoding during snapshotting by @romange in chore: remove old ZIPLIST encoding during snapshotting #5267
• chore: stats for lsn buffer by @kostasrim in chore: stats for lsn buffer #5243
• chore: minor fixes in dash table by @romange in chore: minor fixes in dash table #5274
• fix(json): Ensure full deletion for multi-match JSONPaths by @vyavdoshenko in fix(json): Ensure full deletion for multi-match JSONPaths #5270
• fix(reply_builder): Flush if last message didn't reply and flush but supose to by @mkaruza in fix(reply_builder): Flush if last message didn't reply and flush but supose to #5269
• fix(workflow): upload dev release to aws s3 also by @Abhra303 in fix(workflow): upload dev release to aws s3 also #5262
• chore: move MemoryBytesFlag to facade by @romange in chore: move MemoryBytesFlag to facade #5281
• dfly_bench: Support terminating early by @abhijat in dfly_bench: Support terminating early #5285
• refactor: remove is_oom flag from ConnectionContext by @BorysTheDev in refactor: remove is_oom flag from ConnectionContext #5216
• fix(protocol_client): Check if sock_ is not null before calling class method by @mkaruza in fix(protocol_client): Check if sock_ is not null before calling class method #5291
• chore: add -skip-time-min flag to skip initial N minutes during replay traffic by @vyavdoshenko in chore: add -skip-time-min flag to skip initial N minutes during replay traffic #5288
• fix(json_family): Fix bugs where JSON commands were modifying values of other data types by @BagritsevichStepan in fix(json_family): Fix bugs where JSON commands were modifying values of other data types #5266
• fix: Unsupported status 4 (XCLAIM command) by @vyavdoshenko in fix: Unsupported status 4 (XCLAIM command) #5300
• Revert "fix(stream_family): Fix stream memory tracking issues (fix(stream_family): Fix stream memory tracking issues #5024)" by @romange in Revert "fix(stream_family): Fix stream memory tracking issues (#5024)" #5301
• chore(dash): pass segment id to segment by @romange in chore(dash): pass segment id to segment #5298
• Revert "fix(stream_family): Fix memory tracking for the STREAMS (fix(stream_family): Fix memory tracking for the STREAMs. SECOND PR #4781)" by @romange in Revert "fix(stream_family): Fix memory tracking for the STREAMS (#4781)" #5304
• feat: Added store option for sort command. by @H4R5H1T-007 in feat: Added store option for sort command. #5095
• fix: prevent rename cmd crash by @BorysTheDev in fix: prevent rename cmd crash #5307
• fix(string_family): Add RemoveDoc for the SET command by @BagritsevichStepan in fix(string_family): Add RemoveDoc for the SET command #5306
• chore: minor refactoring around AutoUpdater by @romange in chore: minor refactoring around AutoUpdater #5310
• fix(transaction): Fix schedule queues by @dranikpg in fix(transaction): Fix schedule queues #4925
• chore(tiering): Test utils for faulty io by @dranikpg in chore(tiering): Test utils for faulty io #4928
• fix: huge sbf deserialization by @BorysTheDev in fix: huge sbf deserialization #5313
• fix: partial replication data loss by @kostasrim in fix: partial replication data loss #5297
• chore(string_family): Refactor to prep for adding the GAT command by @abhijat in chore(string_family): Refactor to prep for adding the GAT command #5289
• chore: add another benchmark test for the GlobMatcher by @romange in chore: add another benchmark test for the GlobMatcher #5317
• fix: memory tracking bug in rename and SET by @romange in fix: memory tracking bug in rename and SET #5315
• chore: Improve scope/aggregator usage in replies by @dranikpg in chore: Improve scope/aggregator usage in replies #5286
• chore(db_slice): Add object type to AddOrFind by @BagritsevichStepan in chore(db_slice): Add object type to AddOrFind #5305
• fix: fix db_capacity metrics description by @romange in fix: fix db_capacity metrics description #5318
• Implementation of memcache GAT and GATS commands by @abhijat in Implementation of memcache GAT and GATS commands #5257
• fix: client kill preempts in atomic section on shutdown by @kostasrim in fix: client kill preempts in atomic section on shutdown #5283
• fix(json): JSON.DEL recursive descent duplicate counting for nested same-key elements by @vyavdoshenko in fix(json): JSON.DEL recursive descent duplicate counting for nested same-key elements #5312
• chore(search_family): Add AddGenericDoc/RemoveGenericDoc methods by @BagritsevichStepan in chore(search_family): Add AddGenericDoc/RemoveGenericDoc methods #5320
• chore: integrate Valkey fixes into stringmatchlen_impl by @romange in chore: integrate Valkey fixes into stringmatchlen_impl #5323
• docs(readme): add translated documentation for pt-br by @Santosl2 in docs(readme): add translated documentation for pt-br #5276
• feat(json_family): Copy json_test.py::test_reset_key_to_string to json_family_test by @BagritsevichStepan in feat(json_family): Copy json_test.py::test_reset_key_to_string to json_family_test #5324
• fix: Handle bonus key + init error in squashing by @dranikpg in fix: Handle bonus key + init error in squashing #5303
• feat (server): add on move hook to dash insert new and bumpup by @adiholden in feat (server): add on move hook to dash insert new and bumpup #5278

New Contributors

• @Santosl2 made their first contribution in docs(readme): add translated documentation for pt-br #5276

Full Changelog: v1.30.0...v1.31.0

---

This discussion was created from the release v1.31.0.