v0.2: cascades optimizer, SIMD, and better storage

After 9 months' development, we are happy to announce a new version of RisingLight. Major changes in this version:

• New cascades-style binder / planner / optimizer framework powered by egg.
• New SIMD expression evaluation design with better performance.
• More efficient storage with better encoding and predicate push-downs.
• Better docs on the storage layer.

Thanks for your support!

What's Changed

• docs: migrate to new Slack community by @skyzh in #606
• docs: add architecture overview by @skyzh in #609
• feat(executor): introduce DataChunkBuilder and split chunks in TopN by @wangqiim in #610
• deps: upgrade prost 0.10 by @alissa-tung in #613
• feat(storage): record first_key in block_index_builder by @adlternative in #598
• feat(executor): introduce DataChunkBuilder and split chunks in Order … by @wangqiim in #612
• chore(ci): install protoc by direct download by @skyzh in #614
• feat(executor): implement SortMergeJoin executor by @D2Lark in #600
• chore(test): add more test case by @D2Lark in #616
• docs: add instructions to install protobuf by @skyzh in #617
• docs: add badges of dbdb.io, crates.io and docs.rs by @skyzh in #621
• feat(storage): add support composite sort key in MergeIterator by @BaymaxHWY in #622
• executor: use DataChunkbuilder to refactor ValuesExecutor by @JayiceZ in #623
• build: bump toolchain and deps, remove unused dep by @TennyZhuang in #629
• fix: make header correct when using group by by @shmiwy in #624
• refactor(planner): use AggInputRefResolver to handle group by expr by @Fedomn in #630
• feat(binder): support “primary key (a, b)” syntax by @shmiwy in #631
• feat: support multi ordered primary key by @shmiwy in #633
• community: add xxchan as maintainer by @skyzh in #634
• docs: add overview of storage by @skyzh in #635
• refactor(executor): refactor CopyFromFile using DataChunkBuilder by @noneback in #637
• fix(planner): enhance group by illegal column validation by @Fedomn in #642
• feat: use ref-cast for BlobRef cast by @Kikkon in #643
• feat(storage): support range-filter scan by sort key by @shmiwy in #644
• feat(optimizer): push the filter down below the agg by @lokax in #648
• binder: primary key should by-default be non-null by @Ted-Jiang in #649
• chore: remove Mutex in database/schema/table catalog by @chowc in #650
• feat: support sort agg by @Kikkon in #646
• fix: typo and clippy warnings by @yuzi-neko in #652
• feat(binder): support Having clause and more flexible OrderBy by @lokax in #651
• chore(sqllogictest): use libtest-mimic as test harness by @skyzh in #656
• chore(test): ensure single thread runtime for sqllogictest by @skyzh in #657
• feat(test): add sqlplannertest by @skyzh in #661
• feat(cli): support multi-line interactive mode by @Gun9niR in #662
• chore: upgrade dependencies by @wangrunji0408 in #663
• fix: 'unsupported command' everywhere by @cadl in #664
• fix: typo by @GoGim1 in #665
• feat(storage): use variable size encoding to compress rle count by @xinchengxx in #666
• fix: create table with invalid primary keys constraints by @GoGim1 in #667
• chore: bump dependencies by @skyzh in #672
• docs: add logo for RisingLight by @skyzh in #671
• fix: append NULLs when inserting value with incomplete columns by @GoGim1 in #673
• feat(storage): support dict encoding by @Kikkon in #668
• feat(optimizer): support column pruning for some operators by @lokax in #653
• fix(storage): reserve correct size by @lokax in #678
• chore: bump dependencies, toolchain, and adapt to new sqlparser by @skyzh in #681
• chore: add github codespace devcontainer configuration by @yeya24 in #680
• refactor: refactor risinglight-sqllogictest by @xxchan in #682
• refactor: remove unnecessary current dir by @xxchan in #683
• storage: use Arc during pin and unpin by @chaixuqing in #685
• chore(storage): select compaction strategy using statistics by @chaixuqing in #689
• feat(planner): plan select distinct by @lokax in #686
• feat(executor): support perfect hash aggregate by @lokax in #690
• fix: add row_handler to schema by @yinfredyue in #692
• chore: bump toolchain and update dependencies by @wangrunji0408 in #695
• feat: introduce ordered float by @wangrunji0408 in #699
• feat(storage): support dict encoding in primitive column builder by @Kikkon in #701
• feat: make binary operations auto-vectorized by @lokax in #700
• feat: introduce next-gen planner & optimizer by @wangrunji0408 in #702
• feat(binder): introduce egg binder for queries by @wangrunji0408 in #705
• fix(storage): checksum BlockType in block header by @unconsolable in #704
• feat(planner): support distinct on by @wangrunji0408 in #706
• refactor(type): unify DataTypeKind by @wangrunji0408 in #708
• feat(storage): generic nullable block encoding by @unconsolable in #709
• feat(binder): egg binder for drop statement by @eliasyaoyc in #711
• feat(planner_v2): add type checker by @wangrunji0408 in #717
• refactor(storage): replace primitive nullable encoding with generic way by @unconsolable in #713
• feat(binder): new binder for copy, delete and create_table by @eliasyaoyc in #719
• fix(type): add NULL type and fix type analysis by @wangrunji0408 in #721
• feat(planner_v2): support explain by @wangrunji0408 in #722
• refactor(executor): simply drop the futures to cancel a query by @wangrunji0408 in #720
• feat(planner_v2): cost function and multi-stage optimization by @wangrunji0408 in #723
• feat: introduce executor v2 for the new planner by @wangrunji0408 in #725
• feat(planner_v2): support standalone VALUES clause by @wangrunji0408 in #726
• feat(storage): support NULL on blob and char columns by @unconsolable in #724
• refactor(fetch size): add fast return and compare ROWSET MAX OUTPUT by @eliasyaoyc in #716
• feat(executor_v2): migrate executors to run TPCH by @wangrunji0408 in #727
• feat(planner_v2): pass all sqllogictests by @wangrunji0408 in #728
• chore: enable query engine v2 by default by @wangrunji0408 in #729
• feat(rt): support tokio-console by @wangrunji0408 in #730
• fix(perf): fix performance regression in HashJoin & HashAgg by @wangrunji0408 in #731
• chore: bump toolchain and apply async-fn-in-trait by @wangrunji0408 in #732
• chore(ci): remove bench workflow by @skyzh in #734
• build: bump deps by @TennyZhuang in #735
• doc: Storage basics documentation by @yinfredyue in #738
• fix(storage): compaction type error by @yinfredyue in #737
• chore: bump sqllogictest to 0.9.0 by @xxchan in #736
• feat(storage): Dict encoding for compaction by @yinfredyue in #740
• perf(expr): apply auto-vectorization and remove explicit SIMDs by @wangrunji0408 in #741
• release: v0.2 by @skyzh in #742
• perf(expr): further optimize performance by @wangrunji0408 in #744

New Contributors

• @wangqiim made their first contribution in #610
• @JayiceZ made their first contribution in #623
• @shmiwy made their first contribution in #624
• @noneback made their first contribution in #637
• @Kikkon made their first contribution in #643
• @lokax made their first contribution in #648
• @Ted-Jiang made their first contribution in #649
• @chowc made their first contribution in #650
• @yuzi-neko made their first contribution in #652
• @Gun9niR made their first contribution in #662
• @cadl made their first contribution in #664
• @GoGim1 made their first contribution in #665
• @xinchengxx made their first contribution in #666
• @yeya24 made their first contribution in #680
• @chaixuqing made their first contribution in #685
• @yinfredyue made their first contribution in #692
• @unconsolable made their first contribution in #704
• @eliasyaoyc made their first contribution in #711

Full Changelog: v0.1.3...v0.2.0