ParadeDB Roadmap - 2024

[philippemnoel]

ParadeDB Roadmap

💡 Next Roadmap: https://github.com/orgs/paradedb/discussions/2041

The goal of this discussion board is to transparently communicate our development plans to the community. Please join the discussion, request new features, vote for existing ones, and contribute your ideas!

Scheduled

Q4 2024 — By December 31, 2024

Task	Status	Comments
Multiple concurrent Tantivy writers	✅	Enables concurrent inserts/updates to BM25 indexes
CustomScan nodes for pg_search	✅	Allows most Postgres operators to be pushed down to Tantivy, improves score_bm25 syntax, optimizes join performance
Fast "top N" queries	✅
Make faceted search results fully MVCC-compliant	✅
Fast facets	✅
Replace dynamic schema generation API	✅	Search indexes no longer need to create a new schema in the user's database
Support for partitioned BM25 indexes	✅
Improve BM25 index creation times	✅
CREATE INDEX syntax	✅	Use the Postgres CREATE INDEX syntax instead of CALL for creating BM25 indexes
Provide pre-built macOS binaries	✅
Convert pg_search to Postgres block storage	✅	This will enable high availability and buffer cache
ParadeDB BYOC	✅	Initial implementation of PAradeDB BYOC, a managed deployment of ParadeDB in the customer's own cloud account

Unscheduled

Issues and features not yet prioritized. If any of these features are essential to you, please contact [email protected] to discuss prioritizing them.

• [In Progress] Postgres OTEL sink endpoint
• [In Progress] ORM/Libraries Support
  • Ecto (Elixir)
  • Laravel (PHP)
  • [In Progress] Npgsql (.NET)
  • [In Progress] Django (Python)
  • [In Progress] Prisma
  • Drizzle
  • Knex
• [In Progress] Stress test tool for ParadeDB pg_search
• Distributed full-text search
• ParadeDB Cloud — Hosted ParadeDB-as-a-Service
• Data migration tool from Elastic — “pg_dump" for Elastic

[GPF199541]

Distributed full-text search

[philippemnoel]

You can get very far on single-node. ParadeDB, since it doesn't require denormalizing, can do search over individual tables and JOINs the result. As such, even clusters that are 100s of TBs can be supported already.

We recently added support for parallelized queries on a single node, which also improve performance significantly. If true distributed search is a key requirement for you, I'd love to chat and learn more about your needs.