[DISCUSS] Review broker features for a feature matrix

[tisonkun]

Proposal

Motivation

During the past years, the Apache Pulsar community has developed a lot of outstanding features to serve the objects that Apache Pulsar is a distributed messaging and streaming platform built for the cloud.

Obviously, such objects mainly attract enterprise users. While most of these features work well in the production environment, the Apache Pulsar project still has user-visible features which are under development and are not ready for production.

For example, many users report issues with the Pulsar Helm Chart, the Pulsar Manager, the Pulsar SQL plugin, etc. While it’s usual that the community develops features one by one from time to time, it helps users to gain a proper expectation for visible features.

Thus, I’d propose building a feature matrix proposal for Apache Pulsar. Compared to the existing client feature matrix, this new feature matrix is about broker-side features (including options exposed by Bookie).

What to add, and where is the source?

A broker-side feature matrix in the following form:

(Categorized) Feature Name	Experimental Since	Beta Since	Stable Since	Document
Produce / Exclusive			All versions	..
Delay Message V2		3.0.0

… where

• (Categorized) Feature Name includes the feature’s canonical name, with an optional category like “Transaction”, “Functions”, etc.
• Experimental Since indicates a version; since then, the feature has been experimental. Experimental means it is not meant to be used in production yet.
• Beta Since indicates a version; since then, the feature has been beta. Beta means you can use it in production, but it still needs battle-testing.
• Stable Since indicates a version; since then, the feature has been stable. Stable means it is battle-tested. You can use it in production with full confidence.
• Document includes links to feature docs.

The source of features come from:

• Docs https://pulsar.apache.org/docs/3.1.x/
• PIPs https://github.com/apache/pulsar/tree/master/pip + https://github.com/apache/pulsar/wiki#pulsar-improvement-proposals
• Your contributions!

Here is the table of features I’ve collected yet:

(Categorized) Feature Name	Experimental Since	Beta Since	Stable Since	Document
Produce / Exclusive			All versions
Produce / Shared			All versions
Subscription / Exclusive			All versions	https://pulsar.apache.org/docs/3.1.x/concepts-messaging/#exclusive
Subscription / Failover			All versions	https://pulsar.apache.org/docs/3.1.x/concepts-messaging/#failover
Subscription / Shared			All versions	https://pulsar.apache.org/docs/3.1.x/concepts-messaging/#shared
Subscription / KeyShared				https://pulsar.apache.org/docs/3.1.x/concepts-messaging/#key_shared
Transaction / Failover Subscription
Transaction / Shared Subscription
Transaction / Segmented Snapshot
Schema / KeyValue
Schema / Avro
Schema / JSON
Pulsar SQL				https://pulsar.apache.org/docs/3.1.x/sql-overview/
Functions / Java
Functions / Golang
Functions / Python
Multi-Tenancy				https://pulsar.apache.org/docs/3.1.x/concepts-multi-tenancy/
Geo-Replication				https://pulsar.apache.org/docs/3.1.x/concepts-replication/

Implementation steps

• Agree on the direction.
• Populate more feature items;
• Meanwhile, I’ll develop a feature matrix page to host this table.
• Finish its first version.

[tisonkun]

Flink provides a Roadmap in radar format which we can follow, while the content is the same as the features table listed above.

[tisonkun]

Preparing a sample page for broker features - apache/pulsar-site#720