v2.11.2

Broker

• Fix broker restart logic #20113
• Fast fix infinite HTTP call getSubscriptions caused by wrong topicName #20131
• Fix getPartitionedStats miss subscription's messageAckRate #19870
• Fix deadlock of zkSessionWatcher when zkConnection loss #20122
• Fix Return value of getPartitionedStats doesn't contain subscription type #20210
• Producer created by replicator is not displayed in topic stats #20229
• Fix infinite ack of Replicator after topic is closed #20232
• Skip split bundle if only one broker #20190
• Fix RoaringBitmap.contains can't check value 65535 #20176
• Get the lowest PositionImpl from NavigableSet #18278
• Fix the behavior of delayed message in Key_Shared mode #20233
• Fix use error TimeUnit to record publish latency #20074
• Fix the thread safety issue of BrokerData#getTimeAverageData access #19889
• Fix can't send ErrorCommand when a message's value is null #19899
• Make LedgerOffloaderFactory can load the old nar #19913
• Ignore and remove the replicator cursor when the remote cluster is absent #19972
• Fix the reason label of authentication metrics #20030
• Optimize the ack/send future in TransactionImpl #20271
• Fix NPE caused by topic publish rate limiter #20302
• Fix default bundle size used while setting bookie affinity #20250
• Fix ledger left in OPEN state when inactiveLedgerRollOverTimeMs is enabled #20276
• Fix deadlock of metadata store #20189
• Fix class name typo PrecisPublishLimiter to "Precise" #20310
• Fix consume stuck of shared streaming dispatcher #18315
• Fix entry filter feature for the non-persistent topic #20141
• Fix partitioned __change_events topic is policy topic #20392
• If a ledger is lost, the cursor mark delete position cannot forward #18620
• Invalidate metadata children cache after key deleted #20363
• Avoid PersistentSubscription.expireMessages logic check backlog twice #20416
• Switch to the metadata store thread after zk operation #20303
• Fix skip message API when hole messages exist #20326
• Do not filter system topic while shedding #18949
• Restore solution for certain topic unloading race conditions #20527
• There are two same-named managed ledgers in the one broker #18688
• REST Client Producer fails with TLS only #20535
• Save createIfMissing in TopicLoadingContext #19993
• Unwrap the completion exception #20396
• Release orphan replicator after topic closed #20582
• Fix the publish latency spike issue with a large number of producers #20607
• Support cgroup v2 by using jdk.internal.platform.Metrics in Pulsar Loadbalancer #16832
• Validate authz earlier in delete subscription logic #20549
• Fix the publish latency spike from the contention of MessageDeduplication #20647
• Adding the missed bookie id in the registration manager #20641
• Fix NPE when resetting Replicator's cursor by position #20597
• Topic policy can not be work well if replay policy message has any exception #20613
• Bookie Info lost by notification race condition #20642
• Update new bundle-range to policies after bundle split #17797
• Fix consumer can receive aborted txn message when readType is replay #19815
• Fix return the earliest position when query position by timestamp #20457

Clients

• [Java] Use scheduled executor in BinaryProtoLookupService #20043
• [Java] Release the orphan producers after the primary consumer is closed #19858
• [Java] Fix NPE when acknowledging multiple messages #19874
• [Java] Cache empty schema version in ProducerImpl schemaCache #19929
• [Java] Fix where the function getMsgNumInReceiverQueue always returns 0 when using message listener #20245
• [Java] Fix deadlock issue of consumer while using multiple IO threads #20669
• [Java] Make the whole grabCnx() progress atomic #20595
• [Java] Messages lost when consumer reconnect #20695

Pulsar IO and Pulsar Functions

• [Functions] Use functions classloader in TopicSchema.newSchemaInstance() to fix ClassNotFoundException when using custom SerDe classes #20115
• [Functions] Make pulsar-admin support update py/go with package url #19897
• [Functions] Fix JavaInstanceStarter inferring type class name error #20426
• [Functions] Go functions must retrieve consumers by non-partitioned topic ID #20413
• [Functions] Fix function update error #19895
• [Functions] Reset idle timer correctly #20450 #20450
• [Functions] Go functions need to use static grpcPort in k8s runtime #20404
• [Functions] Configure pulsar admin for TLS #20533
• [Functions] Make KubernetesRuntime translate characters in function tenant, namespace, and name during function removal to avoid label errors #19584
• [Functions] Exit JVM when the main thread throws an exception #20689
• [IO Connector] Close the Kafka source connector if there is uncaught exception #20480
• [IO Connector] Close the Kafka source connector got stuck #20698

Observability

• [Broker] Fix the partitioned publisher topic stat aggregation bug #18807
• [Broker] Fix topic with double quote breaks the Prometheus format #20230
• [Broker] Add JVM start time metric #20381
• [Offload] Fix offload metrics error #20366

CLI

• [CLI] Allow pulser-client consumer to create a replicated subscription #20316
• [CLI] Update nar maven plugin version to fix excessive downloads #20410
• [CLI] Speed up OWASP dependency check in Pulsar CI workflow #20412
• [CLI] Fulfill add-opens to function-localrunner also #20417
• [Admin] Return BAD_REQUEST on cluster data is null for createCluster #20346
• [Admin] Report earliest msg in partitioned backlog #19465

Others

• [Build] Duplicate entry when merging services #17659
• [Offload] Use filesystem offloader with class not found error #20365
• [Perf] PerformanceProducer does not produce expected number of messages #19775
• [SQL] Remove useless configuration for Pulsar SQL #20605
• [WS] Remove unnecessary ping/pong implementation #20605
• [Schema] Only handle exception when there has #20730

Library updates

• Upgra...

v3.0.0

Important notice

• Upgrade the RocksDB version to 7.9.2 to keep sync with BookKeeper's RocksDB dependency in #20072
• Introduce consistent hash ring to distribute the load of multiple topics subscription in #19502
• Remove default 30s ackTimeout when setting DLQ policy on java consumer in #19486
• Require authRole is proxyRole to set originalPrincipal in #19455
• Update Elasticsearch sink idle cnx timeout to 30s in #19377
• Move checked exception into builder when newTransaction. in #19356
• TokenAuthenticationState: authenticate token only once in #19314
• OneStageAuth State: move authn out of constructor in #19295
• Update AuthenticationProvider to simplify HTTP Authn in #19197
• Remove AuthorizationProvider methods deprecated in 2.7 and 2.9 in #19182
• Deprecate blocking AuthorizationService, AuthorizationProvider methods in #19180
• Reject create non existent persistent partitions. in #19086
• Omit making a copy of CommandAck when there are no broker interceptors in #18997
• Use correct file path for tls trust certificates in #18712
• Enable certificate refresh for Quorum and Netty Servers in #18097
• Remove timestamp from Prometheus metrics in #17419
• Revert 5895: fix redeliveryCount in #17060
• Fix producer/consume permission can’t get v1/schema in #16018
• Autorecovery default reppDnsResolverClass to ZkBookieRackAffinityMapping in #15640
• Allow to build and push multi-arch Docker images in #19432

PIPs

• PIP-160 Metrics stats of Transaction buffered writer #15370
• PIP-167 Make it Configurable to Require Subscription Permission #15576
• PIP-169 Support split bundle by flow or qps #16782
• PIP-188: Cluster migration or Blue-Green cluster deployment support in Pulsar #16551
• PIP-192: New Pulsar Broker Load Balancer #16691
• PIP-214: Add broker level metrics statistics and expose to prometheus #18056
• PIP-218: Consumer batchReceive() single partition every receive #18182
• PIP-229: Add a common interface to get fields of the MessageIdData #18950(#18950)
• PIP-250: Add proxyVersion to CommandConnect #19623(#19623)
• PIP-257: Add Open ID Connect Support to Server Components #19771

Broker

• [Broker] incomplete PIP-97: Asynchronous Authentication Provider #12105
  • Implement for ServerCnx by @michaeljmarshall in #19409
  • Implement for ProxyConnection by @michaeljmarshall in #19292
• [Broker] incomplete PIP-180: Shadow Topic, an alternative way to support readonly topic ownership. #16153
  • Add ShadowManagedLedgerImpl by @Jason918 in #18265
  • Support shadow topic creation. by @Jason918 in #17711
  • Add Shadow Replicator by @Jason918 in #17371
• [Broker] PIP-192: New Pulsar Broker Load Balancer #16691
  • Fix getLastMessageId for compressed payload(And add compression and maxBatchSize for the load balance system topic) by @heesung-sn in #20087
  • Improve TransferSheder for overload outlier for large clusters by @heesung-sn in #20059
  • Update the lookup data path to support deployment and rollback by @Demogorgon314 in #19999
  • Filter the transfer dest broker by @Demogorgon314 in #19958
  • Updated metrics and cleanup broker selector by @heesung-sn in #19945
  • Support delete namespace bundle admin API by @Demogorgon314 in #19851
  • Improved Auto Unload Logic by @heesung-sn in #19813
  • Support broker isolation policy by @Demogorgon314 in #19592
  • Add large topic count filter by @Demogorgon314 in #19613
  • Added VersionId in ServiceUnitStateData by @heesung-sn in #19620
  • Added Deleted and Init states in ServiceUnitState by @heesung-sn in #19546
  • Added broker and top-bundles load reporters by @heesung-sn in #19471
  • Added TransferShedder by @heesung-sn in #18865
  • Added ServiceUnitStateCompactionStrategy by @heesung-sn in #19045
  • Added TopicCompactionStrategy for StrategicTwoPhaseCompactor and TableView. by @heesung-sn in #18195
  • Implement load data store by @Demogorgon314 in #18777
  • Implement broker registry for new load manager by @Demogorgon314 in #18810
  • Added ServiceUnitStateChannelImpl by @heesung-sn in #18489
• [Broker] Pip-257: Add AuthenticationProviderOpenID by @michaeljmarshall in #19849
• [Broker] Make LeastResourceUsageWithWeight thread safe by @lifepuzzlefun in #20159
• [Broker] Support disabling delayed bucket merging. by @mattisonchao in #20155
• [Broker] Supply download auth params when provided for k8s runtime by @michaeljmarshall in #20144
• [Broker] AuthenticationProviderOpenID k8s error logs by @michaeljmarshall in #20135
• [Broker] Implement authenticateAsync for AuthenticationProviderList by @michaeljmarshall in #20132
• [Broker] Fix broker restart logic by @wolfstudy in #20113
• [Broker] Ensure previous delayed index be removed from snapshotSegmentLastIndexTable & Make load operate asynchronous by @coderzc in #20086
• [Broker] Fix avoid future of clear delayed message can't complete by @coderzc in #20075
• [Broker] Refresh auth data if ProxyLookupRequests by @michaeljmarshall in #20067
• [Broker] Fix flaky testCreateTopicWithZombieReplicatorCursor by @BewareMyPower in #20037
• [Broker] Prevent range conflicts with Key Shared sticky consumers when TCP/IP connections get orphaned by @lhotari in #20026
• [Broker] Fix leader broker log by @nicoloboschi in #19987
• [Broker] Make LedgerOffloaderFactory can load the old nar. by @horizonzy in #19913
• [Broker] Ignore and remove the replicator cursor when the remote cluster is absent by @BewareMyPower in #19972
• [Broker] Fix DeadLetterProducer creation callback blocking client io thread. by @lifepuzzlefun in #19930
• [Broker] Merge multiple buckets at once by @coderzc in #19927
• [Broker] Fix the loss of bundle stats data reported to zookeeper, when the updateStats method is executed by @lordcheng10 in #19887
• [Broker] Clear delayed message when unsubscribe & Make clear operation asynchronous by @coderzc in #19901
• [Broker] Fix can't send ErrorCommand when message is null value by @coderzc in #19899
• [Broker] PIP-240: new public method unloadSubscription in PersistentTopic by @poorbarcode in #19737
• [Broker] PIP-250: Add proxyVersion to CommandConnect by @michaeljmarshall in #19618
• [Broker] Make bucket merge operation asynchronous by @coderzc in #19873
• [Broker] Suppress err...

v2.9.5

What's Changed

• [fix] [tx] [branch-2.9] Transaction buffer recover blocked by readNext by @poorbarcode in #18970
• [branch-2.9] Fix compile error caused by cherry-pick #19167. by @mattisonchao in #19241
• [fix][broker] Support deleting partitioned topics with the keyword -partition- (#19230) by @mattisonchao in #19240
• [improve][websocket][branch-2.9] Add ping support by @nodece in #19245
• [fix][websocket][branch-2.9] Fix webSocketPingDurationSeconds config by @nodece in #19293
• [branch-2.9][cherry-pick] Close TransactionBuffer when MessageDeduplication#checkStatus failed by @tjiuming in #19287
• [fix][Branch-2.9] Fix bookie JVM param BOOKIE_MEM, BOOKIE_GC miss problem. by @horizonzy in #19441
• [fix][broker][branch-2.9] Fix geo-replication admin by @nodece in #19614
• [Branch-2.9] [Build] Fix branch-2.9 build failed by @hangc0276 in #19645
• [Branch-2.9] [build] fix build failed by @hangc0276 in #19669
• [cherry-pick][branch-2.9] Add UncaughtExceptionHandler for every thread pool (#18211) by @coderzc in #19677
• [fix][branch-2.9] Support zookeeper read-only config by @horizonzy in #19693
• [cherry-pick][branch-2.9] Fix validatePersistencePolicies that Namespace/Topic persistent policies cannot set to < 0 by @dao-jun in #19687
• [cherry-pick][branch-2.9] fix txn coordinator recover handle committing and aborting txn race condition. #19201 by @thetumbled in #19699
• [cherry-pick][branch-2.9] Fix race condition while updating partition number (#19199) by @coderzc in #19691
• [improve][branch-2.9]upgrade the bookkeeper version to 4.14.7 by @horizonzy in #19178
• [fix][cli][branch-2.9] Fix release tool by @liangyepianzhou in #19712
• [Branch-2.9] [build] fix build failed by @coderzc in #19743
• [cherry-pick][branch-2.9] Upgrade Netty to 4.1.87.Final (#19417) by @coderzc in #19718
• [fix][license][brach-2.9] Fix license by @coderzc in #19935
• [branch-2.9][fix][broker] Fix index generator is not rollback after entries are failed added by @gaozhangmin in #19978

New Contributors

• @dao-jun made their first contribution in #19687

Full Changelog: v2.9.4...v2.9.5

v2.11.1

Broker

• AbstractBatchedMetadataStore - use AlreadyClosedException instead of IllegalStateException #19284
• Add ref count for sticky hash to optimize the performance of Key_Shared subscription #19167
• Pass subName for subscription operations in ServerCnx #19184
• Enable custom metadata stores #19208
• Topics stats shows msgBacklog but there reality no backlog #19275
• Fix open cursor with null-initialPosition result with earliest position #18416
• Added isActive in ManagedCursorImpl #19341
• Replaced checkBackloggedCursors with checkBackloggedCursor(single subscription check) upon subscription #19343
• Close TransactionBuffer when MessageDeduplication#checkStatus failed #19289
• Fix the incorrect total size if use ML interceptor #19404
• Support deleting partitioned topics with the keyword -partition- #19230
• Correct the prompt message #17009
• Expect msgs after server initiated CloseProducer #19446
• Fix Npe thrown by splitBundle #17370
• Fix executeWithRetry result is null #17694
• ManagedLedger: move to FENCED state in case of BadVersionException #17736
• Fix can not revoke permission after update topic partition #17393
• Allow deletion of a namespace that was left in deleted status #17592
• Add UncaughtExceptionHandler for every thread pool #18211
• Support setting ForceDeleteNamespaceAllowed dynamically #18181
• Support setting forceDeleteTenantAllowed dynamically #18192
• Remove locallyAcquiredLock when removeOwnership #18197
• Update the log print content of createSubscriptions #18024
• Unify time unit at dropping the backlog on a topic #17957
• Fix mutex never released when trimming #17911
• Fix getChildren in MemoryMetadataStore and EtcdMetadataStore #18172
• Fix the order of resource close in the InMemoryDelayedDeliveryTracker #18000
• Allow to configure and disable the size of lookahead for detecting fixed delays in messages #17907
• Make BookieId work with PulsarRegistrationDriver #17922
• In the trimDeletedEntries method, release the removed entry #18305
• Make Consumer#equals more effective #18662
• PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates #18133
• Fix PulsarRegistrationClient and ZkRegistrationClient not aware rack info problem #18672
• Simplify extract entryMetadata code in filterEntriesForConsumer #18729
• Fix uncompleted future when getting the topic policies of a deleted topic #18824
• Close transactionBuffer after MessageDeduplication#checkStatus failed #19157
• GetLastMessageId returns a wrong batch index of last message if enabled read compacted #18877
• Fix multi invocation for ledger createComplete #18975
• Fix txn coordinator recover handle committing and aborting txn race condition #19201
• Reset individualDeletedMessagesSerializedSize after acked all messages #19428
• Fix potential NPE cause future never complete #19415
• Fix the return value of canConsumeAsync #19412
• Close TransactionBuffer when creating persistent topic timeout #19384
• Handle changeToReadyState failure correctly in TC client #19308
• The atomicity of multiple fields of ml is broken #19346
• MessagesConsumedCounter of NonDurableCursor was initialized incorrectly #19355
• Improve PersistentMessageExpiryMonitor expire speed when ledger not existed #17842
• Always send correct transaction id in end txn response #19137
• Allow superusers to abort transactions #19467
• Make ServerCnx#originalAuthData volatile #19507
• Validate originalPrincipal earlier in ServerCnx #19270
• Require authRole is proxyRole to set originalPrincipal #19455
• ServerCnx broken after recent cherry-picks #19521
• ServerCnx: go to Failed state when auth fails #19312
• Add test to verify authRole cannot change #19430
• Make authentication refresh threadsafe #19506
• OneStageAuth State: move authn out of constructor #19295
• Allow proxy to pass same role for authRole and originalRole #19557
• Copy command fields and fix potential thread-safety in ServerCnx #19517
• Topic load fail by ledger lost #19444
• Incorrect service name selection logic #19505
• Make the service name resolver cache of PulsarWebResource expire after access #19532
• Simplify enabling Broker, WS Proxy hostname verification #19674
• Improve tls config on replication client and cluster cli #19588
• Fix geo-replication admin #19686
• Filter system topic when getting topic list by binary proto #19667
• Fixed history load not releasing #19560
• Fix loadbalance score calculation problem #19420
• Use shrink map for trackerCache #19534
• Fix individual ack batch message with transaction abort redevlier duplicate messages #14327
• Fix receive duplicated messages due to pendingAcks in PendingAckHandle #19581
• Fail to update partition meta of topic due to ConflictException: subscription already exists for topic #17251
• Fix catching ConflictException when update topic partition #17374
• Only create extended partitions when updating partition number #17349
• Create replicated subscriptions for new partitions when needed #18659
• Fix race condition while updating partition number #19199
• Topic close failure leaves subscription in a permanent fence state #19692
• Fix potential exception cause the policy service init fail #19746
• Allow version to start positive and grow by more than one #19503
• Fix deadlock causes session notification not to work #19754
• Fix issue where msgRateExpired may not refresh forever #19759
• Delete topic failed if disabled system topic #19735
• Fix index generator is not rollback after entries are failed added #19727
• Store the original authentication data #19519
• Copy subscription properties during updating the topic partition number #19800
• Fix close borrowed executor #19761
• Fixed race condition between ResourceLock update and invalidation #19817
• Counter of pending send messages in Replicator incorre...

v2.10.4

Broker

• [improve][broker] Add ref count for sticky hash to optimize the performance of Key_Shared subscription (#19167)
• [fix][broker] Pass subName for subscription operations in ServerCnx (#19184)
• [fix][broker] AbstractBatchedMetadataStore - use AlreadyClosedException instead of IllegalStateException (#19284)
• [fix][broker] Fix open cursor with null-initialPosition result with earliest position (#18416)
• [fix] [ml] Topics stats shows msgBacklog but there reality no backlog (#19275)
• [improve][broker] Added isActive in ManagedCursorImpl (#19341)
• [improve][broker] Replaced checkBackloggedCursors with checkBackloggedCursor(single subscription check) upon subscription (#19343)
• [fix][broker] fixed the build error for pattern matching variable in lower JVM versions (#19362)
• [fix][broker]optimize the shutdown sequence of broker service when it close (#16756)
• [fix][broker] Pass subscriptionName to auth service (#17123) (#19423)
• [fix][broker]fix multi invocation for ledger createComplete (#18975)
• [fix] [broker] getLastMessageId returns a wrong batch index of last message if enabled read compacted (#18877)
• [fix][broker] Support deleting partitioned topics with the keyword -partition- (#19230)
• [fix] [ml] Fix the incorrect total size if use ML interceptor (#19404)
• [improve][broker] Added isActive in ManagedCursorImpl (#19341)
• [improve][broker] Added isActive in ManagedCursorImpl (#19341)
• [improve][broker] Copy subscription properties during updating the topic partition number. (#19223)
• [fix][broker] Expect msgs after server initiated CloseProducer (#19446)
• [fix][broker] Fix PulsarRegistrationClient and ZkRegistrationClient not aware rack info problem. (#18672)
• [fix][ml] Fix potential NPE cause future never complete. (#19415)
• [fix] [ml] The atomicity of multiple fields of ml is broken (#19346)
• [fix][broker] Fix race condition while updating partition number (#19199)
• [Improve][broker] Support clear old bookie data for BKCluster (#16744)
• [fix][ml] Reset individualDeletedMessagesSerializedSize after acked all messages. (#19428)
• [fix][broker] Make ServerCnx#originalAuthData volatile (#19507)
• [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
• [fix][broker] Fix loadbalance score caculation problem (#19420)
• [fix] [ml] messagesConsumedCounter of NonDurableCursor was initialized incorrectly (#19355)
• [fix][broker][branch-2.10] Replace sync method call in async call chain to prevent ZK event thread deadlock (#19539)
• [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
• [improve][broker] ServerCnx: go to Failed state when auth fails (#19312)
• [feat][broker] Cherry-pick tests from (#19409)
• [improve][broker] Add test to verify authRole cannot change (#19430)
• [feat][broker] OneStageAuth State: move authn out of constructor (#19295)
• [fix][broker] Correct MockAlwaysExpiredAuthenticationState test impl
• [cleanup][broker] Validate originalPrincipal earlier in ServerCnx (#19270)
• [fix][broker] ServerCnx broken after recent cherry-picks (#19521)
• [fix][broker] Fix loadbalance score caculation problem (#19420)
• [fix] [ml] messagesConsumedCounter of NonDurableCursor was initialized incorrectly (#19355)
• [fix][broker][branch-2.10] Replace sync method call in async call chain to prevent ZK event thread deadlock (#19539)
• [improve][broker] Require authRole is proxyRole to set originalPrincipal (#19455)
• [improve][broker] ServerCnx: go to Failed state when auth fails (#19312）
• [feat][broker] Cherry-pick tests from (#19409)
• [improve][broker] Add test to verify authRole cannot change (#19430)
• [fix][broker] Call originalAuthState.authenticate in ServerCnx
• [fix][broker] Correct MockAlwaysExpiredAuthenticationState test impl
• [fix][broker] Make authentication refresh threadsafe (#19506)
• [fix][broker] Allow proxy to pass same role for authRole and originalRole (#19557)
• [fix][broker] Copy command fields and fix potential thread-safety in ServerCnx (#19517)
• [fix][broker][branch-2.10] Fix geo-replication admin (#19608）
• [fix][broker] PulsarRegistrationClient - implement getAllBookies and follow BookieServiceInfo updates (#18133)
• [Improve][Broker]Reduce GetReplicatedSubscriptionStatus local REST call (#16946)
• [fix] [broker] Incorrect service name selection logic (#19505)
• [improve][broker] Use shrink map for trackerCache (#19534)
• [improve][broker] Add UncaughtExceptionHandler for every thread pool (#18211)
• [cleanup][broker] Simplify extract entryMetadata code in filterEntriesForConsumer (#18729)
• [improve][broker] Follow up #19230 to tighten the validation scope (#19234)
• [fix] [ml] topic load fail by ledger lost (#19444)
• [fix] [broker] Topic close failure leaves subscription in a permanent fence state (#19692)
• [cherry-pick][branch-2.10] Fix deadlock causes session notification not to work (#19754) (#19768)
• [fix][broker] Fix potential exception cause the policy service init fail. (#19746)
• [improve][broker] Authorize originalPrincipal when provided (#19830)
• [fix][broker] Only validate superuser access if authz enabled (#19989)
• [fix][broker] Ignore and remove the replicator cursor when the remote cluster is absent (#19972)
• [fix][broker] Fix index generator is not rollback after entries are failed added (#19980)
• [fix] [proxy] Used in proxyConf file when configuration is missing in the command line (#15938)
• [fix][broker] Remove timestamp from broker metrics (#17419)
• [fix] [broker] Counter of pending send messages in Replicator incorrect if schema future not complete (#19242)

Transaction

• [fix][txn] Correct the prompt message (#17009)
• [fix][txn] Catch and log runtime exceptions in async operations (#19258)
• [fix][txn] fix txn coordinator recover handle committing and aborting txn race condition. (#19201)
• [improve][txn] Handle changeToReadyState failure correctly in TC client (#19308)
• [cherry-pick][branch-2.10] Allow superusers to abort transactions (#1…9467) (#19473)
• [feature][txn] Fix individual ack batch message with transaction abor…
• Close TransactionBuffer when MessageDeduplication#checkStatus failed (#19288)
• [improve][txn] Cleanup how superusers abort txns (#19976)
• [feature][txn] Fix individual ack batch message with transaction abort redevlier duplicate messages (#14327)

Pulsar IO and Pulsar Functions

• [fix][io] Update Elasticsearch sink idle cnx timeout to 30s (#19377)
• [fix][fn] Fix k8s merge runtime opts bug (#19481)
• [refactor][fn] Use AuthorizationServer more in Function Worker API (#19975)

CLI

• [fix] [cli] Fix Broker crashed by too much memory usage of pulsar tools (#20031)

Admin

• [fix][admin] Fix validatePersistencePolicies that Namespace/Topic persistent policies cannot set to < 0 (#18999)
• [improve][admin][branch-2.10] Unset namespace policy to improve deleting namespace (#17033) (#19865)
• [fix] [admin] fix incorrect state replication.connected on API partitioned-topic stat (#19942)
• [fix] [admin] Make response code to 400 instead of 500 when delete topic fails due to enabled geo-replication (#19879)

Security

• [fix][sec] Upgrade jettison to 1.5.3 (#19038)
• [fix][sec] Upgrade woodstox to 5.4.0 (#19041)
• [improve][sec] Suppress false positive OWASP reports (#19105)
• [fix][misc] do not require encryption on system topics (#18898)
• [fix][authorization] Fix the return value of canConsumeAsync (#19412)
• [fix][security] Fix secure problem CVE-2017-1000487 (#19479)
• [improve] Upgrade wildfly-eytron (used by debezium) to fix CVE-2022-3143 (#19333)
• [improve][misc] Upgrade Netty to 4.1.87.Final (#19417)
• [fix][broker] Ignore and remove the replicator cursor when the remote cluster is absent (#19972)
• [Authenticate] fix Invalid signature error when use Kerberos Authentication (#15121)
• [fix][sec] Fix transitive critical CVEs in file-system tiered storage (#19957)
• [fix][sec] Fix transitive critical CVEs in file-system tiered storage (#19957)

CI & Test

• [fix][test] ProxyWithAuthorizationTest remove SAN from test certs (#19594)
• [branch-2.10][test]Run and fix tests (#19636)
• [test] Fix ServerCnxTest failing after merge of #19830

Others

• [fix][build] Resolve OWASP Dependency Check false positives (#19120)
• Debezium sources: Support loading config from secrets (#19163)
• [fix][proxy] Only go to connecting state once (#19331)
• [fix][build] Upgrade dependency-check-maven plugin to fix broken OWASP check (#19170)
• [branch-2.10][fix][proxy] Fix using wrong client version in pulsar proxy (#19576)
• [branch-2.10][fix][proxy] Fix using wrong client version in pulsar proxy (#19576)
• [improve] Upgrade lombok to 1.8.26 (#19426)
• [improve] upgrade the bookkeeper version to 4.14.7 (#19179)
• [branch-2.10][broker] Support zookeeper read-only config. (#19156) (#19637)
• [improve] Simplify enabling Broker, WS Proxy hostname verification (#19674)
• [cherry-pick][branch-2.10] KCA: picking fixes from master (#19788)
• [Build] Make the test JVM exit if OOME occurs (#14509)

v2.11.0

Important notice

• Rename Pulsar txn metrics to specify OpenMetrics #16581
• Rename Pulsar schema metrics to specify OpenMetrics #16610
• Rename Pulsar lb metrics to specify OpenMetrics #16611
• Remove jul-to-slf4j #16320 #16320
• Fix get non-persistent topics issue in Namespaces #16170
• Fix producer/consume permission can’t get schema #15956
• Add verification for configured default backlog quota and retention #15441
• Fix ineffective broker LoadBalance #15314
• Ensure the deletion consistency of topic and schema #14608
• Transaction cumulative ack redeliver change #14371
• Bump prometheus client version from 0.5.0 to 0.15.0 #13785
• [PIP-158] Split client TLS transport encryption from authentication #15634
• [PIP-155] Remove Python 2 support #15376
• [PIP-156] Build Pulsar Server on Java 17 #15264
• [PIP-162] Enable system topic by default #15619

PIPs

• [PIP 37] Support chunking with Shared subscription #16202
• [PIP-74] Pulsar client memory limits #14400
• [PIP-81] Split createNewMetadataLedger into multiple methods for reuse #15425
• [PIP 97] Update Authentication Interfaces to Include Async Authentication Methods #12104
• [PIP-100] Support pluggable topic factory #12235
• [PIP-105] Store Subscription properties #15757
• [PIP-117] Change Pulsar standalone defaults #15478
• [PIP-132] Include message header size when checking maxMessageSize for non-batch messages on the client side #14007
• [PIP-136] Sync Pulsar metadata across multiple clouds #16425
• [PIP-143] Support split bundle by specified boundaries #13796
• [PIP-145] Improve performance of regex subscriptions #14505
• [PIP-146] ManagedCursorInfo compression #14542
• [PIP 149] Make the REST Admin API fully async #14365
• [PIP-152] Support subscription level dispatch rate limiter setting #15295
• [PIP-153] Optimize metadataPositions in MLPendingAckStore #15137
• [PIP-160] Make transactions work more efficiently by aggregation operation for transaction log and pending ack store #15370
• [PIP-161] Exclusive Producer: ability to fence out an existing Producer #15488
• [PIP-163] Add lowWaterMark check before appending entry to TB #15424
• [PIP-166] Add MANUAL delivery semantics for Functions #16279
• [PIP-179] Support the admin API to check unknown request parameters #16135
• [PIP-181] Pulsar Shell #16250
• [PIP-184] Topic specific consumer priorityLevel #16715
• [PIP-187] Add API to analyze a subscription backlog and provide an accurate value #16545
• [PIP-189] No batching if only one message in batch #18548
• [PIP-201] Extensions mechanism for Pulsar Admin CLI tools #17158

Broker

• [PIP 81] Split the individual acknowledgments into multiple entries

  • Add range set wrapper to support record dirty ledgers #15607
  • Notifications for faster topic discovery #16062
  • Split createNewMetadataLedger into multiple methods for reuse #15425

• [PIP 105] Store Subscription properties

  • Store Subscription properties #15757
  • Pulsar Admin: create subscription with Properties #15503
  • Fix error on recycled SubscriptionPropertiesList #15335

• [PIP 160] Make transactions work more efficiently by aggregation operation for transaction log and pending ack store

  • Pending ack log store enables the batch feature #16707
  • Metrics stats of Transaction buffered writer #16758
  • Transaction buffered writer supports Timer #16727
  • Transaction log store enables the batch feature #16685
  • Protocol changes and configuration changes for transaction batch log #16617
  • Txn buffered writer for transaction log batch #16428

• [PIP 145] Improve performance of regex subscriptions

  • Enable evaluating subscription pattern on broker side #14804
  • Notifications for faster topic discovery #16062

• [PIP-156] Build Pulsar Server on Java 17

  • Build changes by dropping some required checks #15496
  • Build Pulsar Server on Java 17 #15264

• Add logs for why namespace bundle been split #19003

• Omit making a copy of CommandAck when there are no broker interceptors #18997

• Fix deadlock in PendingAckHandleImpl #18989

• Copy proto command fields into final variables in ServerCnx #18987

• Transaction pending ack store future not completely problem #18943

• Update interceptor handler exception #18940

• Do not require encryption on system topics #18898

• Fix incorrect Nic usage collected by pulsar #18882

• Fix PendingAckHandleImpl when pendingAckStoreProvider.checkInitializedBefore failed #18859

• Ignore the exception of creating namespace #18837

• Fixing can not delete namespace by force #18826

• Avoid OOM not trigger PulsarByteBufAllocator outOfMemoryListener when use ByteBufAllocator.DEFAULT.heapBuffer #18747

• System topic writer/reader connection not counted #18603

• DnsResolverUtil.TTL should be greater than zero #18565

• Persist correct markDeletePosition to prevent message loss #18237

• Fix delete_when_subscriptions_caught_up doesn't work while have active consumers #18320

• Read local cookie when start pulsar standalone #18260

• Remove the redundant judgment logic of ManagedCursorImpl #18205

• Fix unexpected behaviour by invoke PulsarConfigurationLoader#convertFrom #18816

• Fix the wrong behaviour when set overrideBrokerNicSpeedGbps #18818

• Fix the breaking change of standalone metadata initialization #18909

• Transaction buffer recover blocked by readNext #18969

• Fix duplicated schemas creation #18701

• Read local cookie when start pulsar standalone #18260

• The Pulsar standalone bookie is not getting passed the config from standalone.conf #18126

• Reduce unnecessary calling span() when filtering read entries #18106

• Change update schema auth from tenant to produce #18074

• Fixed delayed delivery after read operation error #18098

• Fix memory leak while Offloading ledgers #18500

• Cache invalidation due to concurrent access #18076

• Fix unab...

v2.10.3

Broker

• [improve][broker] Omit making a copy of CommandAck when there are no broker interceptors #18997
• [fix][broker] Copy proto command fields into final variables in ServerCnx #18987
• [improve][broker]Add a cache eviction policy：Evicting cache data by the slowest markDeletedPosition #14985
• [fix][broker] Fix broker cache eviction of entries read by active cursors #17273
• [enh][broker] Add metrics for entry cache insertion, eviction #17248
• [cleanup][broker] Extracted interface for EntryCacheManager #15933
• [improve][broker] Support dynamic update cache config #13679
• [improve][broker] refactor ManagedLedger cacheEvictionTask implement #14488
• [improve][broker] Add logs for why namespace bundle been split (#19003)
• [fix][broker] Fix deadlock in PendingAckHandleImpl (#18989)
• [fix][broker] Branch-2.10 Avoid endless blocking call. (#18914)
• [fix][broker][functions-worker] Ensure prometheus metrics are grouped by type (#8407, #13865) #15558
• [fix][broker]Update interceptor handler exception (#18940)
• [fix][broker] Fix incorrect bundle split count metric (#17970)
• [fix][broker] Avoid OOM not trigger PulsarByteBufAllocator outOfMemoryListener when use ByteBufAllocator.DEFAULT.heapBuffer in PrometheusMetricsGeneratorUtils (#18747)
• [improve][broker] Make Consumer#equals more effective (#18662)
• [fix][broker] In the trimDeletedEntries method, release the removed entry (#18305)
• [improve][broker] Remove locallyAcquiredLock when removeOwnership (#18197)
• [improve][broker] Avoid unnecessary creation of BitSetRecyclable objects #17998
• [fix][broker] add return for PersistentMessageExpiryMonitor#findEntryFailed #17816
• [fix][ml] Persist correct markDeletePosition to prevent message loss (#18237)
• [improve][broker] Support setting forceDeleteTenantAllowed dynamically (#18192)
• Make BookieId work with PulsarRegistrationDriver (second take) #17922
• [fix][broker] Extract additional servlets to the default directory by… (#17477)
• [improve][broker]Improve PersistentMessageExpiryMonitor expire speed when ledger not existed (#17842)
• [fix][broker]Fix mutex never released when trimming (#17911)
• [fix][broker] Fix if dynamicConfig item in ZK do not exist in broker cause NPE. #17705
• [fix][broker] Fix system service namespace create internal event topic. (#17867)
• [bugfix] Prevent Automatic Topic Creation during namespace deletion #17609
• [fix][broker]Cache invalidation due to concurrent access (#18076)
• [fix][broker]add test case for deleting namespace fail when has partitioned system topic (#17338)
• Skip creating a subscription replication snapshot if no messages have been published after the topic gets activated on a broker #16618
• [fix][broker] Fix uncompleted future when get the topic policies of a deleted topic (#18824)
• [fix][broker] Fix delete system topic clean topic policy (#18823)
• [improve][broker][PIP-149]Make getList async #16221
• [fix][flaky-test]ManagedCursorMetricsTest.testManagedCursorMetrics #16878
• [improve][broker] System topic writer/reader connection not counted (#18603)
• [improve][broker] System topic writer/reader connection not counted. (#18369)
• [fix][broker]unify time unit at dropping the backlog on a topic #17957[fix][broker]unify time unit at dropping the backlog on a topic #17957
• [improve][broker] Support setting ForceDeleteNamespaceAllowed dynamically #18181
• [fix][broker] Fix getPositionAfterN infinite loop. (#17971)
• [fix][broker] Update the log print content of createSubscriptions (#18024)
• [Broker] Make PersistentTopicsBase#internalGetPartitionedMetadata async #14153
• [fix][broker] Fix duplicated schemas creation #18701
• [improve][broker] Using handle instead of handleAsync to avoid using common pool thread (#17403)
• [fix][broker] Fix dispatch duplicated messages with Exclusive mode. (#17237)
• [fix][broker] Fix the order of resource close in the InMemoryDelayedDeliveryTracker (#18000)
• [improve][broker] reduce code duplication to avoid endless wait CompletableFuture (#14853)
• [improve][ML] Print log when delete empty ledger. (#17859)
• [improve] clean the empty topicAuthenticationMap in zk when revoke permission (#16815)
• [fix][broker] fix can not revoke permission after update topic partition (#17393)
• [fix][broker] Fix Npe thrown by splitBundle (#17370)
• [fix][broker] Fix executeWithRetry result is null (#17694)
• [fix][broker] Fix SystemTopicBasedTopicPoliciesService NPE issue (#17602)
• [fix][broker] Fix broker cache eviction of entries read by active cursors #17273
• [fix][broker] Fix namespace can not be deleted by force (#18686)
• [fix][broker] Create replicated subscriptions for new partitions when needed (#18659)
• [fix][broker] DnsResolverUtil.TTL should be greater than zero (#18565)
• [fix][broker] Correctly set byte and message out totals per subscription (#18451)
• [fix][broker] fix delete_when_subscriptions_caught_up doesn't work while have active consumers (#18320)
• [improve][broker]consumer backlog eviction policy should not reset read position for consumer (#18037)
• [fix][broker] Fix NPE when ResourceGroupService execute scheduled task. #17840
• [fix][storage] Autorecovery default reppDnsResolverClass to ZkBookieRackAffinityMapping (#15640)
• [broker] Fixed delayed delivery after read operation error (#18098)
• [fix][broker] Fix issue where leader broker information isn't available after 10 minutes (#17401)
• [fix][broker] Make full async call in PulsarAuthorizationProvider (#18050)

Transaction

• [fix][txn] Transaction buffer recover blocked by readNext #18833
• [fix][txn] transaction pending ack store future not completely problem (#18943)
• [fix][txn] Fix PendingAckHandleImpl when pendingAckStoreProvider.checkInitializedBefore failed (#18859)
• [improve][txn] Add getState in transaction for client API (#17423)
• [improve][txn] Implementation of Delayed Transaction Messages (#17548)

Pulsar IO and Pulsar Functions

• [fix][io] Only bundle kafka schema registry client (#18931)
• [fix][fn] Typo in method name (#18844)
• [fix][function] Fix invalid metric type gauge (#18129)
• [fix][fn] fix function failed to start if no typeClassName provided in FunctionDetails (#18111)

Tiered Storage

• [fix][offload] Fix numerical overflow bug while reading data from tiered storage (#18595)
• [fix][offload] Fix memory leak while Offloading ledgers (#18500)

Pulsar SQL

• [fix][sql] Fix message without schema issue. (#18745)

CLI

• [fix][cli] Fix CLI client produce don't able to use multiple -m send multiple messages (#18238)
• [fix][cli] Check numMessages after incrementing counter #17826

Admin

• [improve][admin] Fix NPE in admin-CLI topic stats command (#18326)
• [improve][admin] add topic name and sub name for NotFound error message (#15606)
• PIP-105: new API to get subscription properties #16095
• [fix][admin] returns 4xx error when pulsar-worker-service is disabled and trying to access it (#17901)
• Pulsar Admin: grab contextual stacktrace for sync methods (#14620)
• [fix][admin] Fix NPE when get OffloadThreshold on namespace (#18061)

Security

• [fix][sec] Upgrade scala-library to get rid of CVE-2022-36944 (#18021)
• [Improve][Auth]Update authentication failed metrics report (#17787)
• [fix][sec] Upgrade jackson-databind to 2.13.4.2 to get rid of CVE-2022-42003 (#18394)
• [fix][sec] Upgrade protobuf to 3.19.6 to get rid of CVE-2022-3171 (#18086)
• [fix][sec] File tiered storage: upgrade jettison to get rid of CVE-2022-40149 (#18022)
• [fix][sec] Upgrade JacksonXML to 2.13.4 (#18020)

Dependency & Library updates

• Bump jackson version from 2.13.2 to 2.13.3 #16508
• Remove invalid Netty system property which never was valid for Netty 4.1.x (#13563)
• [improve][misc] Upgrade Netty to 4.1.86.Final and Netty Tcnative to 2.0.54.Final (#18599)
• [improve][build] Remove versions that are handled by netty-bom (#18629)

CI & Test

• [improve][test] force initialize field to avoid polluted by mocks (#17022)
• [fix][flaky-test]ManagedCursorMetricsTest.testManagedCursorMetrics (#16878)
• [improve][test] remove WhiteBox on MockZooKeeper #17579
• [improve][test] try remove whitebox on MockZooKeeper (#17579)
• [improve][test] Add test case for system topic schema not compatible bug. (#17992)
• [fix][flaky-test]BatchMessageWithBatchIndexLevelTest.testBatchMessageAck (#17436)
• [fix][test]The pulsarMessageOverhead in 2.10 is different (#18704)
• [improve][test] Add subscribing regex topic test for delete_when_subscriptions_caught_up. (#18368)

Java Client

• [fix][client] For exclusive subscriptions, if two consumers are created repeatedly, the second consumer will block #18633
• [fix][client] Fixes batch_size not checked in MessageId#fromByteArrayWithTopic (#18405)
• [fix][client] Fix possible npe (#18406)
• [fix][client] Fix exception when calling loadConf on a ConsumerBuilder that has a KeySharedPolicy (#18345)
• [fix][client] Support LocalDateTime Conversion (#18334)
• [fix] [pulsar-client] Fix pendingLookupRequestSemaphore leak when Ser… (#18219)
• [fix][client] Fix failover/exclusive consumer with batch cumulate ack issue. (#18454)
• [improve][java-client]Add init capacity for messages in BatchMessageContainerImpl (#17822)
• [fix] [pulsar-client] Fix pendingLookupRequestSemaphore leak when channel inactive (#17856)
• [fix][client] Unwrap completion exception for Lookup Services (#17717)
• [fix][client] Avoid redelivering duplicated messages when batching is enabled #18486
• [fix][client] Fix multi-topic consumer stuck after redeliver messages (#18491)
• [fix][client] Fix IllegalThreadStateException when using newThread in ExecutorProvider.ExtendedThreadFactory
• [fix][client] Fix NPE of MultiTopicsConsumerImpl due to race condi...

v2.9.4

What's Changed

• [branch-2.9]add message ack rate by @tjiuming in #16146
• [branch-2.9] Fix cherry-pick issues. by @mattisonchao in #16346
• [Branch-2.9][Cherry-pick] fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true by @HQebupt in #16343
• [Branch 2.9] Fix compaction subscription acknowledge Marker msg issue. by @mattisonchao in #16348
• [Branch-2.9] Fix passing incorrect authentication data by @mattisonchao in #16347
• [Branch-2.9][transaction] Cmd-Subscribe and Cmd-Producer will not succeed even after 100 retries. by @mattisonchao in #16349
• [Branch-2.9][Cherry-pick] Fix get non-persistent topics issue in Namespaces. by @Technoboy- in #16517
• [branch-2.9] Fix MockStatic method caused the exception by @mattisonchao in #16628
• [branch-2.9] Fix compatible issue. by @mattisonchao in #16779
• [branch-2.9]Add back FutureUtil#waitForAll and FutureUtil#waitForAny methods with List parameter by @mattisonchao in #16794
• [broker][monitoring][fix] fix pulsar_subscription_msg_ack_rate by @tjiuming in #16866
• [Branch-2.9][fix][client] Fix client memory limit currentUsage leak and semaphore release duplicated in ProducerImpl. by @Nicklee007 in #16971
• [improve][broker] Optimise msgOutCounter and bytesOutCounter (#16214) (#16286) by @nahguam in #16771
• [Branch-2.9][fix][client] Fix MaxQueueSize semaphore release leak in createOpSendMsg. by @Nicklee007 in #16958
• [Branch 2.9][fix][broker] Upgrade log4j2 version to 2.18.0 (#16884) by @liudezhi2098 in #16995
• [improve][authentication][branch-2.9] Support for get token from HTTP params by @nodece in #16987
• [Branch-2.9]Upgrade rocksdb version to 6.16.4 to keep sync with bookkeeper by @hangc0276 in #17246
• [branch-2.9] Support mock final class with PowerMockito by @mattisonchao in #17327
• [branch-2.9][fix][load-balancer] skip mis-configured resource usage(>100%) in loa… by @heesung-sn in #17285
• [fix][broker][branch-2.9] Fix calculate avg message per entry by @coderzc in #17281
• [branch-2.9] Rename test file name from *Test2.java to *Test.java to run all tests correctly by @equanz in #17048
• [fix][c++][branch-2.9]Fix wrong consumers size: execute callback before executing readerCreatedCallback_ by @coderzc in #17629
• [branch-2.9][fix][broker]Persist cursor info error when cursor close by @poorbarcode in #17604
• [branch-2.9][cherry-pick] Fix producer/consume permission can’t get schema. by @Technoboy- in #17730
• [fix][broker][functions-worker] Ensure prometheus metrics are grouped by type (#8407, #13865) by @marksilcox in #17618
• [branch-2.9] Group prometheus metrics. by @tjiuming in #17852
• [branch-2.9] Fix cherry-pick issue by @codelipenghui in #17894
• [fix][broker][branch-2.9] Fix system topic schema not compatible bug by @dragonls in #17986
• [improve][authentication][branch-2.9] Improve get the basic authentication config by @nodece in #16944
• [fix][client][txn] Use PulsarClient HashWheelTimer to schedule producer batch trigger task by @lifepuzzlefun in #18058
• [fix][sec] File tiered storage: upgrade jettison to get rid of CVE-2022-40149 by @mattisonchao in #18105
• [branch-2.9] Fixed key-shared delivery of messages with interleaved delays. by @mattisonchao in #18108
• [branch-2.9] Fix license check issue by @Technoboy- in #18117
• [fix][cpp][branch-2.9] Fix zlib dowload path. by @shibd in #18127
• [fix][sec][branch-2.9] Updating dependencies to get rid of CVEs brought in with kafka and lo… by @alpreu in #18304
• [cherry-pick][branch-2.9] Fix NPE when get OffloadThreshold on namespace. by @congbobo184 in #18638
• [cherry-pick][branch-2.9] cherry-pick #17736 problem and DnsResolverTest license header by @congbobo184 in #18640
• [cherry-pick][branch-2.9] Fix NPE when ResourceGroupService execute scheduled task. by @congbobo184 in #18685
• [cherry-pick][branch-2.9] fix cherry-pick #17592 missing code and flaky test testFailoverSingleAckedPartitionedTopic by @congbobo184 in #18702
• [branch-2.9] Fix no durable cursor leak problem by @gaozhangmin in #18710
• [branch-2.9][fix][broker] Fix duplicated schemas creation (#18701) by @codelipenghui in #18762
• [cherry-pick][branch-2.9] fix cherry-pick #17609 #17957 import by @congbobo184 in #18793
• [cherry-pick][branch-2.9] make getList async #16221 by @congbobo184 in #18811
• [cherry-pick][branch-2.9] Cherry-pick #18307 (Fix can not delete namespace by force) by @liangyepianzhou in #18803
• [cherry-pick][branch-2.9] change proto license version by @congbobo184 in #18821
• [cherry-pick][branch-2.9] Fix delete system topic clean topic policy (#18823) by @congbobo184 in #18831

Full Changelog: v2.9.3...v2.9.4

v2.10.2

Important notice

• [fix][admin] Fix get non-persistent topics issue in Namespaces. #16170

Broker

• [fix][broker] ManagedLedger: move to FENCED state in case of BadVersionException #17736
• [fix][broker] Issue 17588: Allow deletion of a namespace that was left in deleted status #17592
• [fix][broker] Fix BookKeeper packages npe #17291
• [fix][broker] Fix rack awareness cache expiration data race #16825
• [fix][broker] Fix stats-internal with option -m cause active ledger recover then close #16662
• [fix][broker] Fix deadlock in key-shared dispatcher #16660
• [fix][broker] PulsarLedgerManager: add missed return statement #16607
• [fix][broker] Ensure the ack-timeout task gets re-scheduled when there is an exception in the final stage #16337
• [fix][broker] Fix select broker is none #16316
• [fix][broker] Fixed deadlock when checking topic ownership #16310
• [fix][broker] Fix thread safety issues in accessing ManagedCursorContainer.heap ArrayList #16049
• [fix][broker] Avoid IllegalStateException while client_version is not set #16788
• [fix][broker] BadVersionException when splitting bundles, delay 100ms and try again. #16612
• [fix][broker] Expose timestamp field for SchemaData&SchemaInfo #16380
• [fix][broker] Fix NPE when drop backlog for time limit. #16235
• [fix][broker] Fix NPE when get /admin/v2/namespaces/public/default/maxTopicsPerNamespace #16076
• [fix][broker] Fix RawReader out of order #16390
• [fix][broker] Fix calculate avg message per entry #17046
• [fix][broker] Fix compaction subscription acknowledge Marker msg issue. #16205
• [fix][broker] Fix consumer does not abide by the max unacks limitation for Key_Shared subscription #16718
• [fix][broker] Fix consumer does not abide by the max unacks limitation for Shared subscription #16670
• [fix][broker] Fix create client with TLS config #16014
• [fix][broker] Fix namespace backlog quota check with retention. #17706
• [fix][broker] Fix out of order data replication #17154
• [fix][broker] Fix passing incorrect authentication data #16201
• [fix][broker] Fix potential exception causing the reader to be unable to continue reading #17556
• [fix][broker] Fix pulsarLedgerIdGenerator can't delete index path when zk metadata store config rootPath. #17192
• [fix][broker] Fix schema does not replicate successfully #17049
• [fix][broker] Fix the replicator unnecessary get schema request for BYTES schema #17523
• [fix][broker] Fixed error when delayed messages trackers state grows to >1.5GB #16490
• [fix][broker] Increment topic stats outbound message counters and update rate after messages have been written to the TCP/IP connection #17043
• [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
• [fix][broker] Multiple consumer dispatcher stuck when unackedMessages greater than maxUnackedMessages #17483
• [fix][broker] PulsarLedgerManager to pass correct error code to BK client #16857
• [fix][broker] Release the entry in getEarliestMessagePublishTimeOfPos. #16386
• [fix][broker] Retry to delete the namespace if new topics are created during the namespace deletion #16676
• [fix][broker] Retry when DistributedIdGenerator has BadVersion error #16491
• [fix][broker] Skip reading more entries for a pending read with no more entries #16400
• [fix][broker] Support loadBalancerSheddingIntervalMinutes dynamic configuration #16408
• [fix][broker] The configuration loadBalancerNamespaceMaximumBundles is invalid #16552
• [fix][broker] Topic policy reader can't recover when getting any exception. #17562
• [fix][broker] Unregister topic policy listener if managed ledger close failed #17652
• [fix][broker] Upgrade log4j2 version to 2.18.0 #16884
• [fix][broker] fix No such ledger exception #16420
• [fix][broker] fix broker unackmessages become a negative number #17003
• [fix][broker] remove exception log when access status.html #17025
• [fix][broker] Consumer can't consume messages because there are two same topics in one broker #17526
• [fix][broker] Fix getInternalStats occasional lack of LeaderInfo again #16238
• [fix][broker] Fix subscribe dispatcher limiter not be initialized #16175
• [fix][broker] Fix topic policies update not check message expiry #15941
• [fix][broker] Fix topic-level replicator rate limiter not init #15825
• [fix][broker] ManagedLedger metrics fail cause of zero period #17257
• [fix][broker] Prevent StackOverFlowException in SHARED subscription #16968
• [fix][broker] Fix message ack rate #16108
• [fix][broker] Fix bug: fail to expose managed ledger client stats to prometheus if bookkeeperClientExposeStatsToPrometheus is true #16219
• [fix][broker] Fix npe when invoking replaceBookie. #16239
• [fix][broker] Fail to update partition meta of topic due to ConflictException: subscription already exists for topic #17488
• [fix][broker] Fix NPE when ledger id not found in OpReadEntry #15837
• [Fix][broker] Fix topic dispatch rate limiter not init on broker-level #16084
• [opt][broker] Optimize topic policy with HierarchyTopicPolicies about replicatorDispatchRate #14161
• [fix][broker] Fix the broker close hanged issue. #17689
• [fix][broker] Fix topic dispatch rate limiter not init on broker-level #16084 #17000
• [fix][broker] Prevent StackOverFlowException in SHARED subscription #17053
• [fix][broker] skip mis-configured resource usage(>100%) in load balancer #16937
• [fix][broker] Fix wrong unit of NIC speed on Linux #15304
• [fix][broker] Fix RawReader hasMessageAvailable returns true when no messages #16443
• [improve][broker] Expose topic level storage write and read rate metrics #16855
• [improve][broker] Broker/EntryFilter: make the delay for RESCHEDULED messages configurable (dispatcherFilterRescheduledMessageDelay) #16602
• [improve][broker] Use OrderedExecutor instead of OrderedScheduler for consumer dispatch #16115
• [improve][broker] Avoid going through all the consumers to get the message ack owner #16245
• [improve][broker] Avoid reconnection when a partitioned topic was created concurrently #16043
• [improve][broker] Cancel the loadShedding task when closing pulsar service #17632
• [improve][broker] Consolidate Netty channel flushes to mitigate syscall overhead #16361
• [improve][broker] Improve cursor.getNumberOfEntries if isUnackedRangesOpenCacheSetEnabled=true #17465
• [improve][broker] Make MessageRedeliveryController work more efficiently [#17804](https...

v2.8.4

Broker

• [improve][broker] Do not print the log that the topic does not exist #13535
• [fix][broker] Fix NPE when nextValidLedger is null #13975
• [fix][broker] Fix NPE when ReplicatedSubscriptionsController sends marker message with deduplication enabled #14017
• [fix][broker] Fix avg-messagePerEntry metrics for consumer #14330
• [improve][broker] Validate rack name when setting the bookie rack info #14336
• [improve][broker] Optimize find nics process. #14340
• [fix][broker] Fix NPE when subscription is already removed #14363
• [fix][broker] Fix Broker HealthCheck Endpoint Exposes Race Conditions #14367
• [improve][broker] Optimize RawReader#create when using Compactor #14447
• [fix][broker] Fix the latest message of the compacted topic cannot be read #14449
• [fix][broker] Fix producerFuture not completed in ServerCnx#handleProducer #14467
• [fix][broker] Fix NPW in ManagedLedgerImpl #14481
• [improve][broker] Support shrink in ConcurrentLongHashMap #14497
• [fix][broker] Fixed wrong behaviour caused by not cleaning up topic policy service state. #14503
• [improve][broker] Optimize memory usage: support to shrink for pendingAcks map #14515
• [fix][broker] Cancel offload tasks when managed ledger closed #14744
• [improve][broker] Reduce unnecessary expansions for ConcurrentLong map and set #14562
• [fix][broker] Fix metadata store deadlock when checking BacklogQuota #14634
• [improve][broker] Support shrink for map or set #14663
• [fix][broker] Fix lost message issue due to ledger rollover #14664
• [fix][broker] Set splitNamespaceBundle with readonly=false #14680
• [fix][broker] Fixed duplicated delayed messages when all consumers disconnect #14740
• [fix][broker] Fix wrong state for non-durable cursor #14869
• [fix][broker] Improve error logging for topic not found #14892
• [fix][broker] Fix topic policy reader close bug #14897
• [fix][broker] Return if reset in progress #14978
• [fix][broker] Fix potential NPE in Replicator #15003
• [fix][broker] Fix race condition in updating lastMarkDeleteEntry field #15031
• [fix][broker] Fix potential to add duplicated consumer #16826
• [fix][broker] Follow up on race condition fixes in ManagedCursorImpl #15031 #15067
• [cleanup] [broker] Remove useless code to avoid confusion in OpReadEntry#checkReadCompletion #15104
• [fix][broker] Cancel fencedTopicMonitoringTask when topic closed normally. #15202
• [fix][broker] Fix MessageDeduplication#inactiveProducers may not be persistence correctly #15206
• [fix][broker] Fix race condition between timeout and completion in OpAddEntry #15233
• [fix][broker] Fix problem at RateLimiter#tryAcquire #15306
• [improve][broker] Use shrink map for message redelivery. #15342
• [improve][broker] Support shrink for ConcurrentSortedLongPairSet #15354
• [improve][broker] Close connection if a ping or pong message cannot be sent #15382
• [cleanup][broker] Fix typo in enum name and handle closing of the channel properly since writeAndFlush is asynchronous #15384
• [fix][broker] Avoid heartbeat topic to offload. #15393
• [fix][broker] Fix deadlock in broker after race condition in topic creation failure #15570
• [fix][broker] fix calculation in getNumberOfEntriesInStorage #15627
• [fix][broker] Fix NPE when set AutoTopicCreationOverride #15653
• [fix][broker] Fix creating producer failure when set backlog quota #15663
• [fix][broker] Fix NPE when put value to RangeCache #15707
• [fix][broker] Fix can not enable system topic if AutoUpdateSchemaEnabled=false #15759
• [fix][broker] Fix wrong unit of NIC speed on Linux #15770
• [fix][broker] Fix NPE in MessageDeduplication #15820
• [fix][broker] Fix NPE when ledger id not found in OpReadEntry #15837
• [fix][broker] When skipping updating mark delete position, execute callback with executor to prevent deadlock #15971
• [fix][broker] Fix NPE when drop backlog for time limit. #16235
• [fix][broker]Fix getInternalStats occasional lack of LeaderInfo again #16238
• [fix][broker] Fix RawReader out of order #16390
• [fix][broker] Skip reading more entries for a pending read with no more entries #16400
• [fix][broker] Fix No such ledger exception #16420
• [fix][broker] Fix RawReader hasMessageAvailable returns true when no messages #16443
• [fix][broker] Fix stats-internal with option -m cause active ledger recover then close #16662
• [fix][broker] Fix passing incorrect authentication data #16840
• [fix][broker] ManagedCursor: mark delete no callback when create meta-ledger fail #16841
• [improve][broker] Avoid reconnection when a partitioned topic was created concurrently #16867
• [improve][broker] Recycle OpReadEntry in some corner cases #16869
• [fix][broker] Fix compaction subscription acknowledge Marker msg issue. #16918

Security

• [fix][sec] Use provider's canLookupAsync method for AuthorizationService #11777
• [improve][sec] Optimize the logic of allowing namespace operation #13090
• [improve][sec] Full-support set SSL provider, ciphers and protocols #13740
• [fix][sec] Role with namespace produce authz can also get topics #13773
• [improve][sec] Full-support SSL provider, ciphers, and protocols for broker service and proxy service #15034
• [improve][sec] Add load multiple certificates support #14798
• [fix][sec] Fix handling single role and non-jwt-token in MultiRolesTokenAuthorizationProvider #14857
• [fix][sec] Use tlsCertRefreshCheckDurationSec instead of 0 for refresh value #15075
• [improve][sec] Skip unnecessary DNS resolution when creating AuthenticationDataHttp instance #15221
• [improve][sec] Improve skipping of DNS resolution when creating AuthenticationDataHttp instance #15228
• [fix][sec] Fix MultiRolesTokenAuthorizationProvider authorize issue #15454
• [fix][sec] Fix grant all permissions but can't list topic #15501
• [improve][sec] Switch to rely on Netty for Hostname Verification #15824
• [fix][sec] Fix create client with TLS config #16014
• [fix][sec] Avoid AuthenticationDataSource mutation for subscription name #16065
• [improve][sec] Improve get the basic authentication config #16526
• [fix][sec] Fix multiple roles authorization #16645
• [fix][sec] Add timeout of sync methods and avoid call sync method for AuthoriationServi...