+* Kafka consumer: introduce a per-partition size-limited queue for backpressure and a configurable `fetch_max_wait` [#997](https://github.com/centrifugal/centrifugo/pull/997). Centrifugo now polls Kafka with `500ms` by default and avoids using the `SetOffsets` API of the `franz-go` library. Instead of a buffered channel, we now use a queue (limited by size, configured over `partition_queue_max_size`). The `partition_buffer_size` option was removed since the internal consumer implementation changed, but we do not expect this to be a problem for users based on the added benchmark and improved throughput in the new implementation. However, it may result in a warning in logs about using the non-existing `partition_buffer_size` option, which may be an inconvenience. See [updated docs](https://centrifugal.dev/docs/server/consumers#kafka-consumer).
0 commit comments