You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
Consumers in a Consumer Groups can subscribe to different topics. When a group rebalances, the elected leader uses Assigner#assign to determine what topic-partitions to assign to which group members.
However, the current implementation only looks at the leader's subscribed topics when it does this assignment, rather than the subscribed topics of each of the members.
Here's an example:
Give 2 topics, Topic A with 2 partitions, Topic B with 2 partitions
Consumer Group ID (Leader?)
Subscribed Topics
Client ID (used to generate Group Member ID)
Assigned Topic Partitions
group-1 (Leader)
Topic A
group-1-1
Topic A Partition 1
group-1
Topic B
group-1-2
No assignments
group-1
Topic A
group-1-3
No Assignments
group-1
Topic B
group-1-0
Topic A Partition 0
The client IDs are just to show that there's a alphanumeric sort applied to member IDs to keep partition assignment relatively stable across rebalances.
To Reproduce
Create two topics, each with 1 partition, called topic-a, and topic-b
Run two consumers, with the same group ID. The first consumer should subscribe to topic-a and the client ID of "client-id-1" and the second should subscribe to topic-b with the client ID "client-id-0". The first consumer should become the leader when it starts up.
Use the KafkaAdmin describeGroup to list the group members and assignments. You should see the second consumer assigned topic-a, even though its subscribed to topic-b.
Expected behavior
The round robin assigner should assign topic-partitions only to members that are subscribed to those topics.
Observed behavior
The round robin assigner assigns topics to consumers that they aren't subscribed to.
Environment:
OS: Mac OS 14.4.1 (in Docker)
KafkaJS version: 2.2.4
Kafka version: 7.3.1-ccs (Confluent)
NodeJS version: 20.10.0
Additional context
The Java RoundRobinAssignor more intelligently maps partition assignments via subscribed topics as described in the JavaDocs
The text was updated successfully, but these errors were encountered:
Describe the bug
Consumers in a Consumer Groups can subscribe to different topics. When a group rebalances, the elected leader uses Assigner#assign to determine what topic-partitions to assign to which group members.
However, the current implementation only looks at the leader's subscribed topics when it does this assignment, rather than the subscribed topics of each of the members.
Here's an example:
Give 2 topics, Topic A with 2 partitions, Topic B with 2 partitions
The client IDs are just to show that there's a alphanumeric sort applied to member IDs to keep partition assignment relatively stable across rebalances.
To Reproduce
Expected behavior
The round robin assigner should assign topic-partitions only to members that are subscribed to those topics.
Observed behavior
The round robin assigner assigns topics to consumers that they aren't subscribed to.
Environment:
Additional context
The Java RoundRobinAssignor more intelligently maps partition assignments via subscribed topics as described in the JavaDocs
The text was updated successfully, but these errors were encountered: