Sink-managed consumer group expires after 7 days of no activity in the topic

[fenil25]

Apache Iceberg version

1.7.1 (latest release)

Query engine

Kafka Connect

Please describe the bug 🐞

For the Kafka Connect Iceberg connector,
sink-managed consumer group (cg-control-<connector_name>) expires after 7 days of no activity (assuming broker config of offsets.retention.minutes is set to default value of 10080)

When this happens, whats the behavior with the kafka connector? I wasn't able to find the code where the consumer group gets recreated. Do we have to restart the connector for the connector to consume records correctly?

Is there any way to increase the retention by keeping the consumer group alive without changing the broker config?

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time