Skip to main content

View and reset consumer group offsets

The open source Apache Kafka® code includes a kafka-consumer-groups.sh utility enabling you to view and manipulate the state of consumer groups.

note

Before using the kafka-consumer-groups.sh, configure a consumer.properties file pointing to a Java keystore and truststore which contain the required certificates for authentication. See how to do it in the dedicated page.

Managing consumer group offsets with kafka-consumer-groups.sh​

The kafka-consumer-groups.sh tool enables to manage consumer group offsets, the following commands are available.

List active consumer groups​

To list the currently active consumer groups use the following command replacing the demo-kafka.my-project.aivencloud.com:17072 with your service URI:

kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.my-project.aivencloud.com:17072 \
    --command-config consumer.properties \
    --list

Retrieve the details of a consumer group​

To retrieve the details of a consumer group use the following command replacing the demo-kafka.my-project.aivencloud.com:17072 with the Aiven for Apache Kafka service URI and the my-group with the required consumer group name:

kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.my-project.aivencloud.com:17072 \
    --command-config consumer.properties \
    --group my-group \
    --describe

The details of the consumer group my-group are printed out in the following output:

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID                HOST            CLIENT-ID
my-group        test-topic      0          5509            5515            6               rdkafka-39404560-f8f2-4b0b /151.62.82.140  rdkafka

List the current members of a consumer group​

To retrieve the current members of a consumer group use the following command replacing the demo-kafka.my-project.aivencloud.com:17072 with the Aiven for Apache Kafka service URI and the my-group with the required consumer group name:

kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.my-project.aivencloud.com:17072 \
    --command-config consumer.properties \
    --group my-group \
    --describe \
    --members

The members of the my-group consumer group are printed out in the following output:

GROUP           CONSUMER-ID                                  HOST            CLIENT-ID       #PARTITIONS
my-group        rdkafka-a4c0a09c-8c6e-457e-bf9e-354a8e2f4bb8 /151.62.82.140  rdkafka         0
my-group        rdkafka-39404560-f8f2-4b0b-9518-811e2eb20074 /151.62.82.140  rdkafka         1

Reset the offset of a consumer group​

You might want to reset the consumer group offset when the topic parsing needs to start at a specific (non default) offset. To reset the offset use the following command replacing:

  • demo-kafka.my-project.aivencloud.com:17072 with the Aiven for Apache Kafka service URI
  • my-group with the required consumer group name
  • test-topic with the required topic name
warning

The consumer group must be inactive when you make offset changes.

kafka-consumer-groups.sh \
    --bootstrap-server demo-kafka.my-project.aivencloud.com:17072 \
    --command-config consumer.properties \
    --group test-group \
    --topic test-topic \
    --reset-offsets \
    --to-earliest \
    --execute

The --reset-offsets command has the following additional options:

  • --to-earliest : resets the offset to the beginning of the topic.
  • --to-lastest : resets the offset to the end of the topic.
  • --to-offset : resets the offset to a known, fixed offset number.
  • --shift-by : performs a relative shift from the current offset position using the given integer.
tip

Use positive values to skip forward or negative values to move backwards.

  • --to-datetime <YYYY-MM-DDTHH:mm:SS.sss> : resets to the given timestamp.
  • --topic <topicname>:<partition>: Applies the change to a specific partition, for example --topic test-topic:0. By default, the --topic argument applies to all partitions.