Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16050

consumer was removed from group,but still can poll data from kafka, data duplicate

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.3.1
    • None
    • core
    • None

    Description

      I have 3 brokers: b1,b2,b3

      a topic :  test, partitiion5,replication3

      3 consumer in 1 group:  consumer1,consumer2,consumer3

      groupid: xx

       

      consumer1 running in b1

      consumer2 running in b2

      consumer3 running in b3

      ./kafka-console-consumer.sh -bootstrap-server localhost:9093 --group xx  --topic test   -from-beginning

       

      b2's  clock changed, consumer2 was removed from group xx(reason: removing member consumer-2 on heartbeat expiration) (kafka.coordinator.group.GroupCoordinator)

      kafka-consumer-groups.sh can't see any record about cosumer2

      ./kafka-consumer-groups.sh  --bootstrap-server localhost:9093 --all-topics --describe --all-groups

       

      Then consumer rebalanced,  partiitons assigned to  consumer2 was assigned to other consumer 

      Although consumer2 was removed from group xx ,BUT still poll data from kafka ,kafka can't find it

      After rebalance  another consumer poll the same partition with consumer2 

      This make data was poll duplicate

       

       

       

       

       

       

       

       

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            auroraxlh Xin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: