Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-1075

Consumer will not rebalance upon topic partition change

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.8.0
    • 0.8.0
    • None
    • None

    Description

      Due to the watcher and zk data structure mismatch, consumer will not rebalance upon topic partition change. Details:

      ZK data structure for topic partitions:

      /brokers/topics/[topic-name]/partitions/[partition-id]/state

      When new partitions are added, it will change the data in /brokers/topics/[topic-name], however the ZK client is watching on child change of /brokers/topics/[topic-name], which will always be a single child 'partitions'.

      Proposal: add a data change listener, which will trigger rebalance upon data changes on /brokers/topics/[topic-name].

      Attachments

        1. KAFKA-1075.patch
          4 kB
          Guozhang Wang
        2. KAFKA-1075_2013-10-07_10:25:01.patch
          20 kB
          Guozhang Wang
        3. KAFKA-1075_2013-10-07_10:58:40.patch
          4 kB
          Guozhang Wang

        Activity

          People

            guozhang Guozhang Wang
            guozhang Guozhang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: