XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • None
    • core
    • None

    Description

      We need a way to propagate the leader and the partition information to the producer so that it can do load balancing and semantic partitioning. One way to do that is to have the producer get the information from ZK directly. This means that the producer needs to maintain a ZK session and has to subscribe to watchers, which can be complicated. An alternative approach is to have the following api on the broker.

      TopicMetaData getTopicMetaData(String: topic)

      TopicMetaData {
      Array[PartitionMetaData]: partitionsMetaData
      }

      PartitionMetaData {
      Int: partitionId
      String: leaderHostname
      Int: leaderPort
      }

      Using this api, the producer can get the metadata about a topic during initial startup or leadership change of a partition.

      Attachments

        1. kafka-238-v1.patch
          98 kB
          Neha Narkhede
        2. kafka-238-v2.patch
          98 kB
          Neha Narkhede
        3. kafka-238-v3.patch
          123 kB
          Neha Narkhede

        Activity

          People

            nehanarkhede Neha Narkhede
            junrao Jun Rao
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: