Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8788

Optimize client metadata handling with a large number of partitions

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.3.0
    • 2.4.0, 2.3.1
    • None
    • None

    Description

      Credit to lbradstreet for profiling the producer with a large number of partitions.

      There was a regression in 2.3.0 that made the client metadata handling significantly less efficient when dealing with a large number of partitions. There was a less significant regression in 2.2.0.

      Lucas benchmarked the producer, but the code is shared with the consumer. See the PR for more details.

       

      Attachments

        Issue Links

          Activity

            People

              ijuma Ismael Juma
              ijuma Ismael Juma
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: