Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-14048 The Next Generation of the Consumer Rebalance Protocol
  3. KAFKA-16227

Console consumer fails with `IllegalStateException`

    XMLWordPrintableJSON

Details

    Description

      I have seen a few occurrences like the following one. There is a race between the background thread and the foreground thread. I imagine the following steps:

      • quickstart-events-2 is assigned by the background thread;
      • the foreground thread starts the initialization of the partition (e.g. reset offset);
      • quickstart-events-2 is removed by the background thread;
      • the initialization completes and quickstart-events-2 does not exist anymore.

       

      [2024-02-06 16:21:57,375] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
java.lang.IllegalStateException: No current assignment for partition quickstart-events-2
	at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:367)
	at org.apache.kafka.clients.consumer.internals.SubscriptionState.updateHighWatermark(SubscriptionState.java:579)
	at org.apache.kafka.clients.consumer.internals.FetchCollector.handleInitializeSuccess(FetchCollector.java:283)
	at org.apache.kafka.clients.consumer.internals.FetchCollector.initialize(FetchCollector.java:226)
	at org.apache.kafka.clients.consumer.internals.FetchCollector.collectFetch(FetchCollector.java:110)
	at org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.collectFetch(AsyncKafkaConsumer.java:1540)
	at org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.pollForFetches(AsyncKafkaConsumer.java:1525)
	at org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.poll(AsyncKafkaConsumer.java:711)
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:874)
	at kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:473)
	at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:103)
	at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:77)
	at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:54)
	at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala)

       

       

      Attachments

        Issue Links

          is related to

          Task - A task that needs to be done. KAFKA-16290 Investigate propagating subscription state updates via queues

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #15491

          Activity

            People

              cadonna Bruno Cadonna
              dajac David Jacot
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: