Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-511

Storm-Kafka spout keeps sending fetch requests with invalid offset

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.9.3
    • 0.9.3, 0.9.3-rc2
    • storm-kafka

    Description

      With default behaviour (KafkaConfig.useStartOffsetTimeIfOffsetOutOfRange == true) when Kafka returns the error about offset being out of range, storm.kafka.KafkaUtils.fetchMessages tries to fix offset in local scope and retry fetch request. But if there are no more messages appeared under that specified partition it will never update the PartitionManager, but keep sending tons of requests with invalid offset to Kafka broker. On both sides Storm and Kafka logs grow extremely quick during that time.

      Attachments

        Issue Links

          Activity

            People

              viktortnk Viktor Taranenko
              viktortnk Viktor Taranenko
              Votes:
              4 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: