Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20349

Implement Retry Logic in HiveDruidSplit for Scan Queries

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • Druid integration
    • None

    Description

      while distributing druid scan query we check where the segments are loaded and then each HiveDruidSplit directly queries the historical node.
      There are few cases when we need to retry and refetch the segments.

      1. The segment is loaded on multiple historical nodes and one of them went down. in this case when we do not get response from one segment, we query the next replica.
      2. The segment was loaded onto a realtime task and was handed over, when we query the realtime task has already finished. In this case there is no replica. The Split needs to query the broker again for the location of the segment and then send the query to correct historical node.

      This is also the root cause of failure of druidkafkamini_basic.q test, where the segment handover happens before the scan query is executed.

      Note: This is not a problem when we are directly querying Druid brokers as the broker handles the retry logic.

      Attachments

        1. HIVE-20349.5.patch
          15 kB
          Nishant Bangarwa
        2. HIVE-20349.4.patch
          15 kB
          Nishant Bangarwa
        3. HIVE-20349.3.patch
          14 kB
          Nishant Bangarwa
        4. HIVE-20349.2.patch
          15 kB
          Nishant Bangarwa
        5. HIVE-20349.1.patch
          14 kB
          Nishant Bangarwa
        6. HIVE-20349.patch
          12 kB
          Nishant Bangarwa

        Issue Links

          Activity

            People

              nishantbangarwa Nishant Bangarwa
              nishantbangarwa Nishant Bangarwa
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: