Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22452

CTAS query failure at DDL task stage doesn't clean out the target directory

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0, 3.1.2
    • Fix Version/s: None
    • Component/s: Hive
    • Labels:
      None

      Description

      CTAS query failure at DDL task stage due to HMS connection issue leaves the output file in target directory. Since DDL task stage happens after Tez DAG completion and MOVE Task , output file getsĀ  already moved to target directory and does not get cleaned up after the query failure.

      Re-executing the same query causes a duplicate file under table location hence duplicate data.

        Attachments

          Activity

            People

            • Assignee:
              kuczoram Marta Kuczora
              Reporter:
              rtrivedi12 Riju Trivedi
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: