Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16041

HCatalog doesn't delete temp _SCRATCH dir when job failed

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • 2.2.0
    • None
    • HCatalog
    • None

    Description

      when we use HCatOutputFormat to write to an external partitioned table, a tmp dir (which starts with "_SCRATCH" ) will appear under table path if the job failed.

      drwxr-xr-x - yun hdfs 0 2017-02-27 01:45 /tmp/hive/_SCRATCH0.31946356159329714
      drwxr-xr-x - yun hdfs 0 2017-02-27 01:51 /tmp/hive/_SCRATCH0.31946356159329714/c1=1
      drwxr-xr-x - yun hdfs 0 2017-02-27 00:57 /tmp/hive/c1=1
      drwxr-xr-x - yun hdfs 0 2017-02-27 01:28 /tmp/hive/c1=1/c2=2
      rw-rr- 3 yun hdfs 12 2017-02-27 00:57 /tmp/hive/c1=1/c2=2/part-r-00000
      rw-rr- 3 yun hdfs 12 2017-02-27 01:28 /tmp/hive/c1=1/c2=2/part-r-00000_a_1

      Attachments

        1. HIVE-16041.1.patch
          2 kB
          yunfei liu
        2. HIVE-16041.2.patch
          2 kB
          yunfei liu
        3. HIVE-16041.3.patch
          2 kB
          yunfei liu

        Issue Links

          Activity

            People

              yunfeiliu yunfei liu
              yunfeiliu yunfei liu
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: