Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-23058

Compaction task reattempt fails with FileAlreadyExistsException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 4.0.0-alpha-1
    • Hive
    • None

    Description

      Issue occurs when compaction attempt is relaunched after first task attempt failure due to preemption by Scheduler or any other reason.

      Since _tmp directory was created by first attempt and was left uncleaned after task attempt failure. Second attempt of the the task fails with "FileAlreadyExistsException" exception.

      Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.fs.FileAlreadyExistsException): /warehouse/tablespace/managed/hive/default.db/compaction_test/_tmp_3670bbef-ba7a-4c10-918d-9a2ee17cbd22/base_0000186/bucket_00005 for client 10.xx.xx.xxx already exists

      Attachments

        1. HIVE_23058.1.patch
          4 kB
          Riju Trivedi
        2. HIVE_23058.patch
          1 kB
          Riju Trivedi
        3. HIVE-23058.2.patch
          4 kB
          Riju Trivedi
        4. HIVE-23058.3.patch
          4 kB
          Riju Trivedi

        Issue Links

          Activity

            People

              rtrivedi12 Riju Trivedi
              rtrivedi12 Riju Trivedi
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: