Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-26795

Iceberg integration: clean up temporary files in case of statement cancel

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Iceberg integration
    • None

    Description

      Iceberg write operations are performed in the Tez task but the Iceberg commit of these writes are happening in the move task. To inform the MoveTask what writes has to be committed temp files are created with the path of the actual datafiles.

      Also in case of ctas statements the table is created by the ddl task is serialized into a temp file to be available for the Tez task which does the writes into the newly created table.

      Normally the cleanup of these temp files are happening in the move task but this task is not executed in case of cancel or an error in tez task.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              kkasa Krisztian Kasa
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: