Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Iceberg write operations are performed in the Tez task but the Iceberg commit of these writes are happening in the move task. To inform the MoveTask what writes has to be committed temp files are created with the path of the actual datafiles.
Also in case of ctas statements the table is created by the ddl task is serialized into a temp file to be available for the Tez task which does the writes into the newly created table.
Normally the cleanup of these temp files are happening in the move task but this task is not executed in case of cancel or an error in tez task.
Attachments
Issue Links
- relates to
-
HIVE-26771 Use DDLTask to created Iceberg table when running ctas statement
- Resolved