Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
FanoutWriter is a writer in Iceberg which can be used to write records in a table. This writer keeps all the file handles open, until the write is finished. FanoutWriters is used as the writer when the incoming records are unsorted. We can by default have some mechanism to switch to using FanoutWriters instead of ClusteredWriters when custom sort expressions are not present for the given table/query.
Similar stuff is already implemented in Spark - https://github.com/apache/iceberg/pull/8621
Attachments
Issue Links
- links to