Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
3.0.0
-
None
Description
SPARK-20568 added the possibility to clean up completed files in streaming query. Deleting/archiving uses the main thread which can slow down processing. It would be good to do this on separate thread(s).
Attachments
Issue Links
- links to