Details
-
Improvement
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
1.20.2
-
None
Description
Currently a malicious or merely unwitting user can crash their Drill foreman by sending
select * from dfs.huge_workspace limit 10
causing the query planner to recurse over every file in huge_workspace and culminating in
2022-08-09 15:13:22,251 [1d0da29f-e50c-fd51-43d9-8a5086d52c4e:foreman] ERROR o.a.drill.common.CatastrophicFailure - Catastrophic Failure Occurred, exiting. Information message: Unable to handle out of memory condition in Foreman.java.lang.OutOfMemoryError: null
if there are enough files in huge_workspace. A SHOW FILES command can produce the same effect. This issue proposes a new BOOT option named drill.exec.storage.file.recursive_listing_max_size with a default value of, say 10 000. If a file listing task exceeds this limit then the initiating operation is terminated with a UserException preventing runaway resource usage.