Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
Impala 2.12.0
-
None
-
ghx-label-2
Description
In docs/topics/impala_known_issues.xml, we have a section "ki_file_handle_cache" that covers the interaction of the file handle cache with short circuit reads. This is related to the HDFS issue HDFS-12528.
HDFS-12528 is resolved. The way it was resolved is to introduce an HDFS parameter dfs.domain.socket.disable.interval.seconds that specifies the amount of time that short circuit reads are disabled. The default value for upstream Hadoop is 10 minutes (600 seconds). It can be customized, and the default value for CDH is 1 second.
I think this section should recommend running with dfs.domain.socket.disable.interval.seconds set to a small value (CDH uses 1 second) when using file handle cache. However, 0 is not recommended.