Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.5.0
Description
Memory consumed by HdfsTableSink is tracked by not captured in the query profile, makes it look as if there is untracked memory.
from the Memz tab
Process: Limit=201.73 GB Total=4.78 GB Peak=8.66 GB
Free Disk IO Buffers: Total=1.40 GB Peak=1.84 GB
RequestPool=root.mmokhtar: Total=2.90 GB Peak=7.80 GB
Query(aa40c1753a40a8be:9c27185a00000000): Total=2.90 GB Peak=3.20 GB
Fragment aa40c1753a40a8be:9c27185a00000006: Total=2.90 GB Peak=3.20 GB
HDFS_SCAN_NODE (id=0): Total=1.51 GB Peak=1.97 GB
HdfsTableSink: Total=1.38 GB Peak=1.38 GB
HdfsTableSink Exprs: Total=124.02 MB Peak=124.02 MB
Block Manager: Limit=161.38 GB Total=0 Peak=0
RequestPool=root.jenkins: Total=0 Peak=284.69 MB
From query profile
HdfsTableSink:(Total: 9m47s, non-child: 9m47s, % non-child: 100.00%)
- BytesWritten: 6.79 GB (7290559999)
- CompressTimer: 58s128ms
- EncodeTimer: 8m6s
- FilesCreated: 69 (69)
- FinalizePartitionFileTimer: 1m18s
- HdfsWriteTimer: 1m16s
- PartitionsCreated: 1 (1)
- RowsInserted: 426.90K (426901)
- TmpFileCreateTimer: 1s253ms