Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
1.1.0
-
None
-
None
Description
It seems that in secure clusters, HiveServer2 creates an unbounded amount of crc files related to the Delegation Token, e.g.:
# ls -latr .hive_hadoop_delegation_token*.tmp.crc | more
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token7336761305415229297.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token2098027444797756507.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token570615377517838929.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token2806157469169711507.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token4743780849236152782.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token4779276962989484605.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token614278562135964419.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token5163699580858054526.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:41 .hive_hadoop_delegation_token6764054090932425303.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:42 .hive_hadoop_delegation_token6389909203128084522.tmp.crc
rw-rr- 1 hive hive 12 Jul 22 18:42 .hive_hadoop_delegation_token7810958545651446366.tmp.crc
[...]
this could quickly fill the inode table of the filesystem (usually the one mounted on /tmp).
I've experienced it in Hive 1.1.0 (CDH 5.16.2), but I think the same issue is present in the current master branch, since here:
the crc file gets created, but only the token file gets deleted:
HIVE-13883 greatly relates to the issue; I think that the same logic could be used here to fix this, although I wonder if the Credentials class should also expose some kind of delete method that cleans every file it has produced...