Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
Description
ARROW-6040 fixed the problem that dictionary entries are required in IPC streams even when empty, which only writes dictionaries when there are at least one batch. In this way, if we write empty stream and invoke ArrowWriter#close, the dictionaries are not closed leading to memory leak (they are closed after the write operation), and it’s really hard to debug, this problem was found by TestArrowReaderWriter#testEmptyStreamInStreamingIPC when I tried to close allocator after the test.
Besides, there are several test classes have potential memory leak without closing allocator/vector/buf etc.