Details
-
Bug
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
None
-
None
-
None
Description
When reading from a opening WAL file in replication, we will use FileSystem.getFileStatus to get the file length and then use it to calculate the trailer's position. But if a file is being written, you can not get the correct length from namenode, you could even get a 0 if the file has only one block(for most WAL files).
This will not lead to real data loss as we do not rely on the file length to determine whether we should read the file, but since it will mess up the trailer's position, it may lead to incorrect metrics, like un cleanly closed wal files, and fail some UTs, and also output some confusing logs.
We should find a way to make it better.