Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28278

FileSystem.getFileStatus can not get the correct file length for files which are being written

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Replication
    • None

    Description

      When reading from a opening WAL file in replication, we will use FileSystem.getFileStatus to get the file length and then use it to calculate the trailer's position. But if a file is being written, you can not get the correct length from namenode, you could even get a 0 if the file has only one block(for most WAL files).

      This will not lead to real data loss as we do not rely on the file length to determine whether we should read the file, but since it will mess up the trailer's position, it may lead to incorrect metrics, like un cleanly closed wal files, and fail some UTs, and also output some confusing logs.

      We should find a way to make it better.

      Attachments

        Activity

          People

            Unassigned Unassigned
            zhangduo Duo Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: