Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-28065

Corrupt HFile data is mishandled in several cases

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.5.2
    • 2.6.0, 2.4.18, 2.5.6, 3.0.0-beta-1
    • HFile
    • None

    Description

      While riding over a spat of HDFS data corruption issues, we've observed several places in the read path that do not fall back to HDFS checksum appropriately. These failures manifest during client reads and during compactions. Sometimes failure is detected by the fallback verifyOnDiskSizeMatchesHeader, sometimes we attempt to allocate a buffer with a negative size, and sometimes we read through to a failure from block decompression.

      After code study, I think that all three cases arise from using a block header that was read without checksum validation.

      Will post up the stack traces in the comments. Not sure if we'll want a single patch or multiple.

      Attachments

        Activity

          People

            ndimiduk Nick Dimiduk
            ndimiduk Nick Dimiduk
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: