Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21379 RegionServer Stop by ArrayIndexOutOfBoundsException of WAL when replication enabled
  3. HBASE-21401

Sanity check when constructing the KeyValue

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      Add a sanity check when constructing KeyValue from a byte[]. we use the constructor when we're reading kv from socket or HFIle or WAL(replication). the santiy check isn't designed for discovering the bits corruption in network transferring or disk IO. It is designed to detect bugs inside HBase in advance. and HBASE-21459 indicated that there's extremely small performance loss for diff kinds of keyvalue.
      Show
      Add a sanity check when constructing KeyValue from a byte[]. we use the constructor when we're reading kv from socket or HFIle or WAL(replication). the santiy check isn't designed for discovering the bits corruption in network transferring or disk IO. It is designed to detect bugs inside HBase in advance. and HBASE-21459 indicated that there's extremely small performance loss for diff kinds of keyvalue.

    Description

      In KeyValueDecoder & ByteBuffKeyValueDecoder, we pass a byte buffer to initialize the Cell without a sanity check (check each field's offset&len exceed the byte buffer or not), so ArrayIndexOutOfBoundsException may happen when read the cell's fields, such as HBASE-21379, it's hard to debug this kind of bug.

      An earlier check will help to find such kind of bugs.

      Attachments

        1. HBASE-21401.v7.patch
          29 kB
          Zheng Hu
        2. HBASE-21401.v6.patch
          25 kB
          Zheng Hu
        3. HBASE-21401.v5.patch
          25 kB
          Zheng Hu
        4. HBASE-21401.v4.patch
          25 kB
          Zheng Hu
        5. HBASE-21401.v4.patch
          25 kB
          Zheng Hu
        6. HBASE-21401.v3.patch
          25 kB
          Zheng Hu
        7. HBASE-21401.v2.patch
          25 kB
          Zheng Hu
        8. HBASE-21401.v1.patch
          18 kB
          Zheng Hu

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. HBASE-16213 A new HFileBlock structure for fast random get

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HBASE-22034 Backport HBASE-21401 and HBASE-22032 to branch-1

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link Review board

          Activity

            People

              openinx Zheng Hu
              openinx Zheng Hu
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: