Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22628

Document the custom WAL directory (hbase.wal.dir) usage

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1
    • documentation, wal
    • None
    • Reviewed

    Description

      Custom WAL directory usage must be documented, otherwise it may lead to inconsistent data during migrating to new WAL dir path.

       

      You can consider below scenario while migrating to custom WAL directory.

      1. Setup HBase cluster with the default setting (all WAL files are under the root directory ie. /hbase/WALs).
      2. Create table 't1' and insert few records
      3. Flush meta table (so that table region entries persist in FS)
      4. Forcibly kill HBase processes (HM & RS).
      5. Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL)
      6. Start the HBase servers
      7. Scan 't1'

      Ideally HMaster should submit split task of old RS(s) WAL files (created under /hbase/WALs) and old data should be replayed. But currently, during HM startup we populate the previous dead servers from the current WAL dir ( hbase.wal.dir -> /hbaseWAL).

       

      Since WAL dir path is new, so you need to copy RegionServer WAL directories manualy from old WAL dir to new path. 

      Attachments

        1. 002-view.png
          182 kB
          Reid Chan
        2. HBASE-22628.002.patch
          2 kB
          Pankaj Kumar
        3. HBASE-22628.patch
          2 kB
          Pankaj Kumar
        4. Screenshot-HBASE-22628.png
          93 kB
          Pankaj Kumar

        Issue Links

          Activity

            People

              pankaj2461 Pankaj Kumar
              pankaj2461 Pankaj Kumar
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: