Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
Custom WAL directory usage must be documented, otherwise it may lead to inconsistent data during migrating to new WAL dir path.
You can consider below scenario while migrating to custom WAL directory.
- Setup HBase cluster with the default setting (all WAL files are under the root directory ie. /hbase/WALs).
- Create table 't1' and insert few records
- Flush meta table (so that table region entries persist in FS)
- Forcibly kill HBase processes (HM & RS).
- Configure the hbase.wal.dir to outside the root dir (say /hbaseWAL)
- Start the HBase servers
- Scan 't1'
Ideally HMaster should submit split task of old RS(s) WAL files (created under /hbase/WALs) and old data should be replayed. But currently, during HM startup we populate the previous dead servers from the current WAL dir ( hbase.wal.dir -> /hbaseWAL).
Since WAL dir path is new, so you need to copy RegionServer WAL directories manualy from old WAL dir to new path.
Attachments
Attachments
Issue Links
- links to