hlog loss result in data loss when regionserver down network card

When tested hbase replication, I found that the data in the memory was lost. Through the source code of hbase, I found that after hbase split the meta table, it will delete the entire wal  directory , but hlog is still in the same directory, and which save the data in the memstore. Eventually cause data loss.

The specific process is as follows:

1.  put a few data into hbase table.
2.  turn down the network card of all regionserver nodes.
3.  turn up the network card of all regionserver nodes，regionserver will be killed.
4.  restart the hbase cluster, scan table and find no data in the table.

In hmaster log will print:

      master.MasterWalManager: Log dir for server xxx does not exist

The splitLogDistributed function in SplitLogManager.java caused this issue. hmaster will first call splitLogDistributed function to publish the split task of the meta table. After the meta split task is completed, the directory will be deleted. Then hmaster want to publish another normal split task to restore the data again, but no wal directory was found.

    waitForSplittingCompletion(batch, status);
      ...
    for (Path logDir : logDirs) {
      status.setStatus("Cleaning up log directory...");
      final FileSystem fs = logDir.getFileSystem(conf);
      try {
        if (fs.exists(logDir) && !fs.delete(logDir, false)) {
          LOG.warn("Unable to delete log src dir. Ignoring. " + logDir);
        }
      } catch (IOException ioe) {
        FileStatus[] files = fs.listStatus(logDir);
        if (files != null && files.length > 0) {
          LOG.warn("Returning success without actually splitting and "
              + "deleting all the log files in path " + logDir + ": "
              + Arrays.toString(files), ioe);
        } else {
          LOG.warn("Unable to delete log src dir. Ignoring. " + logDir, ioe);
        }
      }
    }

   My English is poor, so if there is anything unclear, can leave me a message.