Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17437

Support specifying a WAL directory outside of the root directory

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      This patch adds support for specifying a WAL directory outside of the HBase root directory.

      Multiple configuration variables were added to accomplish this:
      hbase.wal.dir: used to configure where the root WAL directory is located. Could be on a different FileSystem than the root directory. WAL directory can not be set to a subdirectory of the root directory. The default value of this is the root directory if unset.

      hbase.rootdir.perms: Configures FileSystem permissions to set on the root directory. This is '700' by default.

      hbase.wal.dir.perms: Configures FileSystem permissions to set on the WAL directory FileSystem. This is '700' by default.
      Show
      This patch adds support for specifying a WAL directory outside of the HBase root directory. Multiple configuration variables were added to accomplish this: hbase.wal.dir: used to configure where the root WAL directory is located. Could be on a different FileSystem than the root directory. WAL directory can not be set to a subdirectory of the root directory. The default value of this is the root directory if unset. hbase.rootdir.perms: Configures FileSystem permissions to set on the root directory. This is '700' by default. hbase.wal.dir.perms: Configures FileSystem permissions to set on the WAL directory FileSystem. This is '700' by default.
    • Patch

    Description

      Currently, the WAL and the StoreFiles need to be on the same FileSystem. Some FileSystems (such as Amazon S3) don’t support append or consistent writes. These two properties are imperative for the WAL in order to avoid loss of writes. However, StoreFiles don’t necessarily need the same consistency guarantees (since writes are cached locally and if writes fail, they can always be replayed from the WAL).

      This JIRA aims to allow users to configure a log directory (for WALs) that is outside of the root directory or even in a different FileSystem. The default value will still put the log directory under the root directory.

      Attachments

        1. hbase-17437-master.patch
          74 kB
          Yishan Yang
        2. hbase-17437-branch-1.2.patch
          134 kB
          Yishan Yang
        3. HBASE-17437.master.012.patch
          95 kB
          Zach York
        4. HBASE-17437.master.011.patch
          96 kB
          Zach York
        5. HBASE-17437.master.010.patch
          100 kB
          Zach York
        6. HBASE-17437.master.009.patch
          100 kB
          Zach York
        7. HBASE-17437.master.008.patch
          100 kB
          Zach York
        8. HBASE-17437.master.007.patch
          102 kB
          Zach York
        9. HBASE-17437.master.006.patch
          102 kB
          Zach York
        10. HBASE-17437.master.005.patch
          103 kB
          Zach York
        11. HBASE-17437.master.004.patch
          94 kB
          Zach York
        12. HBASE-17437.master.003.patch
          94 kB
          Zach York
        13. HBASE-17437.master.002.patch
          94 kB
          Zach York
        14. HBASE-17437.master.001.patch
          94 kB
          Zach York
        15. HBASE-17437.branch-1.004.patch
          128 kB
          Zach York
        16. HBASE-17437.branch-1.003.patch
          128 kB
          Zach York
        17. HBASE-17437.branch-1.002.patch
          128 kB
          Zach York
        18. HBASE-17437.branch-1.001.patch
          104 kB
          Zach York

        Issue Links

          Activity

            People

              zyork Zach York
              yangyishan0901m Yishan Yang
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: