Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19064

Synchronous replication for HBase

    XMLWordPrintableJSON

Details

    Description

      The guys from Alibaba made a presentation on HBaseCon Asia about the synchronous replication for HBase. We(Xiaomi) think this is a very useful feature for HBase so we want to bring it into the community version.

      This is a big feature so we plan to do it in a feature branch.

      Attachments

        1. HBASE-19064.master.patch
          552 kB
          Duo Zhang
        2. HBASE-19064.master.v1.patch
          561 kB
          Duo Zhang
        3. HBASE-19064.master.v2.patch
          562 kB
          Duo Zhang

        Issue Links

          1.
          Add a remote peer cluster wal directory config for synchronous replication Sub-task Resolved Guanghao Zhang
          2.
          Support setting up two clusters with A and S state Sub-task Resolved Duo Zhang
          3.
          Implement a procedure to convert RS from A to DA Sub-task Resolved Unassigned
          4.
          Implement a procedure to convert RS from S to DA Sub-task Resolved Guanghao Zhang
          5.
          Reject read/write from client but accept write from replication in state S Sub-task Resolved Duo Zhang
          6.
          Introduce a new log writer which can write to two HDFSes Sub-task Resolved Duo Zhang
          7.
          Introduce a special WALProvider for synchronous replication Sub-task Resolved Duo Zhang
          8.
          Add a new cluster state flag for synchronous replication Sub-task Resolved Guanghao Zhang
          9.
          Reject the replication request when peer is DA or A state Sub-task Resolved Zheng Hu
          10.
          Allow temporarily skipping the creation of remote writer for SynchronousReplicationWALProvider Sub-task Resolved Unassigned
          11.
          Complete the procedure for adding a sync replication peer Sub-task Resolved Duo Zhang
          12.
          Use protobuf instead of enum.ordinal to store SyncReplicationState Sub-task Resolved Guanghao Zhang
          13.
          Add UT for sync replication peer in DA state Sub-task Resolved Duo Zhang
          14.
          Only allow table replication for sync replication for now Sub-task Resolved Guanghao Zhang
          15.
          Introduce a new base class for replication peer procedure Sub-task Resolved Duo Zhang
          16.
          Allow write request from replication but reject write request from user client when S state. Sub-task Resolved Zheng Hu
          17.
          Only allow removing sync replication peer which is in DA state Sub-task Resolved Zheng Hu
          18.
          General framework to transit sync replication state Sub-task Resolved Duo Zhang
          19.
          General framework to transit sync replication state Sub-task Resolved Unassigned
          20.
          Implement a procedure to replay sync replication wal for standby cluster Sub-task Resolved Guanghao Zhang
          21.
          Create remote wal directory when transitting to state S Sub-task Resolved Duo Zhang
          22.
          Remove the SYNC_REPLICATION_ENABLED flag Sub-task Resolved Guanghao Zhang
          23.
          Forbid major compaction when standby cluster replay the remote wals Sub-task Resolved Guanghao Zhang
          24.
          Also remove the wal file in remote cluster when we finish replicating a file Sub-task Resolved Duo Zhang
          25.
          Operational documents for synchronous replication. Sub-task Resolved Zheng Hu
          26.
          Do not write the cluster id of the current active cluster when writing remote WAL Sub-task Resolved Zheng Hu
          27.
          Give up replicating anything in S state Sub-task Resolved Duo Zhang
          28.
          Cleanup related resources when remove a sync replication peer Sub-task Resolved Zheng Hu
          29.
          Also remove remote wals when peer is in DA state Sub-task Resolved Duo Zhang
          30.
          Support removing a ReplicationSourceShipper for a special wal group Sub-task Resolved Duo Zhang
          31.
          Support removing a WAL from LogRoller Sub-task Resolved Guanghao Zhang
          32.
          Concurrency issue for CombinedAsyncWriter Sub-task Resolved Unassigned
          33.
          NPE in RecoverStandbyProcedure.execute Sub-task Resolved Guanghao Zhang
          34.
          Check remote WAL directory when creating peer and transiting peer to A Sub-task Resolved Duo Zhang
          35.
          Allow writing WAL to local and remote cluster concurrently Sub-task Resolved Duo Zhang
          36.
          Polish the WAL switching when transiting from A to S Sub-task Resolved Duo Zhang
          37.
          Reopen regions using ReopenTableRegionsProcedure Sub-task Resolved Duo Zhang
          38.
          Performance comparison of synchronous replication branch and master branch Sub-task Resolved Zheng Hu
          39.
          NPE encountered when rolling update from master with an async peer to branch HBASE-19064 Sub-task Resolved Zheng Hu
          40.
          Fix the style issues on branch HBASE-19064 before merging back to master Sub-task Resolved Duo Zhang
          41.
          Fix broken TestReplicationShell Sub-task Resolved Duo Zhang

          Activity

            People

              zhangduo Duo Zhang
              zhangduo Duo Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              32 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: