Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-14379

Replication V2

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • None
    • None
    • Replication
    • None

    Description

      Replication V2 is a tear-down of exiting replication code to just the interfaces introduced in HBASE-11367, then a rebuild around the following principles, goals, and suggested features:

      • No state in ZooKeeper. Introduce a new system table for tracking peers, queues, and log positions. (Some discussion on HBASE-10295, probably will be replaced with a set of more focused issues.)
      • Allow replication v1 and v2 to coexist. Note all of the undesirable features of v1 will remain as long as v1 is active, 'fixing' v1 is out of scope. Supporting communication between v1 and v2 endpoints would also be out of scope.
      • Simplified internal programming model based on iterators
      • Streaming data transfer
      • Administrative actions mediated by the master with support for security hooks (like HBASE-11392)
      • Replication state persisted and communicated with protobuf (like HBASE-11393 but everywhere)
      • Detailed metrics
      • Support for at least simple status checks and admin actions via UI and shell
      • Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
      • Support for bulk load, perhaps through augmenting bulk load to build WALs as well as HFiles (see HBASE-13153)
      • Optional consideration for replicating schema as well as data (like HBASE-12947). May fall out of scope.
      • Optional separation of replication function from the regionservers (see HBASE-8772)
      • Optional alternate scheduling of edits besides FIFO-by-region (see HBASE-1734 and HBASE-14014)

      There are a number of existing JIRAs that will eventually be closed as duplicate, wont fix, or reparented here.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              apurtell Andrew Kyle Purtell
              Votes:
              0 Vote for this issue
              Watchers:
              33 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: