Details
-
Umbrella
-
Status: Closed
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
Description
Replication V2 is a tear-down of exiting replication code to just the interfaces introduced in HBASE-11367, then a rebuild around the following principles, goals, and suggested features:
- No state in ZooKeeper. Introduce a new system table for tracking peers, queues, and log positions. (Some discussion on
HBASE-10295, probably will be replaced with a set of more focused issues.) - Allow replication v1 and v2 to coexist. Note all of the undesirable features of v1 will remain as long as v1 is active, 'fixing' v1 is out of scope. Supporting communication between v1 and v2 endpoints would also be out of scope.
- Simplified internal programming model based on iterators
- Streaming data transfer
- Administrative actions mediated by the master with support for security hooks (like
HBASE-11392) - Replication state persisted and communicated with protobuf (like
HBASE-11393but everywhere) - Detailed metrics
- Support for at least simple status checks and admin actions via UI and shell
- Hbck support for fixing corrupt or stuck queues (like
HBASE-14014) - Support for bulk load, perhaps through augmenting bulk load to build WALs as well as HFiles (see
HBASE-13153) - Optional consideration for replicating schema as well as data (like
HBASE-12947). May fall out of scope. - Optional separation of replication function from the regionservers (see
HBASE-8772) - Optional alternate scheduling of edits besides FIFO-by-region (see
HBASE-1734andHBASE-14014)
There are a number of existing JIRAs that will eventually be closed as duplicate, wont fix, or reparented here.
Attachments
Issue Links
- duplicates
-
HBASE-15867 Move HBase replication tracking from ZooKeeper to HBase
- Resolved
- is related to
-
HBASE-14899 Create custom Streaming ReplicationEndpoint
- Closed
- is required by
-
HBASE-12947 Replicating DDL statements like create from one cluster to another
- Closed
1.
|
Replication should not use ZooKeeper at all for coordination | Closed | Unassigned | |
2.
|
Separate replication WAL reading from shipping | Closed | Vincent Poon |