Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27529

Provide RS coproc ability to attach WAL extended attributes to mutations at replication sink

    XMLWordPrintableJSON

Details

    • Reviewed
    • Hide
      New regionserver coproc endpoints that can be used by coproc at the replication sink cluster if WAL has extended attributes.
      Using the new endpoints, WAL extended attributes can be transferred to Mutation attributes at the replication sink cluster.
      Show
      New regionserver coproc endpoints that can be used by coproc at the replication sink cluster if WAL has extended attributes. Using the new endpoints, WAL extended attributes can be transferred to Mutation attributes at the replication sink cluster.

    Description

      HBase provides coproc ability to enhance WALKey attributes (a.k.a. WAL annotations) in order for the replication sink cluster to build required metadata with the mutations. The endpoint is preWALAppend(). This ability was provided by HBASE-22622. The map of extended attributes is optional and hence not directly used by hbase internally. 

      For any hbase downstreamers to build CDC (Change Data Capture) like functionality, it might required additional metadata in addition to the ones being used by hbase already (replication scope, list of cluster ids, seq id, table name, region id etc). For instance, Phoenix uses many additional attributes like tenant id, schema name, table type etc.
      We already have this extended map of attributes available in WAL protobuf, to provide us the capability to (de)serialize it. While creating new ReplicateWALEntryRequest from the list of WAL entires, we are able to serialize the additional attributes. Similarly, at the replication sink side, the deserialized WALEntry has the extended attributes available.

      At the sink cluster, we should be able to attach the deserialized extended attributes to the newly generated mutations so that the peer cluster can utilize the mutation attributes to re-build required metadata.

      Attachments

        Issue Links

          Activity

            People

              vjasani Viraj Jasani
              vjasani Viraj Jasani
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: