Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-15320

HBase connector for Kafka Connect

    XMLWordPrintableJSON

Details

    • Hide
      This commit adds a kafka connector. The connectors acts as a replication peer and sends modifications in HBase to kafka.

      For further information, please refer to kafka/README.md.
      Show
      This commit adds a kafka connector. The connectors acts as a replication peer and sends modifications in HBase to kafka. For further information, please refer to kafka/README.md.

    Description

      Implement an HBase connector with source and sink tasks for the Connect framework (http://docs.confluent.io/2.0.0/connect/index.html) available in Kafka 0.9 and later.

      See also: http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines

      An HBase source (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task) could be implemented as a replication endpoint or WALObserver, publishing cluster wide change streams from the WAL to one or more topics, with configurable mapping and partitioning of table changes to topics.

      An HBase sink task (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks) would persist, with optional transformation (JSON? Avro?, map fields to native schema?), Kafka SinkRecords into HBase tables.

      Attachments

        1. 15320.master.16.patch
          118 kB
          Michael Stack
        2. 15320.master.16.patch
          118 kB
          Ted Yu
        3. HBASE-15320.master.1.patch
          121 kB
          Mike Wingert
        4. HBASE-15320.master.10.patch
          122 kB
          Mike Wingert
        5. HBASE-15320.master.11.patch
          109 kB
          Mike Wingert
        6. HBASE-15320.master.12.patch
          113 kB
          Mike Wingert
        7. HBASE-15320.master.14.patch
          114 kB
          Mike Wingert
        8. HBASE-15320.master.15.patch
          106 kB
          Mike Wingert
        9. HBASE-15320.master.2.patch
          121 kB
          Mike Wingert
        10. HBASE-15320.master.3.patch
          121 kB
          Mike Wingert
        11. HBASE-15320.master.4.patch
          122 kB
          Mike Wingert
        12. HBASE-15320.master.5.patch
          122 kB
          Mike Wingert
        13. HBASE-15320.master.6.patch
          122 kB
          Mike Wingert
        14. HBASE-15320.master.7.patch
          122 kB
          Mike Wingert
        15. HBASE-15320.master.8.patch
          125 kB
          Mike Wingert
        16. HBASE-15320.master.8.patch
          122 kB
          Mike Wingert
        17. HBASE-15320.master.9.patch
          125 kB
          Mike Wingert
        18. HBASE-15320.pdf
          451 kB
          Mike Wingert
        19. HBASE-15320.pdf
          451 kB
          Mike Wingert

        Issue Links

          Activity

            People

              MikeWingert Mike Wingert
              apurtell Andrew Kyle Purtell
              Votes:
              5 Vote for this issue
              Watchers:
              34 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: