Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11065

Apache Beam Template to ingest from Apache Kafka to Google Pub/Sub

Details

    • Improvement
    • Status: Triage Needed
    • P3
    • Resolution: Done
    • None
    • 2.27.0
    • examples-java

    Description

      In the Beam repository, we have awesome examples on java and python, however, there are not presented any templates.

      The reason for templates here, some users would like to use parametrized flex templates for common use cases. For example, if I want to do a typical pipeline, read from some source, and write to another source, I would happy if Beam provides a finally implemented pipeline for my common case in which I can put only parameters to customize input and output sources.

      I propose to add the new directory under the examples folder and name it "templates". It will be a new Gradle module in the project. As a first template, I propose to implement Kafka to Pubsub template.

      Attachments

        Issue Links

          Activity

            People

              IKozyrev Ilya Kozyrev
              IKozyrev Ilya Kozyrev
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 6h 50m
                  6h 50m