Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-11946

Use ReadFromKafkaDoFn for KafkaIO.Read by default when beam_fn_api is enabled

Details

    • Improvement
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.29.0
    • io-java-kafka
    • None

    Description

      There was a data loss bug in Dataflow when executing x-lang pipeline, which prevents us using ReadFromKafkaDoFn for KafkaIO default. Now the bug is fixed on dataflow side and we should use ReadFromKafkaDoFn when with beam_fn_api instead of using sdf wrapper.

      Attachments

        Issue Links

          Activity

            People

              boyuanz Boyuan Zhang
              boyuanz Boyuan Zhang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m