Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12135

Batch optimized translation for Spark Runner

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • runner-spark
    • None

    Description

      Spark Runner and maybe all other runners that deal with batch only data might benefit of a batch optimized translation where details about the full Beam model matter less because we are in Global window, no panes info is needed and all records use the sane (min) timestamp. With this premise the records can be encoded as 'value only' WindowValues and transforms like GroupByKey may ignore windowing (GABW) to improve performance.

      Attachments

        1. image001.png
          254 kB
          Tao Li

        Issue Links

          Activity

            People

              Unassigned Unassigned
              iemejia Ismaël Mejía
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h
                  5h