Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-8892

Move Chicago Taxi Example Benchmarks to Beam orchestrator

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • benchmarking-py
    • None

    Description

      As originally implemented, the Chicago Taxi Example benchmarks for Beam Python SDK rely on on a custom bash script that runs (orchestrates) each of the pipelines that are part of the Chicago Taxi Example.

       

      The TFX team recently developed a Beam-based pipeline orchestrator:  https://www.tensorflow.org/tfx/guide/beam_orchestrator

       

      The example with Beam orchestrator: https://github.com/tensorflow/tfx/blob/master/tfx/examples/chicago_taxi_pipeline/taxi_pipeline_beam.py

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              pabloem Pablo Estrada
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: