Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7045

Element counters in the Web UI graph representations for transforms for Python streaming jobs in Google Cloud Dataflow

    XMLWordPrintableJSON

    Details

    • Flags:
      Important
    • Docs Text:
      Hide
      Unsupported features
      Python streaming execution does not currently support the following features.

      General Beam features
      These unsupported Beam features apply to all runners.

      State and Timers APIs
      Custom source API
      Splittable DoFn API
      Handling of late data
      User-defined custom WindowFn
      DataflowRunner specific features
      Additionally, DataflowRunner does not currently support the following Cloud Dataflow specific features with Python streaming execution.

      Streaming autoscaling
      Updating existing pipelines
      Cloud Dataflow Templates
      Some monitoring features, such as msec counters, display data, metrics, and element counts for transforms. However, logging, watermarks, and element counts for sources are supported.
      Show
      Unsupported features Python streaming execution does not currently support the following features. General Beam features These unsupported Beam features apply to all runners. State and Timers APIs Custom source API Splittable DoFn API Handling of late data User-defined custom WindowFn DataflowRunner specific features Additionally, DataflowRunner does not currently support the following Cloud Dataflow specific features with Python streaming execution. Streaming autoscaling Updating existing pipelines Cloud Dataflow Templates Some monitoring features, such as msec counters, display data, metrics, and element counts for transforms. However, logging, watermarks, and element counts for sources are supported.

      Description

      Users don't see the element counters in transforms in the Web UI graph representation when running a Python streaming job, which is expected behavior according to this Beam page.

      The feature request is to enable the element counters in the Web UI graph representations for transforms for Python streaming jobs in Google Cloud Dataflow.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                efimmuratov Fim
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: