Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9488

Python SDK sending unexpected MonitoringInfo

Details

    • Bug
    • Status: Resolved
    • P3
    • Resolution: Fixed
    • None
    • 2.21.0
    • sdk-py-core

    Description

      element_count metrics is supposed to be tied with pcollection ids, but by inspecting what is sent over by python sdk, we see there are monitoringInfo sent wit ptransforms in it.

      [Double checked the job graph, these seem to be redundant. i.e. the corresponding pcollection does have its own MonitoringInfo reported.]

      Likely a bug.

      Proof:

      urn: "beam:metric:element_count:v1"
      type: "beam:metrics:sum_int_64"
      metric {
      counter_data

      { int64_value: 1 }

      }
      labels

      { key: "PTRANSFORM" value: "start/MaybeReshuffle/Reshuffle/RemoveRandomKeys-ptransform-85" }

      labels

      { key: "TAG" value: "None" }

      timestamp

      { seconds: 1583949073 nanos: 842402935 }

      Attachments

        Issue Links

          Activity

            People

              lcwik Luke Cwik
              ruoyun Ruoyun Huang
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3.5h
                  3.5h