Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7340

DoFn.teardown metrics are lost in Python SDK

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-py-harness
    • Labels:
      None

      Description

      If user code in DoFn.shutdown updates custom user metrics, those updates will not get registered e.g. counter increments are not registered.

      Context: In FnApiRunner.run_stages, DoFn.teardown is called in worker_handler_manager.close_all, but this is called outside of the FnApiRunner.run_stage calls, so no metrics / monitoring info is retrieved there.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              myffical@gmail.com Yifan Mai
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: