Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-5464

Portable beam hangs while running TFX preprocessing step on a distributed cluster

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • P3
    • Resolution: Fixed
    • None
    • Not applicable
    • java-fn-execution
    • None

    Description

      Recently I went through the exercise of running the TFX taxi example on a dataproc cluster.  However it would always hang indefinitely.  The flink UI indicated that the job was halfway done.  However I could not see any clear errors in the job driver logs, the job service logs, or the Flink logs.  The root cause is still a mystery to me.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-2889 Flink runs portable pipelines

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          is blocked by

          Bug - A problem which impairs or prevents the functions of the product. FLINK-10672 Task stuck while writing output to flink

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #6897

          Activity

            People

              Unassigned Unassigned
              axelmagn Axel Magnuson
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m