Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7052

portableWordCountTask raises IOError

    XMLWordPrintableJSON

Details

    Description

      Got this error when trying to run (Python) portableWordCountTask on the Spark portable runner:

      RuntimeError: IOError: [Errno 2] No such file or directory: '/tmp/beam-temp-py-wordcount-direct-a9e7beba888211e8843d025000000001/44ba0386-4221-4abc-ae8d-5a2aa4610c4f.py-wordcount-direct' [while running 'write/Write/WriteImpl/WriteBundles']

      This does not seem to be an issue with Flink (maybe because of environment_cache_millis), but in any case the problem is fixed when I set --environment_type=LOOPBACK.

      Attachments

        Issue Links

          blocks

          Improvement - An improvement or enhancement to an existing feature or task. BEAM-7037 Spark portable runner: run wordcount.py

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. BEAM-7859 Portable Wordcount on Spark runner does not work in DOCKER execution mode.

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          links to

          Web Link GitHub Pull Request #8282

          Activity

            People

              ibzib Kyle Weaver
              ibzib Kyle Weaver
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1.5h
                  1.5h