Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Duplicate
-
2.19.0
-
None
-
Microsoft Windows 10, Java 8
Description
Windows users on our team have started encountering "No files to stage has been found" errors when attempting to run Beam pipelines on Dataflow. They are still able to run on the direct runner fine.
It's difficult for me to provide a minimal reproduction case because I don't use a Windows machine. But I believe the failure happens when running any pipeline on Dataflow from a Windows machine without explicitly setting --filesToStage.
I suspect this change is the cause, because it seems to completely change the code for gathering files to stage from the classpath: https://github.com/apache/beam/commit/9629ff6c5bfd70770046468adf44e2eeb69e606d
I believe this stackoverflow is about the same issue: https://stackoverflow.com/questions/60586141/dataflowrunner-exits-with-no-files-to-stage-has-been-found
Caused by: java.lang.IllegalArgumentException: No files to stage has been found. at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:281) ... 30 more
Attachments
Issue Links
- is fixed by
-
BEAM-9452 PipelineResources algorithm is not working on Windows
- Resolved