Details

    • Sub-task
    • Status: Resolved
    • P2
    • Resolution: Fixed
    • None
    • 2.0.0
    • None

    Description

      Beam's FileSystem has a copy() command, however I can't find a good analog in Hadoop's FileSystem. https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/fs/FileSystem.html shows lots of copy to/from local files, but no "copy between these two arbitrary paths".

      cc davor dhalperi@google.com did either of you have thoughts about this? I don't think that it makes sense to have beam stream data from one node just so it can write it back to another node. (it could be an extension method, but I'd want to make it obvious that it's the inefficient version of things)

      My default answer here is to throw an unimplemented exception on copy or to remove it from the BFS interface altogether.

      Attachments

        Activity

          People

            lcwik Luke Cwik
            sisk Stephen Sisk
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: