Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-6851

HadoopFileSystem doesn't Support Recursive Globs (**)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.12.0
    • Labels:
      None

      Description

      The LocalFileSystem supports recursive globs, **, through use of the glob syntax and it would be nice to similarly allow it in the HadoopFileSystem.

      The FileSystem docs say:

      All FileSystem implementations should support glob in the final hierarchical path component of ResourceIdT. This allows SDK libraries to construct file system agnostic spec. FileSystems can support additional patterns for user-provided specs.
      

      This clearly points out that recursive globs aren't required but may be supported as an additional pattern for user-provided specs.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                winkelman.kyle Kyle Winkelman
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m