Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10101

Add a HttpIO / HttpFileSystem

Details

    • Bug
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-ideas
    • None

    Description

      Add HttpIO (and a related HttpFileSystem and HttpsFileSystem), which can download files from a particular http:// or https:// URL. HttpIO cannot upload / write to files, though, because there's no standardized way to write to files using HTTP.

      Sample usage:

       

              (
                  p
                  | ReadFromText("https://raw.githubusercontent.com/apache/beam/5ff5313f0913ec81d31ad306400ad30c0a928b34/NOTICE")
                  | WriteToText("output.txt", shard_name_template="", num_shards=0)
              )
      

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            epicfaace Ashwin Ramaswami
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 5h 50m
                5h 50m