Details
-
New Feature
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
-
None
Description
Beam's FileSystem creates an abstraction for reading from files in many different places.
We should add a Hadoop FileSystem implementation (https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/fs/FileSystem.html) - that would enable us to read from any file system that implements FileSystem (including HDFS, azure, s3, etc..)
I'm investigating this now.
Attachments
Issue Links
1.
|
Implement beam FileSystem's copy() |
|
Resolved | Luke Cwik |
2.
|
Hadoop FileSystem needs to receive Hadoop Configuration |
|
Resolved | Luke Cwik |
3.
|
Implement delete |
|
Resolved | Luke Cwik |
4.
|
Implement ResourceIds for HadoopFileSystem |
|
Resolved | Luke Cwik |
5.
|
Implement match for HadoopFileSystem |
|
Resolved | Luke Cwik |