Details
-
Sub-task
-
Status: Open
-
Minor
-
Resolution: Unresolved
-
3.4.0
-
None
Description
parquet, avro etc are still stuck building with older hadoop releases.
This makes using new APIs hard (PARQUET-2171) and means that APIs which are 5 years old such as HADOOP-15229 just aren't picked up.
This lack of openFIle() adoption hurts working with files in cloud storage as
- extra HEAD requests are made
- read policies can't be explicitly set
- split start/end can't be passed down
Proposed
- create class org.apache.hadoop.io.WrappedOperations
- add methods to wrap the apis
- test in contract tests via reflection loading -verifies we have done it properly.
Attachments
Issue Links
- links to