Details
-
New Feature
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
3.0.0
-
None
-
None
Description
When one uses HCatInputFormat/HCatLoader to read from file-formats that support predicate pushdown (such as ORC, with hive.optimize.index.filter=true), one sees that the predicates aren't actually pushed down into the storage layer.
The forthcoming patch should allow for filter-pushdown, if any of the partitions being scanned with HCatLoader support the functionality. The patch should technically allow the same for users of HCatInputFormat, but I don't currently have a neat interface to build a compound predicate-expression. Will add this separately, if required.
Attachments
Attachments
Issue Links
- blocks
-
HIVE-17754 HCat InputJobInfo in Pig UDFContext is heavyweight, and causes OOMs in Tez AMs
- Patch Available