Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
C++ supports IpcReadOptions.included_fields which lets you load a subset of (top-level) fields from an IPC file or stream, potentially saving on I/O costs. It would be useful to support this in Java as well. Some refactoring would be required since MessageSerializer currently reads record batch messages in as a whole, and it would be good to quantify how much of a benefit this provides in different scenarios.