Details
-
Improvement
-
Status: Resolved
-
P2
-
Resolution: Not A Bug
-
0.1.0-incubating, 0.2.0-incubating
-
None
Description
Currently, TextIO supports sharding (or withoutSharding() option) only in Write.
I think it makes sense to also supports sharding on the Read behaving like:
read the input in chunks of n MB (to avoid heap OutOfMemory with large files).