Details
-
Bug
-
Status: Resolved
-
P2
-
Resolution: Fixed
-
None
Description
Summary:
RowCoderGenerator caches a delegate Coder<Row> once encode or decode is exercised, but there's not an API for caching this delegate eagerly.
Use Case:
When creating several PCollections to perform distinct reads with the same schema, you might create one RowCoder.of(schema) before creating the list of PCollections / PCollectionsList. However, once the pipeline begins and rows arrive for encoding, these pipelines will simultaneously try to cache a delegate coder for the row's schema.
Workaround:
You can force the eager caching of the code by exercising encode in the main application before creating PCollections using the RowCoder:
try { myRowCoder.encode(null, null); } catch (IOException | NullPointerException e) { // do nothing }
Context:
I've only encountered this during development with the direct runner.
Attachments
Issue Links
- links to