Details
-
Improvement
-
Status: Resolved
-
P3
-
Resolution: Fixed
-
None
-
None
Description
Many runners, store both PipelineOptions and SerializablePipelineOptions in their context whereas SerializablePipelineOptions stores both Options and json serialized Options. Options can be obtain by SerializablePipelineOptions.get() but json form cannot be obtained. Similarly, we cannot create a SerializablePipelineOptions from json. It is needed in some cases (like spark structured streaming) where we cannot inject anything else than Strings into the framework.
Attachments
Issue Links
- links to