Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-7590

Convert PipelineOptionsMap to PipelineOption

Details

    • Improvement
    • Status: Resolved
    • P3
    • Resolution: Later
    • None
    • Not applicable
    • dsl-sql
    • None

    Description

      Currently, BeamCalciteTable keeps a map version of PipelineOptions and that map version is used in JDBCConnection and RelNodes as well. This map is empty when the pipeline is constructed from SQLTransform and it will have the parameters passed from JDBC Client when the pipeline is started by JDBC path. 

      Since for Row-Count estimation we need to use PipelineOptions (or its sub-classes) and we cannot convert a map that is created from a pipelineOptions Subclasses back to PipelineOptions, it is better to keep PipelineOptions object itself.

      Another thing that will be changed as a result is set command. Currently, if in JDBC we use Set Command for a pipeline option, it will only change that option in the map. This means even if the option is incorrect, it does not throw exception until it creates the actual Pipeline Options. However, if we are keeping the PipelineOptions class itself, then wee need to actually set the passed parameters (using reflection) which will throw exception at the time of setting them. 

       

      Attachments

        Issue Links

          Activity

            People

              riazela Alireza Samadianzakaria
              riazela Alireza Samadianzakaria
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 20m
                  4h 20m