Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29373

DataSourceV2: Commands should not submit a spark job.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 3.0.0
    • 3.0.0
    • SQL
    • None

    Description

      DataSourceV2 Exec classes (ShowTablesExec, ShowNamespacesExec, etc.) all extend LeafExecNode. This results in running a job when executeCollect() is called. This breaks the previous behavior SPARK-19650.

      A new command physical operator will be introduced form which all V2 Exec classes derive to avoid running a job.

      Attachments

        Issue Links

          Activity

            People

              imback82 Terry Kim
              imback82 Terry Kim
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: