Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-40242

Only return all physical plans after summitting pyspark script with several spark sql blocks inside

    XMLWordPrintableJSON

Details

    • Wish
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.1.3
    • None
    • Spark Submit, SQL
    • None

    Description

      Backgroud:

          In industry development environment, we got used to write several spark-sql blocks in one pyspark script. Without really submitting to cluster as running application, getting all the physical plans of a application is too hard to do within a parameter statement of "spark-submit" command.

       

      Wish:

          I wish to add a parameter "–genPlan" as for "spark-submit" command, which can return all the physical plans of a application instead of submitting to run indeed. Also, other approach to settle the matter is wellcome.

      Attachments

        Activity

          People

            Unassigned Unassigned
            Jake_00 Liang Fenjie
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: