Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16026

Cost-based Optimizer Framework

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • 2.3.0
    • SQL

    Description

      This is an umbrella ticket to implement a cost-based optimizer framework beyond broadcast join selection. This framework can be used to implement some useful optimizations such as join reordering.

      The design should discuss how to break the work down into multiple, smaller logical units. For example, changes to statistics class, system catalog, cost estimation/propagation in expressions, cost estimation/propagation in operators can be done in decoupled pull requests.

      Attachments

        Issue Links

          contains

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-17129 Support statistics collection and cardinality estimation for partitioned tables

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-21975 Histogram support in cost-based optimizer

          • Major - Major loss of function.
          • Resolved
          is duplicated by

          Sub-task - The sub-task of the issue SPARK-2216 Cost-based join reordering

          • Major - Major loss of function.
          • Closed
          is related to

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-23839 consider bucket join in cost-based JoinReorder rule

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          relates to

          Sub-task - The sub-task of the issue SPARK-17791 Join reordering using star schema detection

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved

          Activity

            People

              ZenWzh Zhenhua Wang
              rxin Reynold Xin
              Votes:
              12 Vote for this issue
              Watchers:
              97 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: