Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16009

HoS: refactor set reducer parallelism

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Spark
    • None

    Description

      There are two paths for Spark set reducer parallelism, depending on the config hive.spark.use.op.stats. We currently handle all the logic in SetSparkReducerParallelism, and the logic is a bit complicated. Ideally we should refactor this to make it clearer, perhaps with separate rules for the paths.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              csun Chao Sun
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated: