Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-8740

Sorted dynamic partition does not work correctly with constant folding

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.14.0
    • None
    • None

    Description

      Sorted dynamic partition optimization looks for partition columns from the operator above FileSinkOperator. As per hive convention it expects partition columns at the last. But with HIVE-8585 equality filters on partition columns gets folded to constant. The column pruner then prunes the constant expression as they don't reference any columns. This in some cases will yield unexpected results (throw ArrayIndexOutOfBounds exception) with sorted dynamic partition insert optimization. In such cases we don't really need sorted dynamic partition optimization.

      Attachments

        1. HIVE-8740.1.patch
          54 kB
          Prasanth Jayachandran
        2. HIVE-8740.2.patch
          54 kB
          Prasanth Jayachandran
        3. HIVE-8740.3.patch
          70 kB
          Prasanth Jayachandran
        4. HIVE-8740.4.patch
          69 kB
          Prasanth Jayachandran

        Issue Links

          Activity

            People

              prasanth_j Prasanth Jayachandran
              prasanth_j Prasanth Jayachandran
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: