Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18622

Vectorization: IF Statements, Comparisons, and more do not handle NULLs correctly

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 3.0.0
    • Hive
    • None

    Description

       
      Many vector expression classes are setting noNulls to true which does not work if the VRB is a scratch column being reused. The previous use may have set noNulls to false and the isNull array will have some rows marked as NULL. The result is wrong query results and sometimes NPEs (for BytesColumnVector).

      So, many vector expressions need this:

            // Carefully handle NULLs...
      
            /*
             * For better performance on LONG/DOUBLE we don't want the conditional
             * statements inside the for loop.
             */
            outputColVector.noNulls = false;
       

      And, vector expressions need to make sure the isNull array entry is set when outputColVector.noNulls is false.

      And, all place that assign column value need to set noNulls to false when the value is NULL.

      Almost all cases where noNulls is set to true are incorrect.

      Attachments

        1. HIVE-18622.03.patch
          590 kB
          Matt McCline
        2. HIVE-18622.04.patch
          736 kB
          Matt McCline
        3. HIVE-18622.05.patch
          702 kB
          Matt McCline
        4. HIVE-18622.06.patch
          786 kB
          Matt McCline
        5. HIVE-18622.07.patch
          907 kB
          Matt McCline
        6. HIVE-18622.08.patch
          1.65 MB
          Matt McCline
        7. HIVE-18622.09.patch
          2.03 MB
          Matt McCline
        8. HIVE-18622.091.patch
          2.26 MB
          Matt McCline
        9. HIVE-18622.092.patch
          2.28 MB
          Matt McCline
        10. HIVE-18622.093.patch
          2.29 MB
          Matt McCline
        11. HIVE-18622.094.patch
          2.33 MB
          Matt McCline
        12. HIVE-18622.095.patch
          2.36 MB
          Matt McCline
        13. HIVE-18622.096.patch
          2.36 MB
          Matt McCline
        14. HIVE-18622.097.patch
          2.44 MB
          Matt McCline
        15. HIVE-18622.098.patch
          2.45 MB
          Matt McCline
        16. HIVE-18622.099.patch
          2.43 MB
          Matt McCline
        17. HIVE-18622.0992.patch
          2.43 MB
          Matt McCline
        18. HIVE-18622.0993.patch
          2.43 MB
          Matt McCline
        19. HIVE-18622.0994.patch
          2.45 MB
          Matt McCline

        Issue Links

          Activity

            People

              mmccline Matt McCline
              mmccline Matt McCline
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: