Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20502

Fix NPE while running skewjoin_mapjoin10.q when column stats is used.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • Statistics
    • None

    Description

      Enabling hive.stats.fetch.column.stats makes this test fail during:

      EXPLAIN
      SELECT a.*, b.* FROM T1_n151 a RIGHT OUTER JOIN T2_n88 b ON a.key = b.key
      

      Seems like joinKeys is null at this point

      Exception:

      2018-09-04T23:47:02,398 DEBUG [fef236ce-e62e-4c20-b0c0-3b15d2b336f7 main] annotation.StatsRulesProcFactory: STATS-JOIN[15]: detects none/multiple PK parents.
      2018-09-04T23:47:02,409 ERROR [fef236ce-e62e-4c20-b0c0-3b15d2b336f7 main] ql.Driver: FAILED: NullPointerException null
      java.lang.NullPointerException
              at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.isJoinKey(StatsRulesProcFactory.java:2169)
              at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.updateNumNulls(StatsRulesProcFactory.java:2210)
              at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.updateColStats(StatsRulesProcFactory.java:2276)
              at org.apache.hadoop.hive.ql.optimizer.stats.annotation.StatsRulesProcFactory$JoinStatsRule.process(StatsRulesProcFactory.java:1785)
              at org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(DefaultRuleDispatcher.java:90)
              at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(DefaultGraphWalker.java:105)
              at org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(DefaultGraphWalker.java:89)
              at org.apache.hadoop.hive.ql.lib.LevelOrderWalker.walk(LevelOrderWalker.java:143)
      

      Attachments

        1. HIVE-20502.1.patch
          1 kB
          Daniel Voros
        2. HIVE-20502.2.patch
          20 kB
          Daniel Voros
        3. HIVE-20502.2.patch
          20 kB
          Zoltan Haindrich
        4. HIVE-20502.2.patch
          20 kB
          Zoltan Haindrich

        Issue Links

          Activity

            People

              dvoros Daniel Voros
              kgyrtkirk Zoltan Haindrich
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: