Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22163

CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • CBO
    • None

    Description

      create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
      alter table claims update statistics set ('numRows'='1154941534','rawDataSize'='1135307527922');
      
      
      set hive.stats.estimate=false;
      
      explain extended select count(1) from claims where typ_c=3;
      
      set hive.stats.ndv.estimate.percent=5e-7;
      
      explain extended select count(1) from claims where typ_c=3;
      

      Expecting the standard /2 for the single filter, but we instead get 5 rows.

      '            Map Operator Tree:'
      '                TableScan'
      '                  alias: claims'
      '                  filterExpr: (typ_c = 3) (type: boolean)'
      '                  Statistics: Num rows: 1154941534 Data size: 4388777832 Basic stats: COMPLETE Column stats: NONE'
      '                  GatherStats: false'
      '                  Filter Operator'
      '                    isSamplingPred: false'
      '                    predicate: (typ_c = 3) (type: boolean)'
      '                    Statistics: Num rows: 5 Data size: 19 Basic stats: COMPLETE Column stats: NONE'
      

      The estimation is in effect, as changing the estimate.percent changes this.

      '                  filterExpr: (typ_c = 3) (type: boolean)'
      '                  Statistics: Num rows: 1154941534 Data size: 4388777832 Basic stats: COMPLETE Column stats: NONE'
      '                  GatherStats: false'
      '                  Filter Operator'
      '                    isSamplingPred: false'
      '                    predicate: (typ_c = 3) (type: boolean)'
      '                    Statistics: Num rows: 230988307 Data size: 877755567 Basic stats: COMPLETE Column stats: NONE'
      

      Attachments

        1. HIVE-22163.1.patch
          14 kB
          Krisztian Kasa
        2. HIVE-22163.1.patch
          14 kB
          Krisztian Kasa
        3. HIVE-22163.1.patch
          14 kB
          Krisztian Kasa
        4. HIVE-22163.2.patch
          34 kB
          Krisztian Kasa
        5. HIVE-22163.3.patch
          33 kB
          Krisztian Kasa
        6. HIVE-22163.4.patch
          33 kB
          Krisztian Kasa
        7. HIVE-22163.4.patch
          33 kB
          Krisztian Kasa
        8. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa
        9. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa
        10. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa
        11. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa
        12. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa
        13. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa
        14. HIVE-22163.5.patch
          33 kB
          Krisztian Kasa

        Issue Links

          Activity

            People

              kkasa Krisztian Kasa
              gopalv Gopal Vijayaraghavan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: