Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22582

Avoid reading table as ACID when table name is starting with "delta" , but table is not transactional and BI Split Strategy is used

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0, 4.0.0-alpha-1
    • None
    • None

    Description

      The issue is fixed in HIVE-22473 but missed a check for BI Split Strategy.

      Steps to reproduce: 

      set hive.exec.orc.split.strategy=BI;
      create table delta_result (a int) stored as orc tblproperties('transactional'='false');
      insert into delta_result select 1;
      select * from delta_result;
      

      Exception Stack Trace:

      Caused by: java.lang.RuntimeException: ORC split generation failed with exception: String index out of range: -1
              at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1929)
              at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:2016)
              at org.apache.hadoop.hive.ql.exec.FetchOperator.generateWrappedSplits(FetchOperator.java:461)
              at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextSplits(FetchOperator.java:430)
              at org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:336)
              at org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:576)
              ... 50 more
      Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range: -1
              at java.lang.String.substring(String.java:1967)
              at org.apache.hadoop.hive.ql.io.AcidUtils.parsedDelta(AcidUtils.java:1128)
              at org.apache.hadoop.hive.ql.io.AcidUtils$ParsedDeltaLight.parse(AcidUtils.java:921)
              at org.apache.hadoop.hive.ql.io.AcidUtils.getLogicalLength(AcidUtils.java:2084)
              at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$BISplitStrategy.getSplits(OrcInputFormat.java:1115)
              at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1905)
              ... 55 more
      

      Attachments

        1. HIVE-22582.patch
          0.8 kB
          Aditya Shah

        Issue Links

          Activity

            People

              aditya-shah Aditya Shah
              aditya-shah Aditya Shah
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: