Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22318

Java.io.exception:Two readers for

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.1.0
    • Fix Version/s: None
    • Component/s: Hive, HiveServer2
    • Labels:
      None

      Description

      I create a ACID table with ORC format:

       

      CREATE TABLE `some.TableA`( 
         ....
         )                                                                   
       ROW FORMAT SERDE                                   
         'org.apache.hadoop.hive.ql.io.orc.OrcSerde'      
       STORED AS INPUTFORMAT                              
         'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'  
       OUTPUTFORMAT                                       
         'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'  
       TBLPROPERTIES (                                    
         'bucketing_version'='2',                         
         'orc.compress'='SNAPPY',                         
         'transactional'='true',                          
         'transactional_properties'='default')

      After executing merge into operation:

      MERGE INTO some.TableA AS a USING (SELECT vend_no FROM some.TableB UNION ALL SELECT vend_no FROM some.TableC) AS b ON a.vend_no=b.vend_no WHEN MATCHED THEN DELETE
      

      the problem happend(when selecting the TableA, the exception happens too):

      java.io.IOException: java.io.IOException: Two readers for {originalWriteId: 4, bucket: 536870912(1.0.0), row: 2434, currentWriteId 25}: new [key={originalWriteId: 4, bucket: 536870912(1.0.0), row: 2434, currentWriteId 25}, nextRecord={2, 4, 536870912, 2434, 25, null}, reader=Hive ORC Reader(hdfs://hdpprod/warehouse/tablespace/managed/hive/some.db/tableA/delete_delta_0000015_0000026/bucket_00001, 9223372036854775807)], old [key={originalWriteId: 4, bucket: 536870912(1.0.0), row: 2434, currentWriteId 25}, nextRecord={2, 4, 536870912, 2434, 25, null}, reader=Hive ORC Reader(hdfs://hdpprod/warehouse/tablespace/managed/hive/some.db/tableA/delete_delta_0000015_0000026/bucket_00000

      Through orc_tools I scan all the files(bucket_00000,bucket_00001,bucket_00002) under delete_delta and find all rows of files are the same.I think this will cause the same key(RecordIdentifer) when scan the bucket_00001 after bucket_00000 but I don't know why all the rows are the same in these bucket files.

       

       

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                max_c max_c
              • Votes:
                1 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated: