Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16332

When create a partitioned text format table with one partition, after we change the format of table to orc, then the array type field may output error.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Critical
    • Resolution: Unresolved
    • 2.1.1
    • None
    • ORC
    • orc array

    Description

      ##The step to reproduce the result.
      1. First crate a text format table with array type field in hive.

      ```
      create table test_text_orc (
      col_int bigint,
      col_text string,
      col_array array<string>,
      col_map map<string, string>
      )
      PARTITIONED BY (
      day string
      )
      ROW FORMAT DELIMITED
      FIELDS TERMINATED BY ','
      collection items TERMINATED BY ']'
      map keys TERMINATED BY ':'
      ;

      ```
      2. Create new text file hive-orc-text-file-array-error-test.txt.
      ```
      1,text_value1,array_value1]array_value2]array_value3, map_key1:map_value1,map_key2:map_value2
      2,text_value2,array_value4, map_key1:map_value3
      ,text_value3,, map_key1:]map_key3:map_value3
      ```
      3. Load the data into one partition.

      ```
      LOAD DATA local INPATH '.hive-orc-text-file-array-error-test.txt' overwrite into table test_text_orc partition(day=20170329)
      ```

      4. select the data to verify the result.
      ```
      hive> select * from test.test_text_orc;
      OK
      1 text_value1 ["array_value1","array_value2","array_value3"]

      {" map_key1":"map_value1","map_key2":"map_value2"}

      20170329
      2 text_value2 ["array_value4"]

      {"map_key1":"map_value3"}

      20170329
      NULL text_value3 []

      {" map_key1":"","map_key3":"map_value3"}

      20170329

      ```
      5. Alter table format of table to orc;
      ```

      alter table test_text_orc set fileformat orc;
      ```
      6. Check the result again, and you can see the error result.

      ```
      hive> select * from test.test_text_orc;
      OK
      1 text_value1 ["array_value1","array_value2","array_value3"]

      {" map_key1":"map_value1","map_key2":"map_value2"}

      20170329
      2 text_value2 ["array_value4","array_value2","array_value3"]

      {"map_key1":"map_value3"}

      20170329
      NULL text_value3 ["array_value4","array_value2","array_value3"]

      {"map_key3":"map_value3"," map_key1":""}

      20170329
      ```

      Attachments

        1. HIVE-16332.1.patch
          1 kB
          Zhizhen Hou

        Activity

          People

            houzhizhen Zhizhen Hou
            houzhizhen Zhizhen Hou
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: