Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
Impala 2.2
-
None
Description
Impala allows the TBLPROPERTIES entries avro.schema.literal and avro.schema.url to be modified to any string, which differs to Hive which uses the metastore to check that a valid value has been provided. Impala should implement the same safety check mechanism as Hive.
Invalid values can cause major problems, such as this example:
[vm1.jeremybeard.net:21000] > create table test_npe_bug stored as avro tblproperties ('avro.schema.literal' = '{
"name": "test_npe_bug"
, "type": "record"
, "fields": [
{"name": "code", "type": "string"}
, {"name": "description", "type": "string"}
, {"name": "total_emp", "type": ["null", "int"]}
, {"name": "salary", "type": ["null", "int"]}
]
}');
Query: create table test_npe_bug stored as avro tblproperties ('avro.schema.literal' = '{
"name": "test_npe_bug"
, "type": "record"
, "fields": [
{"name": "code", "type": "string"}
, {"name": "description", "type": "string"}
, {"name": "total_emp", "type": ["null", "int"]}
, {"name": "salary", "type": ["null", "int"]}
]
}')
Fetched 0 row(s) in 0.08s
[vm1.jeremybeard.net:21000] > describe test_npe_bug;
Query: describe test_npe_bug
+-------------+--------+-------------------+
| name | type | comment |
+-------------+--------+-------------------+
| code | string | from deserializer |
| description | string | from deserializer |
| total_emp | int | from deserializer |
| salary | int | from deserializer |
+-------------+--------+-------------------+
Fetched 4 row(s) in 3.90s
[vm1.jeremybeard.net:21000] > alter table test_npe_bug set tblproperties ('avro.schema.literal'='');
Query: alter table test_npe_bug set tblproperties ('avro.schema.literal'='')
[vm1.jeremybeard.net:21000] > describe test_npe_bug;
Query: describe test_npe_bug
ERROR: AnalysisException: Failed to load metadata for table: default.test_npe_bug
CAUSED BY: TableLoadingException: Failed to load metadata for table: test_npe_bug
CAUSED BY: NullPointerException: null
[vm1.jeremybeard.net:21000] > drop table test_npe_bug;
Query: drop table test_npe_bug
ERROR: AnalysisException: Failed to load metadata for table: default.test_npe_bug
CAUSED BY: TableLoadingException: Failed to load metadata for table: test_npe_bug
CAUSED BY: NullPointerException: null