Details
Description
Use case:
Old table was created using a SerDe and system has gone through some re-configuration resulting in that serde no longer being used and hence removed. Actions necessary to identify and fix the offending tables (e.g. describe, alter, delete) fail due to the SerDe no longer being there.
Repro Steps:
1. Create table that uses a SerDe loaded in externally (via "add jar ...")
2. Remove the jar that the SerDe comes from
3. Attempt to describe, alter (set serde ...), or delete the table created in step #1
Expected Behavior:
Regardless of the status of the SerDe behind the table, perform those actions. Loading the SerDe shouldn't be a requirement to do so.
Actual Behavior:
SerDeException thrown:
org.apache.hadoop.hive.serde2.SerDeUtils.lookupDeserializer(SerDeUtils.java:86) org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:207) org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:266) org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:259) org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:585) org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:168) org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967) org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.addInputsOutputsAlterTable(DDLSemanticAnalyzer.java:1109) org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableSerde(DDLSemanticAnalyzer.java:1037) org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:214) ...
or
java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException SerDe f.q.c.n.MySerDe does not exist)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:268)
at org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:259)
at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:585)
at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:168)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:868)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getAttemptTableName(DDLSemanticAnalyzer.java:1420)
at org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer$QualifiedNameUtil.getTableName(DDLSemanticAnalyzer.java:1478)
....
Workaround:
Find SerDe that the problematic table was configured to use (which may not readily be possible), and manually load it (via 'add jar ...') in order to fix the problem.
Attachments
Issue Links
- duplicates
-
HIVE-3392 Hive unnecessarily validates table SerDes when dropping a table
-
- Closed
-