Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Not A Problem
-
1.3.0
-
None
-
None
Description
When performing an import as avro datafile, Sqoop fails with the following exception:
12/07/24 15:59:20 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.IllegalArgumentException: Duplicate Column identifier specified: '_1D' java.lang.IllegalArgumentException: Duplicate Column identifier specified: '_1D' at com.cloudera.sqoop.orm.ClassWriter.generate(ClassWriter.java:1072) at com.cloudera.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:84) at com.cloudera.sqoop.tool.ImportTool.importTable(ImportTool.java:370) at com.cloudera.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:66) at com.cloudera.sqoop.Sqoop.run(Sqoop.java:146) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at com.cloudera.sqoop.Sqoop.runSqoop(Sqoop.java:182) at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:221) at com.cloudera.sqoop.Sqoop.runTool(Sqoop.java:230) at com.cloudera.sqoop.Sqoop.main(Sqoop.java:239)
The problem seems to be when generating avro schemas and java classes - column and table names starting with illegal identifiers are prefixed with an underscore, so for a table such as the following, a duplicate column name is generated.
+-------+---------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +-------+---------+------+-----+---------+-------+ | 1D | int(11) | YES | | NULL | | | _1D | int(11) | YES | | NULL | | +-------+---------+------+-----+---------+-------+
Attachments
Issue Links
- relates to
-
-
- Resolved
-