Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-3455

Sqoop job fails while importing to S3 as Parquet

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Blocker
    • Resolution: Unresolved
    • 1.4.7
    • None
    • sqoop2-kite-connector
    • None

    Description

      A Sqoop job to import data from a MySQL database into S3 fails on using --as-parquetfile with the error as shown below:


      {{ERROR sqoop.Sqoop: Got exception running Sqoop: org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern: dataset:s3://sqoop-trial-bucket/sqoop-trial/trial
      Check that JARs for s3 datasets are on the classpath
      org.kitesdk.data.DatasetNotFoundException: Unknown dataset URI pattern: dataset:s3://}}sqoop-trial-bucket/sqoop-trial/trial Check that JARs for s3 datasets are on the classpath at org.kitesdk.data.spi.Registration.lookupDatasetUri(Registration.java:128) at org.kitesdk.data.Datasets.exists(Datasets.java:624) at org.kitesdk.data.Datasets.exists(Datasets.java:646) at org.apache.sqoop.mapreduce.ParquetJob.configureImportJob(ParquetJob.java:118) at org.apache.sqoop.mapreduce.DataDrivenImportJob.configureMapper(DataDrivenImportJob.java:132) at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:264) at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692) at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:520) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628) at org.apache.sqoop.Sqoop.run(Sqoop.java:147) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243) at org.apache.sqoop.Sqoop.main(Sqoop.java:252)


      {{}}

      All the JARs for S3 are present in the classpath. Further, the same works on simply removing the argument --as-parquetfile, i.e. with any other format.

      {{}}

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              kritijha Kriti Jha
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: