Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-3287

Import to HDFS not working in parquet not working for LDAP SQL Server

    XMLWordPrintableJSON

Details

    Description

      I am importing data from SQL Server to HDFS. Seems there is a bug when autnetication mode is LDAP and format is parquet. Please let me know if this is a bug or I am doing incorrectly. I am using Cloudera distribution.

      Below different scenarios I have tested

      Command Used are.

      1) SQL Server (Native Connection) -> HDFS Text Format (WORKS FINE)

      sqoop import \
      --connect "jdbc:sqlserver://<<ServerName>>;databasename=<<DB_Name>>"
      --username xxxx \
      --password xxxx \
      --table 'table1' \
      --split-by col1 \
      --target-dir /user/user1/01/

      2) SQL Server (Native Connection) -> HDFS Parquet Format (WORKS FINE)

      sqoop import \
      --connect "jdbc:sqlserver://<<ServerName>>;databasename=<<DB_Name>>"
      --username xxxx \
      --password xxxx \
      --table 'table1' \
      --split-by col1 \
      --target-dir /user/user1/01/ \
      --as-parquetfile

       

      3) SQL Server (LDAP Connection) -> HDFS Text format (WORKS FINE)

      sqoop import \
      --connect "jdbc:jtds:sqlserver://<<Server2>>;useNTLMv2=true;domain=XXXX;databaseName=<<DB_name>>" \
      --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver \
      --username xxxx \
      --password xxxx \
      --table table2 \
      --target-dir /user/user1/02/ \
      – --schema=s1

      4) SQL Server (LDAP Connection) -> HDFS Parquet format (Executes successfully, but the exported files are not parquet, they are in TEXT format)

      sqoop import \
      --connect "jdbc:jtds:sqlserver://<<Server2>>;useNTLMv2=true;domain=XXXX;databaseName=<<DB_name>>" \
      --connection-manager org.apache.sqoop.manager.SQLServerManager --driver net.sourceforge.jtds.jdbc.Driver \
      --username xxxx \
      --password xxxx \
      --table table2 \
      --target-dir /user/user1/02/ \
      – --schema=s1 \
      --as-parquetfile

      Attachments

        Activity

          People

            Unassigned Unassigned
            ashishprem Ashish Kumar Sinha
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: