Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-3281

Support for Hive UDFs on import

    XMLWordPrintableJSON

Details

    Description

      As many companies are using UDF to establish column level encryption during write time, Sqoop should support applying such a UDF during the write process.  This would be an extension of the map-column-hive functionality, where the value of the parseColumnMapping would accept the UDF:

      sqoop import --verbose --connect "jdbcconnectionstring" --username user --password password --hive-import --hive-database hiveschematest --map-column-hive "emptest.id=int,emptest.name=varchar(100),emptest.ssn=UDF_ENCRYPT()" -m 1

      With this approach, the data does not have to be written to HDFS in the clear.  This functionality can also be extended to other UDFs (naturally).

       

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            scyphers C Scyphers
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: