Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-596

ClassCastException when using EMR Hadoop KMS with ORC encryption

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 1.6.2
    • None
    • Java
    • None
    • emr-5.29.0
      Hadoop KMS 2.8.5

    Description

      I am getting cast exception from Writer when using encryption and EMR Hadoop KMS to manage keys. 

      Exception in thread "main" java.lang.ClassCastException: org.apache.hadoop.crypto.key.kms.KMSClientProvider cannot be cast to org.apache.hadoop.crypto.key.KeyProviderCryptoExtension
      	at org.apache.orc.impl.HadoopShimsPre2_7$KeyProviderImpl.createLocalKey(HadoopShimsPre2_7.java:168)
      	at org.apache.orc.impl.WriterImpl.visitTypeTree(WriterImpl.java:894)
      	at org.apache.orc.impl.WriterImpl.visitTypeTree(WriterImpl.java:904)
      	at org.apache.orc.impl.WriterImpl.setupEncryption(WriterImpl.java:927)
      	at org.apache.orc.impl.WriterImpl.<init>(WriterImpl.java:160)
      	at org.apache.orc.OrcFile.createWriter(OrcFile.java:971)
      

      Changing (KeyProviderCryptoExtension)this.provider to (KeyProviderCryptoExtension.CryptoExtension)this.provider at HadoopShimsPre2_7.java:168 and :188 fixes this for me.

      Code to replicate attached - requires to point to a valid Hadoop KMS containing "test_key" key.

      Attachments

        1. WriteOrc.java
          1 kB
          Matt Czyz
        2. castexception.patch
          1 kB
          Matt Czyz

        Issue Links

          Activity

            People

              Unassigned Unassigned
              mattczyz Matt Czyz
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: