Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-9501

Encryption on RDDs or in-memory/cache on Apache Spark

    XMLWordPrintableJSON

Details

    Description

      I am currently working on the latest version of Apache Spark (1.4.1), pre-built package for Hadoop 2.6+.

      Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory/cache (similarly to Altibase's HDB: http://altibase.com/in-memory-database-computing-solutions/security/) when running applications in Spark? Or is there an external library/framework which could be used to encrypt RDDs or in-memory/cache in Spark?

      I discovered it is possible to encrypt the data, and encapsulate it into RDD. However, I feel this affects Spark's fast data processing as it is slower to encrypt the data, and then encapsulate it to RDD. Encryption and storing data should be done parallel.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mdoreilly Matthew O'Reilly
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: