Details
-
Question
-
Status: Resolved
-
Trivial
-
Resolution: Invalid
-
None
-
None
-
None
Description
I am currently working on the latest version of Apache Spark (1.4.1), pre-built package for Hadoop 2.6+.
Is there any feature in Spark/Hadoop to encrypt RDDs or in-memory/cache (similarly to Altibase's HDB: http://altibase.com/in-memory-database-computing-solutions/security/) when running applications in Spark? Or is there an external library/framework which could be used to encrypt RDDs or in-memory/cache in Spark?
I discovered it is possible to encrypt the data, and encapsulate it into RDD. However, I feel this affects Spark's fast data processing as it is slower to encrypt the data, and then encapsulate it to RDD. Encryption and storing data should be done parallel.