Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-1831

Allow encrypted passwords in configuration files

    XMLWordPrintableJSON

Details

    Description

      Storing passwords in plaintext in configuration files is not a security best practice. While file access can be restricted through OS permissions, these configuration files can be accidentally checked into source control, shared or deployed to multiple instances, etc.

      NiFi should allow a deployer to provide an encrypted password in the configuration file to minimize exposure of the passwords. On application start-up, NiFi should decrypt the passwords in memory. NiFi should also include a utility to encrypt the raw passwords (and optionally populate the configuration files and provide additional metadata in the configuration files).

      I am aware this simply shifts the responsibility/delegation of trust from the passwords in the properties file to a new location on the same system, but mitigating the visibility of the raw passwords in the properties file can be one step in a defense in depth approach and is often mandated by security policies within organizations using NiFi.

      The key used for encryption should not be hard-coded into the application source code, nor should it be universally consistent. The key could be determined by reading static information from the deployed system and feeding it to a key derivation function based on a cryptographically-secure hash function, such as PBKDF2, bcrypt, or scrypt. However, this does introduce upgrade, system migration, and portability issues. These challenges will have to be kept in consideration when determining the key derivation process.

      Manual key entry is a possibility, and then the master key would only be present in memory, but this prevents automatic reboot on loss of power or other recovery scenario.

      This must be backward-compatible to allow systems with plaintext passwords to continue operating. Options for achieving this are to only attempt to decrypt passwords when a sibling property is present, or to match a specific format.

      For these examples, I have used the following default values:

      password: thisIsABadPassword
      key: 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210
      iv:  0123456789ABCDEFFEDCBA9876543210
      algorithm: AES/CBC 256-bit
      

      *Note: These values should not be used in production systems – the key and IV are common test values, and an AEAD cipher is preferable to provide cipher text integrity assurances, however OpenSSL does not support the use of AEAD ciphers for command-line encryption at this time*

      Example 1: here the sibling property indicates the password is encrypted and with which implementation; the absence of the property would default to a raw password

      hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
      🔓 0s @ 16:25:56 $ echo "thisIsABadPassword" > password.txt
      hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
      🔓 0s @ 16:26:47 $ ossl aes-256-cbc -e -nosalt -p -K 0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210 -iv 0123456789ABCDEFFEDCBA9876543210 -a -in password.txt -out password.enc
      key=0123456789ABCDEFFEDCBA98765432100123456789ABCDEFFEDCBA9876543210
      iv =0123456789ABCDEFFEDCBA9876543210
      hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
      🔓 0s @ 16:27:09 $ xxd password.enc
      0000000: 5643 5856 6146 6250 4158 364f 5743 7646  VCXVaFbPAX6OWCvF
      0000010: 6963 6b76 4a63 7744 3854 6b67 3731 4c76  ickvJcwD8Tkg71Lv
      0000020: 4d38 6d32 7952 4776 5739 413d 0a         M8m2yRGvW9A=.
      hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
      🔓 0s @ 16:27:16 $ more password.enc
      VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=
      hw12203:/Users/alopresto/Workspace/scratch/encrypted-passwords (master) alopresto
      🔓 0s @ 16:27:55 $
      

      In nifi.properties:

      nifi.security.keystorePasswd=VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=
      nifi.security.keystorePasswd.encrypted=AES-CBC-256
      

      Example 2: here the encrypted password has a header tag indicating both that it is encrypted and the algorithm used

          @Test
          public void testShouldDecryptPassword() throws Exception {
              // Arrange
              KeyedCipherProvider cipherProvider = new AESKeyedCipherProvider()
      
              final String PLAINTEXT = "thisIsABadPassword"
              logger.info("Expected: ${Hex.encodeHexString(PLAINTEXT.bytes)}")
      
              final byte[] IV = Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" as char[])
              final byte[] LOCAL_KEY = Hex.decodeHex("0123456789ABCDEFFEDCBA9876543210" * 2 as char[])
              // Generated via openssl enc -a
              final String CIPHER_TEXT = "VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A="
              byte[] cipherBytes = Base64.decoder.decode(CIPHER_TEXT)
      
              SecretKey localKey = new SecretKeySpec(LOCAL_KEY, "AES")
      
              EncryptionMethod encryptionMethod = EncryptionMethod.AES_CBC
              logger.info("Using algorithm: ${encryptionMethod.getAlgorithm()}")
              logger.info("Cipher text: \$nifipw\$${CIPHER_TEXT} ${cipherBytes.length + 8}")
      
              // Act
              Cipher cipher = cipherProvider.getCipher(encryptionMethod, localKey, IV, false)
              byte[] recoveredBytes = cipher.doFinal(cipherBytes)
              
              // OpenSSL adds a newline character during encryption
              String recovered = new String(recoveredBytes, "UTF-8").trim()
              logger.info("Recovered: ${recovered} ${Hex.encodeHexString(recoveredBytes)}")
      
              // Assert
              assert PLAINTEXT.equals(recovered)
          }
      

      In nifi.properties:

      nifi.security.keystorePasswd=$nifipw$VCXVaFbPAX6OWCvFickvJcwD8Tkg71LvM8m2yRGvW9A=
      

      Ideally, NiFi would use a pluggable implementation architecture to allow users to integrate with a variety of secret management services. There are both commercial and open source solutions, including CyberArk Enterprise Password Vault [1], Hashicorp Vault [2], and Square Keywhiz [3]. In the future, this could also be extended to Hardware Security Modules (HSM) like SafeNet Luna [4] and Amazon CloudHSM [5].

      [1] http://www.cyberark.com/products/privileged-account-security-solution/enterprise-password-vault/
      [2] https://www.vaultproject.io/
      [3] https://square.github.io/keywhiz/
      [4] http://www.safenet-inc.com/data-encryption/hardware-security-modules-hsms/luna-hsms-key-management/luna-sa-network-hsm/
      [5] https://aws.amazon.com/cloudhsm/

      Attachments

        Issue Links

          Activity

            People

              alopresto Andy LoPresto
              alopresto Andy LoPresto
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 504h
                  504h
                  Remaining:
                  Remaining Estimate - 504h
                  504h
                  Logged:
                  Time Spent - Not Specified
                  Not Specified