Implement consistent security controls for cluster, site-to-site, and API communications

As discovered in NIFI-1981, edge cases in configuration of cluster communications over TLS without client authentication caused errors in the application. We should provide a consistent experience, from documentation to configuration to execution:

• Machine to machine communication should have two settings – plaintext or TLS with mutual authentication.
  • Cluster
  • Site to Site
• The API / UI should allow more granular control – plaintext, TLS with server authentication only, or TLS with mutual authentication. Some clients (API consumers, users in an enterprise environment) may have client certificates, but the majority will not, and TLS authentication of the server, and data integrity and confidentiality assurances should still be available.
  • Site to site over the API (see NIFI-1857) will respect this setting for the TLS handshake negotiation, but will manually enforce the presence of a client certificate in an HTTP header on any request arriving over HTTPS.

The nifi.security.needClientAuth setting should be removed from nifi.properties. A new setting nifi.security.api.needClientAuth will be added, and documented to explicitly apply only to the API (and, by extension, Web UI).