PutS3Object: Invalid V4 Authorization Header When Using Custom S3 Blobstore

Hello!

Some background: I'm currently attempting to use a PutS3Object processor in Nifi 1.10.0 to upload an object to a MinIO cluster. The MinIO cluster is configured to act as an S3-compatible blobstore in the us-east-1 region. The MinIO cluster is running on an internal private network at my company at https://s3.mydomain.mycompany.com .

The PutS3Object processor is configured thusly:

• Bucket: mybucket
• Region: US East (N. Virginia)
• Endpoint Override URL: https://s3.mydomain.mycompany.com:9000
• Signer Override: Signature v4

All other options are left at their default values.

What happens when I attempt to use the processor to put a file into MinIO is that the processor shows an error like the following: Status Code: 400, Error Code: AuthorizationHeaderMalformed.

After some debugging, it looks like that the HTTP Authorization header being generated by Nifi isn't quite what I would expect. The Authorization header starts off like this:

Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20200111/mydomain/s3/aws4_request ...

Whereas what I would expect is something more like this:

Authorization: AWS4-HMAC-SHA256 Credential=AKIAIOSFODNN7EXAMPLE/20200111/us-east-1/s3/aws4_request ...

The current behaviour seems to be: take part of the domain from the Endpoint Override URL and use that as the region inside of the Authorization header, instead of using the Region that was specified.

As a workaround for now we can use Signature v2 instead, but how long MinIO will continue to support Signature v2 at this time is unknown.

Would it be possible to fix the S3 family of processors so that they use the Region being specified instead of attempting to extract the region from the URL instead?