Details
-
Wish
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.25.0
-
None
-
None
Description
Hi,
HTTP has content-encoding with gzip as a standard. See here: https://www.iana.org/assignments/http-parameters/http-parameters.xhtml
By using AWS S3 or other S3 compliant storages users compress the content to be uploaded with gzip and set the HTTP Header Content-Encodig: gzip and the mime.type accordingly.
The PutS3Object processor does not have an attribute to set the Content-Encoding header. User defined properties are put in generic HTTP Headers which do not help.
My wish would be:
- FetchS3Object:
- reads the content-encoding and decodes gzip into non gzip with an attribute: Evaluate Content-Encoding
- FetchS3 should create s3 header attributes which are stored in the s3 object
- PutS3Object
- Has a Content-Encoding attribute where users can put "gzip" as a possible value
- Has an option to compress the flow file content in gzip with true|false. This would have a huge performance impact, because currently we need to compress the content, s3 upload and then decompress it afterwards
Thank you in advance