Description
The goal of this ticket is to extend the PutHDFS processor with the ability to append avro records. The processor already provides an option to set 'append' as conflict resolution strategy, but that does not work correctly in case of avro files, because the serialized avro file cannot be deserialized again (because the binary content is invalid).
Some notes about the implementation:
- The user needs to explicitly select avro as file format and append as conflict resolution mode to enable 'avro append' mode, otherwise regular append mode will work just as before. There is no auto detection of mimetype for the incoming flowfile.
- The records of the incoming flowfile and the ones in the existing avro file need to conform to the same avro schema, otherwise the append operation fails with incompatible schema.
- The 'avro append' mode should only work when compression type is set to 'none', if any other compression type is selected in 'avro append' mode the user should get a validation error.
The changes will have to be added to support/nifi-1.x branch also.
Attachments
Attachments
Issue Links
- links to