Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
Description
Avro's output stream can not be used in combination with the MagicS3GuardCommitter
Error: java.lang.ClassCastException: class org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter cannot be cast to class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
The reason for this problem is that AvroOutputFormatBase.getAvroFileOutputStream tries to cast the outputcommiter to a FileOutputCommitter.
It can be solved by casting to a PathOutputCommitter instead (which is a superclass of both the FileOutputCommitter and the MagicS3GuardCommitter)
Attachments
Issue Links
- relates to
-
MAPREDUCE-7341 Add a task-manifest output committer for Azure and GCS
- Resolved
- links to