Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6961

Pull up FileOutputCommitter.getOutputPath to PathOutputCommitter

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Duplicate
    • 3.0.0-beta1
    • None
    • mrv2
    • None

    Description

      SPARK-21549 has shown that downstream code is relying on the internal property

      if we pulled FileOutputCommitter.getOutputPath to the PathOutputCommitter of MAPREDUCE-6956, then there'd be a public/stable way to get this. Admittedly, it does imply that the committer will always have some output path, but FileOutputFormat depends on that anyway.

      Attachments

        Issue Links

          Is contained by

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-13786 Add S3A committers for zero-rename commits to S3 endpoints

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6823 FileOutputFormat to support configurable PathOutputCommitter factory

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. SPARK-21549 Spark fails to complete job correctly in case of OutputFormat which do not write into hdfs

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-6956 FileOutputCommitter to gain abstract superclass PathOutputCommitter

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              stevel@apache.org Steve Loughran
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: