Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-21514

Hive has updated with new support for S3 and InsertIntoHiveTable.scala should update also

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.2.0
    • None
    • SQL

    Description

      Hive has updated adding new parameters to optimize the usage of S3, now you can avoid the usage of S3 as the stagingdir using the parameters hive.blobstore.supported.schemes & hive.blobstore.optimizations.enabled.

      The InsertIntoHiveTable.scala file should be updated with the same improvement to match the behavior of Hive.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HIVE-15121 Last MR job in Hive should be able to write to a different scratch directory

          • Major - Major loss of function.
          • Reopened

          Sub-task - The sub-task of the issue HIVE-14270 Write temporary data to HDFS when doing inserts on tables located on S3

          • Major - Major loss of function.
          • Resolved
          requires

          Bug - A problem which impairs or prevents the functions of the product. HIVE-7476 CTAS does not work properly for s3

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #27690

          Activity

            People

              Unassigned Unassigned
              javieros Javier Ros
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: