Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15887

Bring back the hive-site.xml support for Spark 2.0

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.0.0
    • SQL
    • None

    Description

      Right now, Spark 2.0 does not load hive-site.xml. Based on users' feedback, it seems make sense to still load this conf file.
      Originally, this file was loaded when we load HiveConf class and all settings can be retrieved after we create a HiveConf instances. Let's avoid of using this way to load hive-site.xml. Instead, since hive-site.xml is a normal hadoop conf file, we can first find its url using the classloader and then use Hadoop Configuration's addResource (or add hive-site.xml as a default resource through Configuration.addDefaultResource) to load confs.
      Please note that hive-site.xml needs to be loaded into the hadoop conf used to create metadataHive.

      Attachments

        Issue Links

          Activity

            People

              cloud_fan Wenchen Fan
              cloud_fan Wenchen Fan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: