Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17718 Hive on Spark Debugging Improvements
  3. HIVE-19910

hive.spark.log.dir isn't honored for TestSparkCliDriver

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Spark
    • None

    Description

      I haven't actually confirmed this, but I don't think hive.spark.log.dir is getting honored for any Spark test that sets spark.master to local-cluster because it adds hive.spark.log.dir as a system property via the spark.driver.extraJavaOptions configuration, but according to the Spark docs passing system properties via this parameter doesn't work in client mode, users have to use --driver-java-options instead.

      Attachments

        Activity

          People

            Unassigned Unassigned
            stakiar Sahil Takiar
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: