Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-17718

Hive on Spark Debugging Improvements

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • Spark
    • None

    Description

      There are multiple places where it is hard to debug HoS - e.g. the HoS Remote Driver and Client, the Spark RDD graph, etc.

      Attachments

        Issue Links

          is related to

          Improvement - An improvement or enhancement to an existing feature or task. HIVE-18533 Add option to use InProcessLauncher to submit spark jobs

          • Major - Major loss of function.
          • Closed
          1.
          		 Explain plan should show if a Map/Reduce Work is being cached Sub-task Open liyunzhang
          2.
          		 Custom Hive on Spark Tab in Spark Web UI Sub-task Open Unassigned
          3.
          		 Race condition in RemoteSparkJobMonitor Sub-task Open Sahil Takiar
          4.
          		 Hive logs in Spark Executor and Driver should show thread-id. Sub-task Open Unassigned
          5.
          		 Improve SparkTask OOM Error Parsing Logic Sub-task Open Unassigned
          6.
          		 SparkClientImpl should react to errors sent from the RemoteDriver Sub-task Open Unassigned
          7.
          		 Better console logging for lifecycle of a Spark job Sub-task Open Sahil Takiar
          8.
          		 Add units to displayed Spark metrics Sub-task Open Unassigned
          9.
          		 Organize Spark metrics into multiple groups Sub-task Open Unassigned
          10.
          		 Create Docker env for running HoS locally Sub-task Open Aihua Xu
          11.
          		 Race condition when timeout task is invoked during SASL negotation Sub-task In Progress Aihua Xu
          12.
          		 hive.spark.log.dir isn't honored for TestSparkCliDriver Sub-task Open Unassigned
          13.
          		 NPE in SparkTask#printConsoleMetrics Sub-task Open Unassigned
          14.
          		 Propagate ExecutionExceptions from the driver thread to the client Sub-task Open Unassigned
          15.
          		 Re-add HIVE-19787: Log message when spark-submit has completed Sub-task Open Sahil Takiar
          16.
          		 Typo in MetricsCollection for OutputMetrics Sub-task Patch Available Adesh Kumar Rao
          17.
          		 Improve logging when HoS Driver is killed due to exceeding memory limits Sub-task Open Unassigned
          18.
          		 JobResultSerializer uses wrong registration id in KyroMessageCodec Sub-task Open Sahil Takiar
          19.
          		 Remove 30m min value for hive.spark.session.timeout Sub-task Patch Available Unassigned
          20.
          		 SparkSession should be able to close a session while it is being opened Sub-task Open Antal Sinkovits
          21.
          		 Parse Spark error blacklist errors Sub-task Open Unassigned

          Activity

            People

              Unassigned Unassigned
              stakiar Sahil Takiar
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: