Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19894

remote spark driver can use ip to connect Hive Client

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Spark
    • None

    Description

      There is no DNS in some real cluster network.

      When we create a HiveClient in a Gateway machine, and submit a sql for HiveonSpark, then
      1. the HiveClient will spark-submit a RemoteDriver class to YARN cluster, with args --remote-host which value is the Gateway hostname
      2. the RemoteDriver which run on one slave of the YARN cluster, and it will connect to Gateway use the value of arg(--remote-host)
      3. if the slave do not resolve the Gateway hostname(no DNS),it will connect failed

      So, we can use ip of the Gateway for the --remote-host , and the slave can use ip to connect Hive Client on the Gateway

      Attachments

        1. HIVE-19894.1.patch
          3 kB
          Song Jun

        Activity

          People

            windpiger Song Jun
            windpiger Song Jun
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: