Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29459

Spark takes lot of time to initialize when the network is not connected to internet.

    XMLWordPrintableJSON

Details

    • Question
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 1.5.2
    • None
    • ML
    • None

    Description

      Hi, I have faced two problems in using Spark ML when my network is not connected to the internet (airplane mode). All these issues occurred in windows 10 and java 8 environment.

      1. Spark by default searches for the internal IP for various operations. (like hosting spark web ui). There was some problem initializing spark and I have fixed this issue by setting "spark.driver.host" to "localhost".
      2. It takes around 120 secs to initialize the spark. May I know that is there any way to reduce this time. Like, If there is any param/config which related to timeout can resolve this?

      Attachments

        Activity

          People

            Unassigned Unassigned
            ramanrajarathinam Raman
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: