Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-28245

Problem to recognize Kubernetes cluster address from Scala

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.4.0
    • None
    • Kubernetes, Spark Core

    Description

      This works in PySpark,

      from pyspark.sql import SparkSession
spark = SparkSession.builder \
    .master('k8s://https://kubernetes.default') \

      But in Scala,

      val spark = SparkSession.builder() 
    .master("k8s://https://kubernetes.default")

      I get the following exception,

      Caused by: org.apache.spark.SparkException: Could not parse Master URL: 'k8s://https://kubernetes.default'

       

      Attachments

        1. image-2019-07-05-11-45-21-609.png
          100 kB
          Liu Runzhong

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-28244 Problem to recognize Kubernetes cluster address from Scala

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              albertoandreotti@gmail.com Alberto Andreotti
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: