Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Invalid
-
2.4.5, 3.0.0
-
None
-
None
-
OS : Windows 10
Spark : spark-3.0.0-preview2-bin-hadoop2.7
Env_variables :
SPARK_HOME : C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7
HADOOP_HOME : C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7
JAVA_HOME : C:\Program Files\Java\jdk1.8.0_191 ; C:\Program Files\Java\jre1.8.0_241\bin
PYTHON VERSION : Python 3.7.1
ANACONDA VERSION : conda 4.8.2I am not running this pyspark code locally (no hadoop setup)- to develop on NLP script.
OS : Windows 10 Spark : spark-3.0.0-preview2-bin-hadoop2.7 Env_variables : SPARK_HOME : C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7 HADOOP_HOME : C:\Users\rohit\spark\spark-3.0.0-preview2-bin-hadoop2.7\spark-3.0.0-preview2-bin-hadoop2.7 JAVA_HOME : C:\Program Files\Java\jdk1.8.0_191 ; C:\Program Files\Java\jre1.8.0_241\bin PYTHON VERSION : Python 3.7.1 ANACONDA VERSION : conda 4.8.2 I am not running this pyspark code locally (no hadoop setup)- to develop on NLP script.
-
Important
Description
Issue :
I am trying to initialize spark context using below code :
from pyspark.sql import SparkSession
spark = SparkSession.builder.getOrCreate()
import pandas as pd
sc = spark.sparkContext
At getOrCreate() step , code is going to infinite loop and never respond back either with exception or timeout.
On execution if i check on VS Terminal or Anaconda cmd prompot - below statement is getting displayed :
[I 19:13:47.973 NotebookApp] Saving file at /LearnPython/Assignment3/NLP.ipynb
The filename, directory name, or volume label syntax is incorrect.
Note : This used to work fine till a month back - not sure what chnaged, it stopped working from yesterday.
