Details
-
New Feature
-
Status: Closed
-
Minor
-
Resolution: Duplicate
-
2.0.1
-
None
-
None
Description
Currently, with DataFrame API, we can't load standard json file directly, maybe we can provide an override method to process this, the logic is as below:
```
val df = spark.sparkContext.wholeTextFiles("data/test.json")
val json_rdd = df.map( x => x.toString.replaceAll("
s+","")).map
val json_df = spark.read.json(json_rdd)
```
Attachments
Issue Links
- duplicates
-
SPARK-18352 Parse normal, multi-line JSON files (not just JSON Lines)
- Resolved
- links to