Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
1.5.1
-
None
-
None
Description
When setting spark.executor.memory too high, queries starts to return wrong result silently.
My setting: 48GB ram servers
When setting spark.executor.memory to 32G, any select on text column on parquet files, truncates the 1st 8 chars.
When setting it to 30G, the problem goes away.
Attachments
Issue Links
- duplicates
-
SPARK-10914 UnsafeRow serialization breaks when two machines have different Oops size
- Resolved