Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
3.5.0, 4.0.0
-
None
-
None
Description
Enabling the JSON data source causes a test case to fail:
[info] - SPARK-39557 INSERT INTO statements with tables with map defaults *** FAILED *** (1 second, 498 milliseconds) [info] Results do not match for query: [info] Timezone: sun.util.calendar.ZoneInfo[id="America/Los_Angeles",offset=-28800000,dstSavings=3600000,useDaylight=true,transitions=185,lastRule=java.util.SimpleTimeZone[id=America/Los_Angeles,offset=-28800000,dstSavings=3600000,useDaylight=true,startYear=0,startMode=3,startMonth=2,startDay=8,startDayOfWeek=1,startTime=7200000,startTimeMode=0,endMode=3,endMonth=10,endDay=1,endDayOfWeek=1,endTime=7200000,endTimeMode=0]] [info] Timezone Env: [info] [info] == Parsed Logical Plan == [info] 'UnresolvedRelation [t], [], false [info] [info] == Analyzed Logical Plan == [info] i: int, s: struct<x:array<struct<a:int,b:int>>,y:array<map<string,boolean>>>, t: array<map<string,boolean>> [info] SubqueryAlias spark_catalog.default.t [info] +- Relation spark_catalog.default.t[i#13929,s#13930,t#13931] json [info] [info] == Optimized Logical Plan == [info] Relation spark_catalog.default.t[i#13929,s#13930,t#13931] json [info] [info] == Physical Plan == [info] FileScan json spark_catalog.default.t[i#13929,s#13930,t#13931] Batched: false, DataFilters: [], Format: JSON, Location: InMemoryFileIndex(1 paths)[file:/home/mark.jarvin/photon/spark/sql/core/spark-warehouse/org.apach..., PartitionFilters: [], PushedFilters: [], ReadSchema: struct<i:int,s:struct<x:array<struct<a:int,b:int>>,y:array<map<string,boolean>>>,t:array<map<stri... [info] [info] == Results == [info] [info] == Results == [info] !== Correct Answer - 4 == == Spark Answer - 4 == [info] !struct<> struct<i:int,s:struct<x:array<struct<a:int,b:int>>,y:array<map<string,boolean>>>,t:array<map<string,boolean>>> [info] ![1,[List([1,2]),List(Map(def -> false, jkl -> true))],List(Map(xyz -> true))] [1,[ArraySeq([1,2]),ArraySeq(Map(def -> false, jkl -> true))],ArraySeq(Map(xyz -> true))] [info] ![2,null,List(Map(xyz -> true))] [2,[ArraySeq([1,2]),ArraySeq(Map(def -> false, jkl -> true))],ArraySeq(Map(xyz -> true))] [info] ![3,[List([3,4]),List(Map(mno -> false, pqr -> true))],List(Map(xyz -> true))] [3,[ArraySeq([3,4]),ArraySeq(Map(mno -> false, pqr -> true))],ArraySeq(Map(xyz -> true))] [info] ![4,[List([3,4]),List(Map(mno -> false, pqr -> true))],List(Map(xyz -> true))] [4,[ArraySeq([3,4]),ArraySeq(Map(mno -> false, pqr -> true))],ArraySeq(Map(xyz -> true))] (QueryTest.scala:267)