|
|
|
SPARK-31658
|
SPARK-31412
SQL UI doesn't show write commands of AQE plan
|
Manu Zhang
|
Manu Zhang
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31384
|
SPARK-31412
NPE in OptimizeSkewedJoin when there's a inputRDD of plan has 0 partition
|
wuyi
|
wuyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31253
|
SPARK-31412
add metrics to AQE shuffle reader
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31206
|
SPARK-31412
AQE will use the same SubqueryExec even if subqueryReuseEnabled=false
|
Unassigned
|
wuyi
|
|
Resolved |
Incomplete
|
|
|
|
|
|
|
|
SPARK-31201
|
SPARK-31412
add an individual config for skewed partition threshold
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31134
|
SPARK-31412
optimize skew join after shuffle partitions are coalesced
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31124
|
SPARK-31412
change the default value of minPartitionNum in AQE
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31096
|
SPARK-31412
Replace `Array` with `Seq` in AQE `CustomShuffleReaderExec`
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31070
|
SPARK-31412
make skew join split skewed partitions more evenly
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31046
|
SPARK-31412
Make more efficient and clean up AQE update UI code
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31045
|
SPARK-31412
Add config for AQE logging level
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-31037
|
SPARK-31412
refine AQE config names
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30999
|
SPARK-31412
Don't cancel a QueryStageExec when it's already finished
|
wuyi
|
wuyi
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30991
|
SPARK-31412
Refactor AQE readers and RDDs
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30922
|
SPARK-31412
Remove the max split config after changing the multi sub joins to multi sub partitions
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30918
|
SPARK-31412
improve the splitting of skewed partitions
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30906
|
SPARK-31412
Turning off AQE in CacheManager is not thread-safe
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30864
|
SPARK-31412
Add the user guide for Adaptive Query Execution
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30801
|
SPARK-31412
Subqueries should not be AQE-ed if main query is not
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30751
|
SPARK-31412
Combine the skewed readers into one in AQE skew join optimizations
|
Wenchen Fan
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30719
|
SPARK-31412
AQE should not issue a "not supported" warning for queries being by-passed
|
Wenchen Fan
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30571
|
SPARK-31412
coalesce shuffle reader with splitting shuffle fetch request fails
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30549
|
SPARK-31412
Fix the subquery metrics showing issue in UI When enable AQE
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30524
|
SPARK-31412
Disable OptimizeSkewJoin rule if introducing additional shuffle.
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30407
|
SPARK-31412
reset the metrics info of AdaptiveSparkPlanExec plan when enable aqe
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30403
|
SPARK-31412
Fix the NoSuchElementException exception when enable AQE with InSubquery use case
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30331
|
SPARK-31412
The final AdaptiveSparkPlan event is not marked with `isFinalPlan=true`
|
Manu Zhang
|
Manu Zhang
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30315
|
SPARK-31412
Add adaptive execution context
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30307
|
SPARK-31412
remove ReusedQueryStageExec
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30291
|
SPARK-31412
Catch the exception when do materialize in AQE
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-30188
|
SPARK-31412
Fix tests when enable Adaptive Query Execution
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29954
|
SPARK-31412
collect the runtime statistics of row count in map stage
|
Unassigned
|
Ke Jia
|
|
Open |
Unresolved
|
|
|
|
|
|
|
|
SPARK-29906
|
SPARK-31412
Reading of csv file fails with adaptive execution turned on
|
Wenchen Fan
|
koert kuipers
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29893
|
SPARK-31412
Improve the local reader performance by changing the task number from 1 to multi
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29759
|
SPARK-31412
LocalShuffleReaderExec.outputPartitioning should use the corrected attributes
|
Wenchen Fan
|
Wenchen Fan
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29544
|
SPARK-31412
Optimize skewed join at runtime with new Adaptive Execution
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29060
|
SPARK-31412
Add tree traversal helper for adaptive spark plans
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-29002
|
SPARK-31412
Avoid changing SMJ to BHJ if the build side has a high ratio of empty partitions
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-28753
|
SPARK-31412
Dynamically reuse subqueries in AQE
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-28739
|
SPARK-31412
Add a simple cost check for Adaptive Query Execution
|
Wei Xue
|
Wei Xue
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-28560
|
SPARK-31412
Optimize shuffle reader to local shuffle reader when smj converted to bhj in adaptive execution
|
Ke Jia
|
Ke Jia
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-28177
|
SPARK-31412
Adjust post shuffle partition number in adaptive execution
|
Carson Wang
|
Carson Wang
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-23128
|
SPARK-31412
The basic framework for the new Adaptive Query Execution
|
Carson Wang
|
Carson Wang
|
|
Resolved |
Fixed
|
|
|
|
|
|
|
|
SPARK-9853
|
SPARK-31412
Optimize shuffle fetch of contiguous partition IDs
|
Yuanjian Li
|
Matei Alexandru Zaharia
|
|
Resolved |
Fixed
|
|
|
|
|