[IMPALA-7350] More accurate memory estimates for admission - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: Impala 3.4.0
Component/s: Frontend
Labels:
- admission-control
- resource-management

Epic Link:
Memory-based admission control policies
Target Version:

Impala 3.4.0
Epic Color:
ghx-label-7

Description

For ~~IMPALA-7349~~, we will be relying more on memory estimates. This is an umbrella JIRA to track improvements to memory estimates where the current estimates are way off and result in over- or under- admission. over-admission is probably the more significant concern.

Attachments

Issue Links

is related to

IMPALA-10377 Improve the accuracy of resource estimation

Resolved

Sub-Tasks

1.	Add memory estimates for plan nodes and sinks with missing estimates	Resolved	Bikramjeet Vig
2.	HdfsTableSink doesn't take into account insert clustering	Resolved	Pooja Nilangekar
3.	Fix bogus too-high memory estimates	Resolved	Bikramjeet Vig
4.	Validate memory estimates for standard workloads in planner tests	Resolved	Tim Armstrong
5.	Ensure planner test coverage for memory estimates of all PlanNodes and DataSinks	Resolved	Tim Armstrong
6.	Merge aggregation node memory estimate is incorrectly influenced by limit	Resolved	Pooja Nilangekar
7.	Aggregation Node memory estimates don't account for number of fragment instances	Resolved	Pooja Nilangekar
8.	AggregationNode's memory estimate should be based on NDV only for non-grouping aggs	Resolved	Unassigned

Activity

People

Assignee:: Bikramjeet Vig

Reporter:: Tim Armstrong

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 25/Jul/18 22:43

Updated:: 04/Dec/20 16:20

Resolved:: 23/Sep/19 21:29