Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
ghx-label-5
Description
Implement BufferedPlanRootSink so that FlushFinal blocks until all rows are fetched. The implementation should use the RowBatchQueue introduced by IMPALA-8779. By blocking in FlushFinal all non-coordinator fragments will be closed if all results fit in the RowBatchQueue. BufferedPlanRootSink::Send should enqueue each given RowBatch onto the queue and then return. If the queue is full, it should block until there is more space left in the queue. BufferedPlanRootSink::GetNext reads from the queue and then fills in the given QueryResultSet by using the DataSink ScalarExprEvaluator-s. Since the producer thread can call BufferedPlanRootSink::Close while the consumer is calling BufferedPlanRootSink::GetNext the two methods need to be synchronized so that the DataSink MemTracker-s are not closed while GetNext is running.
The implementation of BufferedPlanRootSink should remain the same regardless of whether a std::queue backed RowBatchQueue or a BufferedTupleStream backed RowBatchQueue is used.
BufferedPlanRootSink and BlockingPlanRootSink are similar in the sense that BlockingPlanRootSink buffers one RowBatch, so for queries that return under 1024 rows, all non-coordinator fragments are closed immediately as well. The advantage of BufferedPlanRootSink is that allows buffering for 1+ RowBatch-es.