Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
Description
This is a sub-FLIP for the disaggregated state management and its related work, please read the FLIP-423 first to know the whole story.
FLIP-424 introduces asynchronous state APIs with callbacks allowing state access to be executed in threads separate from the task thread, making better usage of I/O bandwidth and enhancing throughput. This FLIP proposes an execution framework for asynchronous state APIs. The execution code path for the new API is completely independent from the original one, where many runtime components are redesigned. We intend to delve into the challenges associated with asynchronous execution and provide an in-depth design analysis for each module. Furthermore, we will conduct a performance analysis of the new framework relative to the current implementation and examine how it measures up against other potential alternatives.
Attachments
Issue Links
- is a child of
-
FLINK-34984 FLIP-423: Disaggregated State Storage and Management (Umbrella FLIP)
- Open
- mentioned in
-
Page Loading...
1.
|
Basic framework of async execution for state | Resolved | Zakelly Lan | |
2.
|
Implement record order preservation and buffering of AsyncExecutionController | Resolved | Yanfei Lei | |
3.
|
Introduce async execution configurations | Resolved | Yanfei Lei | |
4.
|
Wire AsyncExecutionController to AbstractStreamOperator | Resolved | Zakelly Lan | |
5.
|
Implement checkpoint drain in AsyncExecutionController | Resolved | Yanfei Lei | |
6.
|
Timer firing under async execution model | Resolved | Yanfei Lei | |
7.
|
Wire new operators for async state with DataStream V2 | In Progress | Zakelly Lan | |
8.
|
Store timer in JVM heap when async execution enabled | In Progress | Yanfei Lei | |
9.
|
Introduce Epoch Manager for async execution | In Progress | Yanfei Lei | |
10.
|
Latency marker emitting under async execution model | Open | Unassigned | |
11.
|
Error handling in StateFuture's callback | Resolved | Yanfei Lei | |
12.
|
Bridge between AsyncKeyedStateBackend and AsyncExecutionController | Resolved | Hangxiang Yu |