[SPARK-47329] Persist df while using foreachbatch and stateful streaming query to prevent state from being re-loaded in each batch - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 4.0.0
Fix Version/s: 4.0.0
Component/s: Structured Streaming
Labels:
- pull-request-available

Description

Persist df while using foreachbatch and stateful streaming query to prevent state from being re-loaded in each batch

Attachments

Issue Links

links to

GitHub Pull Request #45432

Activity

People

Assignee:: Anish Shrigondekar

Reporter:: Anish Shrigondekar

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 08/Mar/24 18:25

Updated:: 19/Mar/24 02:15

Resolved:: 19/Mar/24 02:15