[SPARK-46962] Implement python worker to run python streaming data source - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 4.0.0
Fix Version/s: 4.0.0
Component/s: Structured Streaming
Labels:
- pull-request-available

Epic Link:
unlabelled-SPARK-46866

Description

Implement python worker to run python streaming data source and communicate with JVM through socket. Create a PythonMicrobatchStream to invoke RPC function call

Attachments

Issue Links

links to

GitHub Pull Request #45023

Activity

People

Assignee:: Chaoqin Li

Reporter:: Chaoqin Li

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 02/Feb/24 18:40

Updated:: 12/Mar/24 03:35

Resolved:: 12/Mar/24 03:35