[SPARK-44644] Improve error messages for creating Python UDTFs with pickling errors - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.5.0
Fix Version/s: 4.0.0
Component/s: PySpark
Labels:
None

Target Version/s:

3.5.0

Description

Currently, when users create a Python UDTF with a non-pickleable object, it throws this error:

_pickle.PicklingError: Cannot pickle files that are not opened for reading: w

We should make this more user-friendly

Attachments

Activity

People

Assignee:: Allison Wang

Reporter:: Allison Wang

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 02/Aug/23 22:12

Updated:: 04/Aug/23 20:20

Resolved:: 04/Aug/23 20:20