[SPARK-47824] Nondeterminism in pyspark.pandas.series.asof - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.4.2, 4.0.0, 3.5.1, 3.3.4
Fix Version/s: 4.0.0, 3.5.2, 3.4.3
Component/s: PySpark
Labels:
- pull-request-available

Description

`max_by` in `pyspark.pandas.series.asof` uses a literal string instead of a generated column as its ordering condition, resulting in nondeterminism.

Attachments

Issue Links

links to

GitHub Pull Request #46018

Activity

People

Assignee:: Mark Jarvin

Reporter:: Mark Jarvin

Votes:: 0 Vote for this issue

Watchers:: 2 Start watching this issue

Dates

Created:: 11/Apr/24 23:18

Updated:: 12/Apr/24 00:38

Resolved:: 12/Apr/24 00:38