Details
-
Sub-task
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
4.0.0
Description
This happens in Python 3.12 CI only.
- https://github.com/apache/spark/actions/runs/6959106836/job/18935673389
Starting test(python3.12): pyspark.streaming.tests.test_context (temp output: /__w/spark/spark/python/target/73ed28d0-ae18-426e-9760-d03bea982a9b/python3.12__pyspark.streaming.tests.test_context__l4z6a7a2.log) Traceback (most recent call last): File "<frozen runpy>", line 198, in _run_module_as_main File "<frozen runpy>", line 88, in _run_code File "/__w/spark/spark/python/pyspark/streaming/tests/test_context.py", line 23, in <module> from pyspark.testing.streamingutils import PySparkStreamingTestCase File "/__w/spark/spark/python/pyspark/testing/__init__.py", line 19, in <module> from pyspark.testing.pandasutils import assertPandasOnSparkEqual File "/__w/spark/spark/python/pyspark/testing/pandasutils.py", line 58, in <module> import pyspark.pandas as ps File "/__w/spark/spark/python/pyspark/pandas/__init__.py", line 33, in <module> require_minimum_pandas_version() File "/__w/spark/spark/python/pyspark/sql/pandas/utils.py", line 27, in require_minimum_pandas_version import pandas File "/usr/local/lib/python3.12/dist-packages/pandas/__init__.py", line 46, in <module> from pandas.core.api import ( File "/usr/local/lib/python3.12/dist-packages/pandas/core/api.py", line 1, in <module> from pandas._libs import ( File "/usr/local/lib/python3.12/dist-packages/pandas/_libs/__init__.py", line 18, in <module> from pandas._libs.interval import Interval File "interval.pyx", line 1, in init pandas._libs.interval File "hashtable.pyx", line 1, in init pandas._libs.hashtable File "missing.pyx", line 42, in init pandas._libs.missing AttributeError: partially initialized module 'pandas' has no attribute '_pandas_datetime_CAPI' (most likely due to a circular import)
Attachments
Issue Links
- links to