Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-43831 Build and Run Spark on Java 21
  3. SPARK-45143

Make PySpark compatible with PyArrow 13.0.0

    XMLWordPrintableJSON

Details

    Description

      https://github.com/apache/spark/actions/runs/6167186123/job/16738683872

       

      ======================================================================
      FAIL [0.095s]: test_from_to_pandas (pyspark.pandas.tests.data_type_ops.test_datetime_ops.DatetimeOpsTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/__w/spark/spark/python/pyspark/testing/pandasutils.py", line 122, in _assert_pandas_equal
          assert_series_equal(
        File "/usr/local/lib/python3.9/dist-packages/pandas/_testing/asserters.py", line 931, in assert_series_equal
          assert_attr_equal("dtype", left, right, obj=f"Attributes of {obj}")
        File "/usr/local/lib/python3.9/dist-packages/pandas/_testing/asserters.py", line 415, in assert_attr_equal
          raise_assert_detail(obj, msg, left_attr, right_attr)
        File "/usr/local/lib/python3.9/dist-packages/pandas/_testing/asserters.py", line 599, in raise_assert_detail
          raise AssertionError(msg)
      AssertionError: Attributes of Series are different
      
      Attribute "dtype" are different
      [left]:  datetime64[ns]
      [right]: datetime64[us] 

      Attachments

        Issue Links

          Activity

            People

              podongfeng Ruifeng Zheng
              podongfeng Ruifeng Zheng
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: