Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-7857

[Python] Failing test with pandas master for extension type conversion

    XMLWordPrintableJSON

Details

    Description

      The pandas master test build has one failure

      _______________ test_conversion_extensiontype_to_extensionarray ________________
      
      monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fcd6c580bd0>
      
          def test_conversion_extensiontype_to_extensionarray(monkeypatch):
              # converting extension type to linked pandas ExtensionDtype/Array
              import pandas.core.internals as _int
          
              storage = pa.array([1, 2, 3, 4], pa.int64())
              arr = pa.ExtensionArray.from_storage(MyCustomIntegerType(), storage)
              table = pa.table({'a': arr})
          
              if LooseVersion(pd.__version__) < "0.26.0.dev":
                  # ensure pandas Int64Dtype has the protocol method (for older pandas)
                  monkeypatch.setattr(
                      pd.Int64Dtype, '__from_arrow__', _Int64Dtype__from_arrow__,
                      raising=False)
          
              # extension type points to Int64Dtype, which knows how to create a
              # pandas ExtensionArray
      >       result = table.to_pandas()
      
      opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/tests/test_pandas.py:3560: 
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      pyarrow/ipc.pxi:559: in pyarrow.lib.read_message
          ???
      pyarrow/table.pxi:1369: in pyarrow.lib.Table._to_pandas
          ???
      opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:764: in table_to_blockmanager
          blocks = _table_to_blocks(options, table, categories, ext_columns_dtypes)
      opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:1102: in _table_to_blocks
          for item in result]
      opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:1102: in <listcomp>
          for item in result]
      opt/conda/envs/arrow/lib/python3.7/site-packages/pyarrow/pandas_compat.py:723: in _reconstruct_block
          pd_ext_arr = pandas_dtype.__from_arrow__(arr)
      opt/conda/envs/arrow/lib/python3.7/site-packages/pandas/core/arrays/integer.py:108: in __from_arrow__
          array = array.cast(pyarrow_type)
      pyarrow/table.pxi:240: in pyarrow.lib.ChunkedArray.cast
          ???
      _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
      
      >   ???
      E   pyarrow.lib.ArrowNotImplementedError: No cast implemented from extension<arrow.py_extension_type> to int64
      

      Attachments

        Issue Links

          Activity

            People

              jorisvandenbossche Joris Van den Bossche
              jorisvandenbossche Joris Van den Bossche
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 50m
                  50m