Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-14268

Cannot convert pd.DataFrame with complex128 cells to pa.Table

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • 5.0.0
    • None
    • Python

    Description

      Example:

       

      import numpy as np
      import pandas as pd
      import pyarrow as pa
      
      data = np.array([1 + 1j])
      df = pd.DataFrame(data)
      table = pa.Table.from_pandas(df)
      print(table)
      

      Throws the following error:

      Traceback (most recent call last):
       File "/Users/Henrikh/Desktop/tmp.py", line 7, in <module>
       table = pa.Table.from_pandas(df)
       File "pyarrow/table.pxi", line 1553, in pyarrow.lib.Table.from_pandas
       File "/usr/local/lib/python3.9/site-packages/pyarrow/pandas_compat.py", line 594, in dataframe_to_arrays
       arrays = [convert_column(c, f)
       File "/usr/local/lib/python3.9/site-packages/pyarrow/pandas_compat.py", line 594, in <listcomp>
       arrays = [convert_column(c, f)
       File "/usr/local/lib/python3.9/site-packages/pyarrow/pandas_compat.py", line 581, in convert_column
       raise e
       File "/usr/local/lib/python3.9/site-packages/pyarrow/pandas_compat.py", line 575, in convert_column
       result = pa.array(col, type=type_, from_pandas=True, safe=safe)
       File "pyarrow/array.pxi", line 302, in pyarrow.lib.array
       File "pyarrow/array.pxi", line 79, in pyarrow.lib._ndarray_to_array
       File "pyarrow/array.pxi", line 67, in pyarrow.lib._ndarray_to_type
       File "pyarrow/error.pxi", line 118, in pyarrow.lib.check_status
      pyarrow.lib.ArrowNotImplementedError: ('Unsupported numpy type 15', 'Conversion failed for column 0 with type complex128')

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            kantuni Henrikh Kantuni
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: