Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
0.9.0
Description
Using from_pandas to convert decimals fails if encounters a value of None. For example:
In [1]: import pyarrow as pa ...: import pandas as pd ...: from decimal import Decimal ...: In [2]: s_dec = pd.Series([Decimal('3.14'), None]) In [3]: pa.Array.from_pandas(s_dec, type=pa.decimal128(3, 2)) --------------------------------------------------------------------------- ArrowInvalid Traceback (most recent call last) <ipython-input-3-2da56007a0da> in <module>() ----> 1 pa.Array.from_pandas(s_dec, type=pa.decimal128(3, 2)) array.pxi in pyarrow.lib.Array.from_pandas() array.pxi in pyarrow.lib.array() error.pxi in pyarrow.lib.check_status() error.pxi in pyarrow.lib.check_status() ArrowInvalid: Error converting from Python objects to Decimal: Got Python object of type NoneType but can only handle these types: decimal.Decimal
The above error is raised when specifying decimal type. When no type is specified, a seg fault happens.
This previously worked in 0.8.0.
Attachments
Issue Links
- blocks
-
SPARK-23874 Upgrade apache/arrow to 0.10.0
- Closed
- links to