Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Not A Problem
-
None
-
None
-
None
Description
The following is the piece of code I am trying to run but it fails with the following error message
The version of pyarrow is 5.0.0. How do I fix this?
//ArrowInvalid Traceback (most recent call last) <ipython-input-38-9c279286c928> in <module> 1 f = '../data/wikidata-20220926-all-ichunk_0.json' 2 stream = pa.memory_map(f) ----> 3 opened_stream = pa.ipc.open_stream(stream) 4 table = opened_stream.read_all()~/anaconda3/lib/python3.8/site-packages/pyarrow/ipc.py in open_stream(source) 152 reader : RecordBatchStreamReader 153 """ --> 154 return RecordBatchStreamReader(source) 155 156 ~/anaconda3/lib/python3.8/site-packages/pyarrow/ipc.py in __init__(self, source) 43 44 def __init__(self, source): ---> 45 self._open(source) 46 47 ~/anaconda3/lib/python3.8/site-packages/pyarrow/ipc.pxi in pyarrow.lib._RecordBatchStreamReader._open()~/anaconda3/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.pyarrow_internal_check_status()~/anaconda3/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()ArrowInvalid: Expected to read 578488923 metadata bytes, but only read 374478920
f = '../data/wikidata-20220926-all-ichunk_0.json'
stream = pa.memory_map(f)
opened_stream = pa.ipc.open_stream(stream)
table = opened_stream.read_all()