[C++][Python] installing pyarrow breaks the conda environment

I am working with pyarrow for parquet and feather IO. My environment has the following requirements python=3.8, pandas, geopandas, pyarrow, matplotlib, seaborn, jupyter

You can reproduce the env using

$ conda create -n testpyarrow python=3.8 pandas geopandas matplotlib seaborn jupyter notebook pyarrow 

$ conda activate testpyarrow

you can skip pyarrow in the above command and then install it separately from conda-forge as instructed in the pyarrow github (doesn't make a difference) page using

$ conda install -c conda-forge pyarrow

After the env is created and activated, opening a python terminal and try importing to import anything

$ python
>>>import pandas

yields

Illegal instruction (core dumped)

If you uninstall pyarrow, all the imports start working. So I have zeroed down that pyarrow is breaking the imports.

As it is core dumped I can't trace the issue further. Illegal instruction is raised irrespective of the library I try to import  (i.e. seaborn, matplotlib, geopandas, pandas, etc)