-
[x ] I have checked that this issue has not already been reported.
-
[ x] I have confirmed this bug exists on the latest version of pandas.
-
[ x] (optional) I have confirmed this bug exists on the master branch of pandas.
Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.
Code Sample, a copy-pastable example
import pandas
Problem description
Pandas import is failing.
Expected Output
Pandas to work.
Output of pd.show_versions()
Traceback (most recent call last):
File "/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/__init__.py", line 29, in <module>
File "/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/__init__.py", line 13, in <module>
ImportError: /me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so: invalid ELF header
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/__init__.py", line 37, in <module>
ImportError: C extension: /me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so: invalid ELF header not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --force' to build the C extensions first.Traceback (most recent call last):
File "/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/__init__.py", line 29, in <module>
File "/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/__init__.py", line 13, in <module>
ImportError: /me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so: invalid ELF header
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/__init__.py", line 37, in <module>
ImportError: C extension: /me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so: invalid ELF header not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --force' to build the C extensions first.
Comment From: phofl
Hi, which pandas version?
Comment From: symbolix
This happened with Python 3.7.9 and Pandas 1.2.2. I did not do anything special, just followed the steps:
- Create a conda environment with python 3.7.9.
- Then install Pandas into that environment.
The problem does not exist with a Python 3.8 based conda environment and Pandas 1.2.2. I can confirm that this works.
Comment From: rhshadrach
What command was used to install pandas?
Comment From: symbolix
conda install pandas
Comment From: phofl
The following works for me:
conda create --prefix /tmp/test python=3.7.9
conda activate /tmp/test
conda install pandas
python
and then
import pandas
Channel is conda-forge
Could you describe the commands to setup the env on your side if you did something differently?
Comment From: symbolix
I repeated everything @phofl did, and same problem.
conda create --prefix /tmp/test python=3.7.9
conda activate /tmp/test
conda install pandas
python
followed by
import pandas
Getting same error.
Comment From: rhshadrach
@symbolix What is the output of conda config --describe channels? I can reproduce installing from the "defaults" channel.
Comment From: rhshadrach
Use conda install -c conda-forge pandas to install from the conda-forge channel.
Comment From: symbolix
Here:
# # channels (sequence: primitive)
# # aliases: channel
# # env var string delimiter: ','
# # The list of conda channels to include for relevant operations.
# #
# channels:
# - defaults
Comment From: symbolix
Got it. I can confirm that conda-forge fixed the issue. I used the following command to install Pandas:
conda install -c conda-forge pandas
... and this worked. Thank you. I haven't seen this conda-forge before, at least in any of the initial documentation that a new user stumbles upon.
Comment From: rhshadrach
cc @simonjayhawkins - installation from defaults channel seems to be broken with python 3.7.9. Is there some action we can take here?
Also, should our docs be specifying to install from conda-forge? Note that environment.yml uses conda-forge.
Comment From: symbolix
Thank you for your quick help and for being patient with me despite the fact that I barged on the issues board with a quick and messy post. I was in a panic last night, it was getting late and my dev environment was literally falling apart because I updated some of my stack (which I should have never done so deep into my dev process). Anyways, all good now and I have learned about conda-forge after all.
Comment From: jorisvandenbossche
We can open an issue at https://github.com/ContinuumIO/anaconda-issues/issues/ to report problems with the default channel (we don't package this ourselves, so we don't have direct control over it).
I tried to replicate this, but for me it imports fine (on linux, @symbolix you are also using linux?)
One other question: @symbolix you mention that you have the problem with pandas 1.2.2, but as far as I can see, the default channel doesn't yet have that version (it is at 1.2.1).
Do you still have the broken environment? If so, can you show the output of conda list in there?
Comment From: symbolix
Yes still have it. I'm on Linux as well. Ubuntu:
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
Output of conda list is as followins:
# packages in environment at /home/me/anaconda3/envs/py37:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
blas 1.0 mkl
ca-certificates 2021.1.19 h06a4308_0
certifi 2020.12.5 py37h06a4308_0
intel-openmp 2020.2 254
ld_impl_linux-64 2.33.1 h53a641e_7
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libstdcxx-ng 9.1.0 hdf63c60_0
mkl 2020.2 256
mkl-service 2.3.0 py37he8ac12f_0
mkl_fft 1.2.0 py37h23d657b_0
mkl_random 1.1.1 py37h0573a6f_0
ncurses 6.2 he6710b0_1
numpy 1.19.2 py37h54aff64_0
numpy-base 1.19.2 py37hfa32c7d_0
openssl 1.1.1i h27cfd23_0
pandas 1.2.1 py37ha9443f7_0
pip 20.3.3 py37h06a4308_0
python 3.7.9 h7579374_0
python-dateutil 2.8.1 pyhd3eb1b0_0
pytz 2021.1 pyhd3eb1b0_0
readline 8.1 h27cfd23_0
setuptools 52.0.0 py37h06a4308_0
six 1.15.0 py37h06a4308_0
sqlite 3.33.0 h62c20be_0
tk 8.6.10 hbc83047_0
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h7b6447c_0
zlib 1.2.11 h7b6447c_3
My python signature:
Python 3.7.9 (default, Aug 31 2020, 12:42:55)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
Comment From: jorisvandenbossche
Hmm, I have:
$ conda list
# packages in environment at /tmp/test2:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
blas 1.0 mkl
ca-certificates 2021.1.19 h06a4308_0
certifi 2020.12.5 py37h06a4308_0
intel-openmp 2020.2 254
ld_impl_linux-64 2.33.1 h53a641e_7
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libstdcxx-ng 9.1.0 hdf63c60_0
mkl 2020.2 256
mkl-service 2.3.0 py37he8ac12f_0
mkl_fft 1.2.0 py37h23d657b_0
mkl_random 1.1.1 py37h0573a6f_0
ncurses 6.2 he6710b0_1
numpy 1.19.2 py37h54aff64_0
numpy-base 1.19.2 py37hfa32c7d_0
openssl 1.1.1i h27cfd23_0
pandas 1.2.1 py37ha9443f7_0
pip 20.3.3 py37h06a4308_0
python 3.7.9 h7579374_0
python-dateutil 2.8.1 pyhd3eb1b0_0
pytz 2021.1 pyhd3eb1b0_0
readline 8.1 h27cfd23_0
setuptools 52.0.0 py37h06a4308_0
six 1.15.0 py37h06a4308_0
sqlite 3.33.0 h62c20be_0
tk 8.6.10 hbc83047_0
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h7b6447c_0
zlib 1.2.11 h7b6447c_3
which looks like the same env, and see the same python signature when starting python. Only using Ubuntu 20.04
Comment From: symbolix
Not sure :( Maybe I messed up something in my Linux distro. I have noticed that it was acting up recently with some error messages when trying to update or install things with apt-get install so maybe that's it.
Comment From: symbolix
Ok, one more thing.
This is my Python 3.7.9 env that is broken [py37] (pandas installed without the conda-forge flag).
The following command:
ldd /home/me/anaconda3/envs/py37/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so
Returns this:
not a dynamic executable
And this is the Python 3.7.9 env that is NOT broken [dev37] (pandas installed with the conda-forge option).
The following command:
ldd /home/me/anaconda3/envs/dev37/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so
Returns this:
linux-vdso.so.1 (0x00007ffc6ddff000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f4766a37000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f4766646000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4766c56000)
So it seems like the dynamic lib .so file is broken (not built properly).
Comment From: rhshadrach
I mentioned above I can replicate; I'm finding this is not the same issue. My test conda env was interacting with my dev env, causing the issue. Activating the test environment more carefully, I don't get any error using python 3.7.9 and pandas 1.2.1. Also the *.so file mentioned directly above is valid.
Comment From: mroeschke
Unfortunately our current version of pandas does not support 3.7 anymore so I don't think this is actionable on our end so closing