For example this chunk of unit test code from pandas.tests.frame.test_indexing.py test_where_datetime
from datetime import datetime
from pandas import DataFrame, date_range
import numpy as np
from pandas.util.testing import assert_frame_equal
df = DataFrame(dict(A=date_range('20130102', periods=5),
B=date_range('20130104', periods=5),
C=np.random.randn(5)))
stamp = datetime(2013, 1, 3)
result = df[df > stamp]
expected = df.copy()
expected.loc[[0, 1], 'A'] = np.nan
assert_frame_equal(result, expected)
produces these result & expected on arm64
>>> result
A B C
0 1970-01-01 2013-01-04 -0.238902
1 1970-01-01 2013-01-05 0.055246
2 2013-01-04 2013-01-06 0.583666
3 2013-01-05 2013-01-07 -2.401230
4 2013-01-06 2013-01-08 2.580124
>>> expected
A B C
0 NaT 2013-01-04 -0.238902
1 NaT 2013-01-05 0.055246
2 2013-01-04 2013-01-06 0.583666
3 2013-01-05 2013-01-07 -2.401230
4 2013-01-06 2013-01-08 2.580124
>>>
and these results on amd64
A B C
0 NaT 2013-01-04 -1.151324
1 NaT 2013-01-05 0.925227
2 2013-01-04 2013-01-06 -2.267937
3 2013-01-05 2013-01-07 2.129802
4 2013-01-06 2013-01-08 0.103666
In [11]: expected
Out[11]:
A B C
0 NaT 2013-01-04 -1.151324
1 NaT 2013-01-05 0.925227
2 2013-01-04 2013-01-06 -2.267937
3 2013-01-05 2013-01-07 2.129802
4 2013-01-06 2013-01-08 0.103666
The full build log showing more arm64 tests failures is available at: https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=arm64&ver=0.20.3-2&stamp=1506059703&raw=0 and the buildd status page showing all of the failures & successes: https://buildd.debian.org/status/package.php?p=pandas
Obviously it'd be nice if the tests behaved consistently independent of architecture.
Output of pd.show_versions()
pandas.show_versions() on arm64
From a working amd64 system:
dateutil is 2.6.1 in both environments. The amd64 test environment is not exactly the same as the buildd. but these tests do pass in the buildd.
Comment From: detrout
slowly poking at this:
On arm64
>>> df[df>stamp]
A B C
0 1970-01-01 2013-01-04 -0.675049
1 1970-01-01 2013-01-05 -0.242239
2 2013-01-04 2013-01-06 -0.785362
3 2013-01-05 2013-01-07 -0.207237
4 2013-01-06 2013-01-08 -0.059387
>>> type(df[df>stamp]['A'][0])
<class 'pandas._libs.tslib.Timestamp'>
on amd64
In [8]: df[df>stamp]
Out[8]:
A B C
0 NaT 2013-01-04 -0.213889
1 NaT 2013-01-05 1.045750
2 2013-01-04 2013-01-06 -0.148973
3 2013-01-05 2013-01-07 -1.205978
4 2013-01-06 2013-01-08 -0.424786
In [9]: type(df[df>stamp]['A'][0])
Out[9]: pandas.tslib.NaTType
So why NaTType vs Timestamp?
Comment From: jreback
we only support little endian platforms. Really not sure much can do here. We don't have CI support, nor development resources anything but the core platforms.
Comment From: llimeht
Worth noting, then, that arm64, armel, armhf, mips64el, and mipsel are little endian so endianness isn't the issue. Of the list of failing architectures, only mips and s390x are big endian.
arm* are a common enough architectures for which pandas has users, so they are probably worth supporting, even though the others are considerably more exotic; numpy and dateutil test suites pass ok on all of them.
Comment From: llimeht
For CI, would https://github.com/travis-ci/travis-ci/issues/3376 be enough? (pandas has a lot of tests that take a long time to run so perhaps it won't be fast enough to avoid time-outs under emulation)
Comment From: jreback
@llimeht yes that might work, didn't even know that was an option. if you were willing to do a PR to add a (allowed failures) job to travis (and you can make it minimal for the moment, only installed base dependencies and even have a very limited tests suite) would be ok to start. at least seeing issues would be good. can always add to it and/or have multiple builds.
Comment From: detrout
Getting a CI solution up seems like a good general solution, but I am willing to at least try and find what the problem is. Any suggestions would be appreciated though.
Comment From: yarikoptic
@detrout any luck with trying to get NaT problems (re)solved on ARMs? those failures hinder Debian builds so I would be really excited to get them resolved properly . FWIW those seems to be the only once holding up the build to announce clean in Debian: sample build log where you could find also tests failures/outputs: https://buildd.debian.org/status/fetch.php?pkg=pandas&arch=armel&ver=0.22.0-2&stamp=1519375446&raw=0
Comment From: yarikoptic
Ah -- another issue (still open) is https://github.com/pandas-dev/pandas/issues/11282 and points to numpy https://github.com/numpy/numpy/issues/8325 (I did feel that we already investigated it before ;-) )
