Code Sample, a copy-pastable example if possible
import pandas as pd
from collections import OrderedDict
data = OrderedDict((i, pd.Series([0], dtype=bool)) for i in range(20))
data[20] = pd.Series([0], dtype='>i2')
x = pd.DataFrame(data)
repr(x)
Problem description
When run this fails with:
Traceback (most recent call last):
File "blah.py", line 9, in <module>
repr(x)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/base.py", line 72, in __repr__
return str(self)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/base.py", line 51, in __str__
return self.__unicode__()
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/frame.py", line 586, in __unicode__
line_width=width, show_dimensions=show_dimensions)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/frame.py", line 1528, in to_string
show_dimensions=show_dimensions)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/io/formats/format.py", line 407, in __init__
self._chk_truncate()
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/io/formats/format.py", line 468, in _chk_truncate
frame.iloc[:, -col_num:]), axis=1)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/indexing.py", line 1325, in __getitem__
return self._getitem_tuple(key)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/indexing.py", line 1678, in _getitem_tuple
retval = getattr(retval, self.name)._getitem_axis(key, axis=axis)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/indexing.py", line 1724, in _getitem_axis
return self._get_slice_axis(key, axis=axis)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/indexing.py", line 1697, in _get_slice_axis
return self._slice(slice_obj, axis=axis, kind='iloc')
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/indexing.py", line 142, in _slice
return self.obj._slice(obj, axis=axis, kind=kind)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/generic.py", line 1749, in _slice
result = self._constructor(self._data.get_slice(slobj, axis=axis))
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/internals.py", line 3389, in get_slice
new_blocks = self._slice_take_blocks_ax0(slobj)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/internals.py", line 3973, in _slice_take_blocks_ax0
fill_tuple=None))
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/internals.py", line 1042, in take_nd
allow_fill=False)
File "/home/david/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pandas/core/algorithms.py", line 1471, in take_nd
func(arr, indexer, out, fill_value)
File "pandas/_libs/algos_take_helper.pxi", line 1769, in pandas._libs.algos.take_2d_axis1_int16_int16 (pandas/_libs/algos.c:87069)
ValueError: Big-endian buffer not supported on little-endian compiler
The number (20) of initial columns seems to be significant: It doesn't fail if it's 19. The order does not appear to be significant.
The problem appears to be specific to repr. I can create a data frame with these column dtypes and it appears to work normally.
Expected Output
I would expect this to give no output and not crash. I would also find it reasonable to reject the DataFrame at point of construction if this is in some way an unsupported use case, but crashing only at the point of calculating the repr is very surprising.
Output of pd.show_versions()
Comment From: chris-b1
docs here if you haven't seen them, we don't support a non-native byteorder. The reason it comes up with more columns is the truncated representation requires an indexing operation, which is what is failing. https://pandas.pydata.org/pandas-docs/stable/gotchas.html#byte-ordering-issues
I agree it would be much better to check on construction, open issue about that #4737, PR welcome!