Pandas DataFrame slice of dtype np.float64 returns all zeros when using .loc for row/column selection

Hi all,

I noticed this bug today when I was trying to cast a large table (~56,000 rows by 8500 columns) from np.float64 to np.float32. Since not every column in the table is made of float.64s I first selected columns that were.

Code Sample, a copy-pastable example if possible

float_cols = curtable.dtypes[curtable.dtypes=="float64"]

The output then looked something like this:

float_cols

['GTEX-1117F-0426-SM-5EGHI',
 'GTEX-1117F-0626-SM-5N9CS',
 'GTEX-1117F-0726-SM-5GIEN',
 'GTEX-1117F-1326-SM-5EGHH',
 'GTEX-1117F-2426-SM-5EGGH',
 'GTEX-1117F-2526-SM-5GZY6',
 'GTEX-1117F-2826-SM-5GZXL',
 'GTEX-1117F-2926-SM-5GZYI',
 'GTEX-1117F-3026-SM-5GZYU',
 'GTEX-1117F-3226-SM-5N9CT',
 'GTEX-111CU-0126-SM-5GZWZ',
 'GTEX-111CU-0226-SM-5GZXC',
 'GTEX-111CU-0326-SM-5GZXO',
 'GTEX-111CU-0526-SM-5EGHK',
 'GTEX-111CU-0626-SM-5EGHL',
 'GTEX-111CU-0726-SM-5GZYD',
 'GTEX-111CU-0826-SM-5EGIJ',
 'GTEX-111CU-0926-SM-5EGIK',
 'GTEX-111CU-1026-SM-5EGIL',
 'GTEX-111CU-1126-SM-5EGIM',
 'GTEX-111CU-1226-SM-5EGIN',
 'GTEX-111CU-1326-SM-5NQ8L',
 'GTEX-111CU-1426-SM-5GZYP',
 'GTEX-111CU-1526-SM-5N9FS',
 'GTEX-111CU-1726-SM-5EGHM',
 'GTEX-111CU-1826-SM-5GZYN',
 'GTEX-111CU-1926-SM-5GZYZ',
 'GTEX-111CU-2026-SM-5GZZC',
 'GTEX-111CU-2226-SM-5N9G5',
 'GTEX-111FC-0126-SM-5N9DL',
 'GTEX-111FC-0226-SM-5N9B8',
 'GTEX-111FC-0326-SM-5GZZ1',
 'GTEX-111FC-0426-SM-5N9CV',
 'GTEX-111FC-0526-SM-5GZZ8',
 'GTEX-111FC-0626-SM-5N9CU',
 'GTEX-111FC-0826-SM-5GZWO',
 'GTEX-111FC-1026-SM-5GZX1',
 'GTEX-111FC-1126-SM-5GZWU',
 'GTEX-111FC-1926-SM-5GZYC',
 'GTEX-111FC-2026-SM-5GZYO',
 'GTEX-111FC-2926-SM-5GZY7',
 'GTEX-111FC-3126-SM-5GZZ2',
 'GTEX-111FC-3326-SM-5GZYV',
 'GTEX-111VG-0008-SM-5Q5BG',
 'GTEX-111VG-0326-SM-5GZX7',
 'GTEX-111VG-0526-SM-5N9BW',
 'GTEX-111VG-0726-SM-5GIDC',
 'GTEX-111VG-1626-SM-5EGIO',
 'GTEX-111VG-1926-SM-5GIDO',
 'GTEX-111VG-2226-SM-5N9DU',
 'GTEX-111VG-2326-SM-5N9BK',
 'GTEX-111VG-2426-SM-5GZXD',
 'GTEX-111VG-2526-SM-5GZXP',
 'GTEX-111VG-2626-SM-5GZY2',
 'GTEX-111YS-0006-SM-5NQBE',
 'GTEX-111YS-0008-SM-5Q5BH',
 'GTEX-111YS-0126-SM-5987T',
 'GTEX-111YS-0326-SM-5GZZ3',
 'GTEX-111YS-0426-SM-5987O',
 'GTEX-111YS-0526-SM-5GZXJ',
 'GTEX-111YS-0626-SM-5GZXV',
 'GTEX-111YS-0726-SM-5GZY8',
 'GTEX-111YS-0826-SM-5GZYK',
 'GTEX-111YS-0926-SM-5EGGI',
 'GTEX-111YS-1026-SM-5GZYE',
 'GTEX-111YS-1126-SM-5GZYQ',
 'GTEX-111YS-1226-SM-5EGGJ',
 'GTEX-111YS-1326-SM-5EGGK',
 'GTEX-111YS-1426-SM-5GID8',
 'GTEX-111YS-1526-SM-5GZYW',
 'GTEX-111YS-1626-SM-5GZZ9',
 'GTEX-111YS-1726-SM-5GIED',
 'GTEX-111YS-1826-SM-5GIEP',
 'GTEX-111YS-1926-SM-5GICC',
 'GTEX-111YS-2026-SM-5EGGL',
 'GTEX-111YS-2126-SM-5EGGM',
 'GTEX-111YS-2226-SM-5987P',
 'GTEX-111YS-2326-SM-5987L',
 'GTEX-111YS-2426-SM-5GZZQ',
 'GTEX-1122O-0003-SM-5Q5DL',
 'GTEX-1122O-0005-SM-5O99J',
 'GTEX-1122O-0008-SM-5QGR2',
 'GTEX-1122O-0126-SM-5GICA',
 'GTEX-1122O-0226-SM-5N9DA',
 'GTEX-1122O-0326-SM-5H124',
 'GTEX-1122O-0426-SM-5H12G',
 'GTEX-1122O-0526-SM-5N9DM',
 'GTEX-1122O-0626-SM-5N9B9',
 'GTEX-1122O-0726-SM-5GIEV',
 'GTEX-1122O-0826-SM-5GICV',
 'GTEX-1122O-0926-SM-5N9C9',
 'GTEX-1122O-1126-SM-5NQ8X',
 'GTEX-1122O-1226-SM-5H113',
 'GTEX-1122O-1326-SM-5H11F',
 'GTEX-1122O-1426-SM-5H11R',
 'GTEX-1122O-1526-SM-5N9CL',
 'GTEX-1122O-1626-SM-5N9CX',
 'GTEX-1122O-1726-SM-5GICM',
 'GTEX-1122O-1826-SM-5EGIP',
 'GTEX-1122O-1926-SM-5EGIQ',
 'GTEX-1122O-2026-SM-5NQ91',
 'GTEX-1122O-2126-SM-5EGIR',
 'GTEX-1122O-2426-SM-5GIDN',
 'GTEX-1128S-0005-SM-5P9HI',
 'GTEX-1128S-0008-SM-5Q5DP',
 'GTEX-1128S-0126-SM-5H12S',
 'GTEX-1128S-0326-SM-5GZZF',
 'GTEX-1128S-0726-SM-5N9D6',
 'GTEX-1128S-0826-SM-5GZZI',
 'GTEX-1128S-0926-SM-5GZZU',
 'GTEX-1128S-1626-SM-5H12O',
 'GTEX-1128S-2126-SM-5H12U',
 'GTEX-1128S-2326-SM-5GZZY',
 'GTEX-1128S-2426-SM-5H11B',
 'GTEX-1128S-2526-SM-5H11N',
 'GTEX-1128S-2726-SM-5H12C',
 'GTEX-1128S-2826-SM-5N9DI',
 'GTEX-113IC-0006-SM-5NQ9C',
 'GTEX-113IC-0008-SM-5QGRF',
 'GTEX-113IC-0226-SM-5HL5C',
 'GTEX-113IC-0426-SM-5HL5O',
 'GTEX-113IC-0526-SM-5HL76',
 'GTEX-113JC-0006-SM-5O997',
 'GTEX-113JC-0008-SM-5QGR6',
 'GTEX-113JC-0126-SM-5EGJW',
 'GTEX-113JC-0726-SM-5GZZR',
 'GTEX-113JC-0926-SM-5H114',
 'GTEX-113JC-1026-SM-5H117',
 'GTEX-113JC-1926-SM-5N9GE',
 'GTEX-113JC-2226-SM-5EGJG',
 'GTEX-113JC-2326-SM-5EQ4E',
 'GTEX-113JC-2726-SM-5EGIS',
 'GTEX-113JC-2826-SM-5EGIT',
 'GTEX-117XS-0005-SM-5PNU6',
 'GTEX-117XS-0008-SM-5Q5DQ',
 'GTEX-117XS-0426-SM-5GZZN',
 'GTEX-117XS-0526-SM-5987Q',
 'GTEX-117XS-0726-SM-5H131',
 'GTEX-117XS-1926-SM-5GICO',
 'GTEX-117XS-2026-SM-5GID1',
 'GTEX-117XS-2226-SM-5GIE1',
 'GTEX-117XS-2526-SM-5H11G',
 'GTEX-117XS-2726-SM-5N9BL',
 'GTEX-117XS-2926-SM-5N9BX',
 'GTEX-117XS-3026-SM-5N9CA',
 'GTEX-117XS-3126-SM-5GIDP',
 'GTEX-117YW-0005-SM-5NQ8Z',
 'GTEX-117YW-0008-SM-5QGRD',
 'GTEX-117YW-0126-SM-5EGGN',
 'GTEX-117YW-0226-SM-5N9CM',
 'GTEX-117YW-0326-SM-5N9CY',
 'GTEX-117YW-0526-SM-5H11C',
 'GTEX-117YW-0826-SM-5H11O',
 'GTEX-117YW-1426-SM-5EGGO',
 'GTEX-117YW-1526-SM-5EGGP',
 'GTEX-117YW-1826-SM-5PNY5',
 'GTEX-117YW-1926-SM-5H121',
 'GTEX-117YW-2026-SM-5N9BH',
 'GTEX-117YW-2226-SM-5N9DB',
 'GTEX-117YW-2426-SM-5Q5AE',
 'GTEX-117YW-2626-SM-5GZZH',
 'GTEX-117YW-2826-SM-5H116',
 'GTEX-117YX-0126-SM-5EGH5',
 'GTEX-117YX-0226-SM-5EGH6',
 'GTEX-117YX-0326-SM-5GICL',
 'GTEX-117YX-0526-SM-5EGJH',
 'GTEX-117YX-0626-SM-5EGJI',
 'GTEX-117YX-0726-SM-5GIET',
 'GTEX-117YX-0826-SM-5H11J',
 'GTEX-117YX-1026-SM-5H11V',
 'GTEX-117YX-1126-SM-5H128',
 'GTEX-117YX-1226-SM-5H11S',
 'GTEX-117YX-1326-SM-5H125',
 'GTEX-117YX-1426-SM-5H12H',
 'GTEX-117YX-1526-SM-5H12T',
 'GTEX-117YX-1626-SM-5GZZG',
 'GTEX-117YX-1726-SM-5GZZS',
 'GTEX-117YX-1826-SM-5H12K',
 'GTEX-117YX-1926-SM-5GIEC',
 'GTEX-117YX-2126-SM-5GIEL',
 'GTEX-117YX-2226-SM-5EGJJ',
 'GTEX-117YX-2326-SM-5H12W',
 'GTEX-117YX-2426-SM-5GZZJ',
 'GTEX-117YX-2526-SM-5EQ4Q',
 'GTEX-117YX-2626-SM-5EQ53',
 'GTEX-1192W-0005-SM-5NQBQ',
 'GTEX-1192W-0008-SM-5QGRE',
 'GTEX-1192W-0126-SM-5EGGS',
 'GTEX-1192W-0226-SM-5EGGT',
 'GTEX-1192W-2626-SM-5Q5AF',
 'GTEX-1192W-3026-SM-5GZZV',
 'GTEX-1192X-0008-SM-5Q5B7',
 'GTEX-1192X-0226-SM-5H12D',
 'GTEX-1192X-0326-SM-5H11U',
 'GTEX-1192X-0426-SM-5GIEE',
 'GTEX-1192X-0726-SM-5987R',
 'GTEX-1192X-1126-SM-5EGGU',
 'GTEX-1192X-1526-SM-5H11I',
 'GTEX-1192X-1826-SM-5GIE2',
...]

However, when I use this column list to select these rows, the returned DataFrame is all zeros:

np.all(curtable.loc[:, float_cols] == 0)
True

But when I just use one column, seems like there is no issue.

np.all(curtable.loc[:, float_cols[0]] == 0)
False

Problem description

My understanding is that the .loc indexer can be supplied with a list of values from the index/columns to select those cells within a table. Perhaps this is an issue when working with particularly large tables (in my case, 56000 by 8500)

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.5.2.final.0 python-bits: 64 OS: Darwin OS-release: 15.6.0 machine: x86_64 processor: i386 byteorder: little LC_ALL: None LANG: en_US.UTF-8 LOCALE: en_US.UTF-8 pandas: 0.19.2 nose: None pip: 9.0.1 setuptools: 27.2.0 Cython: None numpy: 1.11.2 scipy: 0.18.1 statsmodels: None xarray: None IPython: 5.1.0 sphinx: 1.5.1 patsy: None dateutil: 2.6.0 pytz: 2016.10 blosc: None bottleneck: None tables: 3.3.0 numexpr: 2.6.1 matplotlib: 2.0.0 openpyxl: None xlrd: 1.0.0 xlwt: 1.2.0 xlsxwriter: 0.9.6 lxml: None bs4: None html5lib: None httplib2: None apiclient: None sqlalchemy: None pymysql: None psycopg2: None jinja2: 2.9.4 boto: None pandas_datareader: None

Comment From: jreback

can you show a repro example. not even sure what the issue is.

Comment From: jreback

pls comment if you have a repo