Code Sample, a copy-pastable example if possible
import pandas as pd
import numpy as np
print('pandas version: ', pd.__version__)
df = pd.DataFrame([np.float64(123.0)])
val = df.iloc[0, 0]
print('value/type in df: {} / {}'.format(val, type(val)))
for i, row in df.itertuples():
val = row
print('value/type in row: {} / {}'.format(val, type(val)))
result:
pandas version: 0.21.0
value/type in df: 123.0 / <class 'numpy.float64'>
value/type in row: 123.0 / <class 'float'>
Problem description
The itertuples method of DataFrame objects converts dtypes of columns without notification. Here: numpy.float64 to float.
This unexpected behaviour did not occur in pandas version 0.20.3.
Output of pd.show_versions()
INSTALLED VERSIONS
------------------
commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Darwin
OS-release: 15.6.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: en_US.UTF-8
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.21.0
pytest: 2.9.2
pip: 9.0.1
setuptools: 32.3.1
Cython: 0.24
numpy: 1.13.3
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 5.1.0
sphinx: 1.4.1
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2016.4
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.4
feather: None
matplotlib: 2.1.0
openpyxl: 2.4.1
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.2
lxml: 3.6.0
bs4: 4.4.1
html5lib: None
sqlalchemy: 1.1.14
pymysql: 0.7.9.None
psycopg2: 2.7.3.2 (dt dec pq3 ext lo64)
jinja2: 2.9.4
s3fs: 0.0.8
fastparquet: 0.1.3
pandas_gbq: None
pandas_datareader: None
Process finished with exit code 0
Comment From: jreback
what you want is actually not very common. the internal repr is always hidden, but was actually leaking thru in prior versions of pandas. This change is here: http://pandas.pydata.org/pandas-docs/stable/whatsnew.html#iteration-of-series-index-will-now-return-python-scalars
so this is as expected
Comment From: jreback
coercion of selected scalars is another issue (which is open)
Comment From: jdoepfert
thx for clarification @jreback