Code Sample, a copy-pastable example if possible

data = [['Alex',10],['Bob',12],['Clarke',13]]
df = pd.DataFrame(data, columns=['_name','age'])
try:
    for row in df.itertuples():
        val = getattr(row, '_name')
        print(val)
except AttributeError:
    print('_name column not accessible')

Problem description

DataFrame columns named with a leading underscore are not accessible. While it is possible to rename the column to access the data, Pandas users unfamiliar with this issue might spend a bit of time developing their own workaround. The perplexing thing about this is when the column names are output for inspection you can see the column name with the leading underscore, e.g. print(df.dtypes.index)

[this should explain why the current behaviour is a problem and why the expected output is a better solution.]

Note: We receive a lot of issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates!

Note: Many problems can be resolved by simply upgrading pandas to the latest version. Before submitting, please check if that solution works for you. If possible, you may want to check if master addresses this issue, but that is not necessary.

For documentation-related issues, you can check the latest versions of the docs on master here:

https://pandas-docs.github.io/pandas-docs-travis/

If the issue has not been resolved there, go ahead and file it in the issue tracker.

Expected Output

The expected output is the data from the column whose name begins with an underscore.

Output of pd.show_versions()

[paste the output of ``pd.show_versions()`` here below this line] INSTALLED VERSIONS ------------------ commit: None python: 3.6.5.final.0 python-bits: 64 OS: Windows OS-release: 2012ServerR2 machine: AMD64 processor: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel byteorder: little LC_ALL: None LANG: None LOCALE: None.None pandas: 0.23.4 pytest: None pip: 10.0.1 setuptools: 39.0.1 Cython: None numpy: 1.15.3 scipy: None pyarrow: None xarray: None IPython: None sphinx: None patsy: None dateutil: 2.7.5 pytz: 2018.7 blosc: None bottleneck: None tables: None numexpr: None feather: None matplotlib: None openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml: None bs4: None html5lib: None sqlalchemy: 1.2.6 pymysql: None psycopg2: None jinja2: 2.10 s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None

Comment From: mroeschke

itertuples() returns namedtuples, so row is no longer a pandas object.

In [14]: row
Out[14]: Pandas(Index=0, _1='Alex', age=10)

These are created with rename=True, and _name is an invalid fieldname for namedtuples so:

If rename is true, invalid fieldnames are automatically replaced with positional names.

So this is the expected behavior. This could be better documented in itertuples()noting this behavior and a link to the documentation.

Comment From: gaetano-guerriero

I think it would be important to document that the silent renaming can happen elsewhere:

>>> df = pd.DataFrame({'a': [1, 2], '_b': [3, 4]})
>>> df.to_dict(orient='records')
[{'a': 1, '_1': 3}, {'a': 2, '_1': 4}]

Comment From: TomAugspurger

@gaetano-guerriero that's a bug. Doing a release today with a fix.

Comment From: mroeschke

This example looks fixed on master. Could use a test

In [16]: >>> df = pd.DataFrame({'a': [1, 2], '_b': [3, 4]})
    ...: >>> df.to_dict(orient='records')
Out[16]: [{'a': 1, '_b': 3}, {'a': 2, '_b': 4}]

Comment From: arromabofreu

take

Comment From: pranavc28

take

Comment From: jackgoldsmith4

@pranavc28 are you still working on this?

Comment From: G-J-Evans

Leading numbers followed by an underscore cause the same issue.

"1_column" and "_column" as column names will both be silently renamed. "col_column" as a column name will not be renamed