Pandas version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pandas.

  • [ ] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

df = pd.DataFrame(dict(a=[0,1,1,0], b=[42,43,44,45]), index=[10,20,30,40])
pivoted = df.pivot(index=None, columns='a', values='b')
print(pivoted)

Issue Description

DataFrame.pivot() documentation states:

index: str or object or a list of str, optional

    Column to use to make new frame’s index. If None, uses existing index.

https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.pivot.html

In pandas 1.5, index=None provides the expected result. In pandas 2.0.0, it raises a KeyError exception.

Expected Behavior

index=None should use the existing index as per documentation, or documentation should be modified and 2.0.0 breaking changes list updated to add that new behavior.

Installed Versions

INSTALLED VERSIONS ------------------ commit : 478d340667831908b5b4bf09a2787a11a14560c9 python : 3.8.10.final.0 python-bits : 64 OS : Linux OS-release : 5.15.0-60-generic Version : #66~20.04.1-Ubuntu SMP Wed Jan 25 09:41:30 UTC 2023 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8 pandas : 2.0.0 numpy : 1.24.2 pytz : 2023.3 dateutil : 2.8.2 setuptools : 67.6.1 pip : 23.0.1 Cython : None pytest : 7.2.2 hypothesis : None sphinx : 6.1.3 blosc : None feather : None xlsxwriter : None lxml.etree : 4.9.2 html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : 8.12.0 pandas_datareader: None bs4 : 4.12.0 bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None matplotlib : 3.7.1 numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : 11.0.0 pyreadstat : None pyxlsb : None s3fs : None scipy : 1.10.1 snappy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None zstandard : None tzdata : 2023.3 qtpy : None pyqt5 : None

Comment From: MarcoGorelli

Thanks @douglas-raillard-arm for the report

This was caused by https://github.com/pandas-dev/pandas/pull/48293 cc @phofl

https://www.kaggle.com/code/marcogorelli/pandas-regression-example?scriptVersionId=124667072

Comment From: MarcoGorelli

Looks like

df.pivot(columns='a', values='b')

gives you your expected output, so maybe the docs just need to be updated to say "Column to use to make new frame’s index. If not given, uses existing index."

Comment From: phofl

I‘d say that this behaves as intended and needs a doc update

Comment From: srkds

take

Comment From: srkds

Hi I understood the problem and tried reproducing the issue. as described by Marco and Patrick we just need a doc update.

Just to confirm we have to change the doc here if I'm not wrong can anyone help me here? I would like to contribute. https://github.com/pandas-dev/pandas/blob/3ce07cb46986ce59427b601c6878c81633e436eb/pandas/core/frame.py#L8677

Thanks!

Comment From: MarcoGorelli

yup

Comment From: srkds

yup

Thanks, Marco for confirming 👍

Comment From: douglas-raillard-arm

Please also make sure to include it in the breaking change list on pandas 2.0.0: https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html

Comment From: srkds

Please also make sure to include it in the breaking change list on pandas 2.0.0: https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html

I think it's already written here https://github.com/pandas-dev/pandas/blob/main/doc/source/whatsnew/v2.0.0.rst#reshaping

Bug Fixes / reshaping 👉 Bug in :meth:DataFrame.pivot not respecting None as column name (:issue:48293)

or should I put "Disallow passing index=None argument to DataFrame.pivot() if not given, uses existing index." under removal-of-prior-version-deprecations-changes section?

Comment From: douglas-raillard-arm

I see how it is considered a bug fix but I think for the vast majority of people using strings as column names that will be a breaking change where the working default value has changed from None to no_default