Pandas version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pandas.

  • [ ] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd

pd.__version__  # 2.0.0

df1 = pd.DataFrame(
    {
        "n": [11, 13, 17],
        "answers": ["A", "A", "B"],
    }
)

df1 = df1.pivot(columns="answers", values="n")

df1.loc[:, ["A", "B"]] = df1.loc[:, ["A", "B"]].fillna(0).astype("int")

# df1
# answers     A     B
# 0        11.0   0.0
# 1        13.0   0.0
# 2         0.0  17.0

df2 = pd.DataFrame({
    "A": [11.0, 13.0, pd.NA],
    "B": [pd.NA, pd.NA, 17.0]
})

df2.loc[:, ["A", "B"]] = df2.loc[:, ["A", "B"]].fillna(0).astype("int")

# df2
#     A   B
# 0  11   0
# 1  13   0
# 2   0  17

Issue Description

The expression df1.loc[:, ["A", "B"]].fillna(0).astype("int") gives the expected data with NAs filled with 0 and the dtype changed to int.

But assigning that back to df1.loc[:, ["A", "B"]] does not keep the dtype change.

I have also verified that copying df2's column into df1's columns before the fillna and astype does not change the outcome.

This is not reproducible in a dataframe created from the pivoted data (df2).

NB. This MRE is a much simplified version of the indexing/assignment I use in the code in which I found the issue, which uses pd.IndexSlice and hierarchical columns.

I am aware df1[["A", "B"]] = df1[["A", "B"]].fillna(0).astype("int") works but using .loc is necessary with pd.IndexSlice.

I am currently using a workaround similar to the above by creating an exhaustive pd.MultiIndex and using it in place of ["A", "B"] rather than using a .loc + pd.IndexSlice, happy to work on a hierarchical MRE if that's useful or necessary.

Expected Behavior

The pivoted df1 should behave like df2 and astype not be lost upon assignment.

Installed Versions

INSTALLED VERSIONS ------------------ commit : 478d340667831908b5b4bf09a2787a11a14560c9 python : 3.9.16.final.0 python-bits : 64 OS : Darwin OS-release : 22.3.0 Version : Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2 /RELEASE_X86_64 machine : x86_64 processor : i386 byteorder : little LC_ALL : en_GB.UTF-8 LANG : en_GB.UTF-8 LOCALE : en_GB.UTF-8 pandas : 2.0.0 numpy : 1.24.2 pytz : 2023.3 dateutil : 2.8.2 setuptools : 58.1.0 pip : 23.0.1 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : None IPython : 8.12.0 pandas_datareader: None bs4 : None bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : 3.1.2 pandas_gbq : None pyarrow : None pyreadstat : None pyxlsb : None s3fs : None scipy : None snappy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None zstandard : None tzdata : 2023.3 qtpy : None pyqt5 : None

Comment From: ryanyao2002

take

Comment From: phofl

Hey, thanks for your report. This is a duplicate of #52593