Code Sample, a copy-pastable example if possible
df = pd.DataFrame(np.random.randn(3, 4), columns=list('ABCD'))
df.iloc[1, 2:4] = np.nan
df.loc[:, ['C', 'D']].fillna(-1, inplace=True)
display(df)
Output:
A B C D
0 1.387547 -1.299578 0.360015 1.290783
1 -0.395182 -0.112581 NaN NaN
2 -0.649372 -1.831869 -0.103746 0.533153
Problem description
It's expected to modify the Nan to -1 but it does NOT.
Please see the following comparisons.
Comparison (1)
On contrary, the following codes behave as expected. (The only difference is selection by iloc or by loc)
df = pd.DataFrame(np.random.randn(3, 4), columns=list('ABCD'))
df.iloc[1, 2:4] = np.nan
df.iloc[:, 2:4].fillna(-1, inplace=True)
display(df)
Output:
A B C D
0 -0.522821 -1.600520 -1.468871 0.715790
1 0.493071 0.722474 -1.000000 -1.000000
2 0.545852 -0.877946 0.993169 -0.582661
Comparison (2)
When only one column is selected with loc, it behaves properly.
df = pd.DataFrame(np.random.randn(3, 4), columns=list('ABCD'))
df.iloc[1, 2:4] = np.nan
df.loc[:, 'C'].fillna(-1, inplace=True)
display(df)
Output:
A B C D
0 -0.549106 0.261093 -1.278554 2.017178
1 -1.424498 0.439482 -1.000000 NaN
2 -1.281520 1.190736 0.356319 0.416363
Expected Output of the first code sample
A B C D
0 1.181106 1.101231 -0.198445 0.295238
1 -0.654265 -1.129840 -1.000000 -1.000000
2 -1.070404 0.096556 0.499020 -1.835347
Output of pd.show_versions()
Comment From: jreback
you are filling a copy. Using inplace
is an anti-pattern. Most operations will show a SettingWithCopyWarning
, but in this case this is a not easily detectable.
Use
In [11]: df[['C', 'D']] = df[['C', 'D']].fillna(-1)
In [12]: df
Out[12]:
A B C D
0 0.236782 1.408896 -0.199882 0.803165
1 -1.763881 0.232414 -1.000000 -1.000000
2 0.878515 -0.394800 0.429696 -1.829569
Comment From: matheushrd
Try this: df.loc[:, ['C', 'D']] = df.loc[:, ['C', 'D']].fillna(-1) I was having the same difficulty with a .relplace in my code. This worked.
Comment From: shuiyuejihua
not only multiple columns, but also one column.
df.loc[df.id==123, 'num'].fillna(0, inplace=True)
don't work ,
but
df.loc[df.id==123, 'num'] = 123
it works
why not edit the fillna function to adapt it in the future. It seems like a bug.