Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
df = pd.DataFrame({"int": [1, 2, 3], "float": [1.0, 2.0, 3.0]})
filter_ = [1, 2]
print('## Good')
print(df["int"].isin([1, 2]))
print(df["float"].isin([1, 2]))
print(df["int"].isin(i for i in [1.0, 2.0]))
print('## Bad')
print(df["float"].isin(i for i in [1.0, 2.0]))
Issue Description
Passing an iterator to pd.Series.isin
does not seem to work reliably. Depending on the dtype (I checked int
and float`), the behavior differs.
The int
dtype filters as I would expect (even when passing floats) while the float dtype always returns False
.
I was running into this issue in production where all but one column were ints.
Expected Behavior
The output is always
0 True
1 True
2 False
Or at least consistent independent of the type.
Instead the last case returns
0 False
1 False
2 False
Installed Versions
Comment From: phofl
Fixed on main, but needs tests
Comment From: luke396
import pandas as pd
df = pd.DataFrame({"int": [1, 2, 3], "float": [1.0, 2.0, 3.0]})
filter_ = [1, 2]
print('## Good')
print(df["int"].isin([1, 2]))
print(df["float"].isin([1, 2]))
print(df["int"].isin(i for i in [1.0, 2.0]))
print('## Bad')
print(df["float"].isin(i for i in [1.0, 2.0]))
I try it in pandas-dev (from here), the example works well.
But in another environment, pandas == 1.5.2
it works bad, just like the issues dercribe.
Comment From: keenborder786
take
Comment From: keenborder786
I will have a look into the issue.
Comment From: keenborder786
I have not been able to find any issue in the code.
Comment From: phofl
this is already fixed, but we need a test for this
Comment From: jessica-writes-code
take