Pandas version checks
-
[x] I have checked that this issue has not already been reported.
-
[x] I have confirmed this bug exists on the latest version of pandas.
-
[x] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
df = pd.DataFrame({'a': pd.to_datetime(pd.Series(['2020-01-01']))})
df['b'] = df['a'].convert_dtypes(dtype_backend='pyarrow')
print(df['b'] < df['a'])
Issue Description
Should the above be supported? it currently raises
Expected Behavior
i think
0 False
dtype: bool[pyarrow]
Installed Versions
Comment From: abhigyan631
I think this fix can be used
import pandas as pd
df = pd.DataFrame({'a': pd.to_datetime(pd.Series(['2020-01-01']))})
df['b'] = df['a'].convert_dtypes(dtype_backend='pyarrow')
df['b'] = df['b'].astype('datetime64[ns]') # Convert 'b' back to datetime64[ns]
print(df['b'] < df['a'])
We need to convert the PyArrow-backed column (df['b']) back to a NumPy-backed datetime64[ns] type before performing the comparison to get the expected result.
Comment From: rhshadrach
I think the discussion in #60639 implies that yes, this should be supported.