Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[X] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
In [34]: s = pd.Series([1233242342344, 232432434324, 332434242344], dtype='datetime64[ms]')
In [35]: s - np.datetime64("nat", 'ms')
Out[35]:
0 NaT
1 NaT
2 NaT
dtype: timedelta64[ns] # Can we preserve `ms` instead of `ns` here?
In [36]: s
Out[36]:
0 2009-01-29 15:19:02.344
1 1977-05-14 04:33:54.324
2 1980-07-14 14:50:42.344
dtype: datetime64[ms]
In [37]: s - np.datetime64("nat")
Out[37]:
0 NaT
1 NaT
2 NaT
dtype: timedelta64[ns] # Can we preserve `ms` instead of `ns` here?
In [38]: s + np.timedelta64("nat")
Out[38]:
0 NaT
1 NaT
2 NaT
dtype: datetime64[ns] # Can we preserve `ms` instead of `ns` here?
In [39]: s + np.timedelta64("nat", 'ms')
Out[39]:
0 NaT
1 NaT
2 NaT
dtype: datetime64[ns] # Can we preserve `ms` instead of `ns` here?
In [42]: p = pd.Series([None, None, None], dtype='datetime64[ms]')
In [43]: p
Out[43]:
0 NaT
1 NaT
2 NaT
dtype: datetime64[ms]
In [44]: s - p
Out[44]:
0 NaT
1 NaT
2 NaT
dtype: timedelta64[ms] # This looks good.
Issue Description
When we perform a binary operation against a nat
scalar, we seem to be always type-casting the result to ns
resolution. Whereas incase of a nat
series, we seem to be returning the expected type consistently. Can we make this behavior consistent in scalar nat
cases too?
Expected Behavior
array vs array binop & array vs scalar binop must yield similar time resolutions.
Installed Versions
Comment From: jbrockmendel
Agreed it would be nice to retain unit in these. Looks like in core.ops.array_ops.maybe_prepare_scalar_for_op we cast the numpy scalars to nanosecond unit. Disabling that fixes two of the cases above, but causes us to raise in the cases where the numpy scalars have no unit.
Comment From: galipremsagar
but causes us to raise in the cases where the numpy scalars have no unit.
I think it's reasonable to default to ns
unit when there is no unit.
Comment From: jbrockmendel
PR would be welcome