Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
x_start = pd.Series(pd.Timestamp(2023,3,14,12))
x_end = pd.Series(pd.Timestamp(2023,3,14,14))
# Pandas 1.5.3
# 0 7200000.0
# dtype: float64
#
# Pandas 2.0.0rc0
# 0 0 days 02:00:00
# dtype: timedelta64[ms]
(x_end - x_start).astype("timedelta64[ms]")
Issue Description
When test-driving the RC we noticed that generating Plotly graphs failed (snippet for reference below).
When investigating, we found the following in the Plotly code: (x_end - x_start).astype("timedelta64[ms]"). With Pandas < 2 this returned float64 representing the ms between the timestamps.
Now that Pandas 2 has a timedelta64[ms] dtype this dtype is used instead.
When existing code expects a float64 dtype it might break - as in the Plotly case.
As workaround, the following works with all Pandas versions:
(x_end - x_start).astype("timedelta64[ns]") / pd.Timedelta(1, 'ms')`
Maybe it makes sense to add this change to the list of breaking changes?
It might also be worth to mention that casting from np.datetime64[D] results in timedelta64[s] now as noticed here.
Plotly reproduction
import plotly.express as px
import plotly.graph_objects as go
import pandas as pd
df = pd.DataFrame([{"Task": "Job A", "Start": "2022-01-01", "Finish": "2022-01-02"}])
fig_1 = px.timeline(df, x_start="Start", x_end="Finish", y="Task")
fig_1.update_yaxes(autorange="reversed")
Leads to
[...]
File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py:258, in JSONEncoder.iterencode(self, o, _one_shot)
253 else:
254 _iterencode = _make_iterencode(
255 markers, self.default, _encoder, self.indent, floatstr,
256 self.key_separator, self.item_separator, self.sort_keys,
257 self.skipkeys, _one_shot)
--> 258 return _iterencode(o, 0)
File ~/venv/weplan3.11/lib/python3.11/site-packages/_plotly_utils/utils.py:136, in PlotlyJSONEncoder.default(self, obj)
134 except NotEncodable:
135 pass
--> 136 return _json.JSONEncoder.default(self, obj)
File /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/json/encoder.py:180, in JSONEncoder.default(self, o)
161 def default(self, o):
162 """Implement this method in a subclass such that it returns
163 a serializable object for ``o``, or calls the base implementation
164 (to raise a ``TypeError``).
(...)
178
179 """
--> 180 raise TypeError(f'Object of type {o.__class__.__name__} '
181 f'is not JSON serializable')
TypeError: Object of type timedelta is not JSON serializable
Expected Behavior
Changelog mentions the new behavior.
Installed Versions
Comment From: aberres
Ok, ok. I was blind. A colleague pointed me to the correct section.
