Pandas version checks
- I have checked that this issue has not already been reported.
- I have confirmed this bug exists on the latest version of pandas.
- I have confirmed this bug exists on the main branch of pandas.
Reproducible example
import pandas as pd
datetimes = pd.to_datetime(["2022-10-30 01:15:00", "2022-10-30 00:15:00",], utc=True).tz_convert("Europe/Berlin")
print(datetimes) # DatetimeIndex(['2022-10-30 02:15:00+01:00', '2022-10-30 02:15:00+02:00'], dtype='datetime64[ns, Europe/Berlin]', freq=None)
datetimes.floor("1h")
Error message:
pytz.exceptions.AmbiguousTimeError: Cannot infer dst time from 2022-10-30 02:00:00, try using the 'ambiguous' argument
Expected ouput:
DatetimeIndex(['2022-10-30 02:00:00+01:00', '2022-10-30 02:00:00+02:00'], dtype='datetime64[ns, Europe/Berlin]', freq=None)
Issue Description Pandas 1.5.1 with python 3.9.12
When flooring my datetime index to the previous hour, I get the error message. This should not be ambiguous as the time zone is clearly defined. Providing ambiguous="infer" does not help as times are not always sorted and ambiguous=True/False does not provide the expected output. Currently, workaround is to convert to UTC:
datetimes = datetimes.tz_convert("UTC").floor("1h").tz_convert("Europe/Berlin")
Comment From: thlautenschlaeger
For me your workaround doesn't throw an error. :
python datetimes = datetimes.tz_convert("UTC").floor("1h").tz_convert("Europe/Berlin")
But the 1h floor isn't applied to the DatetimeIndex. Also without converting to tz="Europe/Berlin".
Checking on that if I can find out more about that.
Comment From: thlautenschlaeger
The problem is that on this day around that time the clock was shifted back for 1 hour in tz="Europe/Berlin". That's why you have to use ambiguous. Check detailed explanation here: https://stackoverflow.com/a/51873334
Comment From: thlautenschlaeger
I think that issue can be closed.
Comment From: mroeschke
Thanks for the explanation @thlautenschlaeger, closing.
Comment From: lucasjamar
Sorry but the solution provided does not solve the problem. The example mentionned in the stackoverflow is ambiguous since no timezone is provided. The example I provided is not ambiguous since the TZ (+1/+2) is clearly defined. The output of floor is not ambiguous. The ambiguous argument would return an erroneous output here since the datetimes are not sorted. As mentionned earlier, converting to UTC, flooring and converting back to Europe/Berlin solves the issue but it adds unnecessary code and it is not something all users would think of doing and will therefore lead to errors. Under the hood, should we not expect floor/ ceil/ round datetimes operations to convert to UTC, perform the operation and then convert again back to initial datetime?
Comment From: lucasjamar
Hi @thlautenschlaeger, sorry to get back to you with this, but do you see the issue i am trying to raise?