Hi there,

it seems that something changed in python 3.8 that makes pandas.Timestamp.utctimetuple() raise a TypeError instead of producing a time.struct_time.

Code Sample

In [1]: date = pd.to_datetime("2000-01-01", utc=True)                                                       
In [2]: date.utctimetuple()                                                                                 
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-17-e863c650e721> in <module>
----> 1 date.utctimetuple()

pandas/_libs/tslibs/timestamps.pyx in pandas._libs.tslibs.timestamps.Timestamp.__new__()

TypeError: an integer is required

Expected Output

With earlier Python versions the output is:

In [3]: date.utctimetuple()                                                                                  
Out[3]: time.struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=0)

An intermediate workaround can be to convert to datetime.datetime (python 3.8):

In [4]: date.to_pydatetime().utctimetuple()                                                                 
Out[4]: time.struct_time(tm_year=2000, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=5, tm_yday=1, tm_isdst=0)

Output of pd.show_versions()

commit : None python : 3.8.1.final.0 python-bits : 64 OS : Linux OS-release : 4.15.0-1066-oem machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : C LOCALE : en_GB.UTF-8 pandas : 1.0.1 numpy : 1.18.1 pytz : 2019.3 dateutil : 2.8.1 pip : 20.0.2 setuptools : 45.1.0.post20200119 Cython : None pytest : 5.3.5 hypothesis : 5.5.1 sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 2.11.1 IPython : 7.12.0 pandas_datareader: None bs4 : None bottleneck : None fastparquet : None gcsfs : None lxml.etree : None matplotlib : 3.1.3 numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : None pytables : None pytest : 5.3.5 pyxlsb : None s3fs : None scipy : 1.4.1 sqlalchemy : None tables : None tabulate : None xarray : 0.15.0 xlrd : None xlwt : None xlsxwriter : None numba : None

Comment From: sarusso

Is this still open..? No fix yet?

Comment From: jreback

@sarusso you or anyone in the community can contribute a patch

Comment From: charlietsai

I haven't found the root cause yet but it seems to be related to the time zone (on python 3.9 using latest pandas==1.3.5):

>>> import pandas as pd
>>> ts = pd.Timestamp.now()
>>> ts
Timestamp('2022-01-11 14:14:05.936634')
>>> ts.utctimetuple()
time.struct_time(tm_year=2022, tm_mon=1, tm_mday=11, tm_hour=14, tm_min=14, tm_sec=5, tm_wday=1, tm_yday=11, tm_isdst=0)

when adding a timezone:

>>> ts.tz_localize("UTC").utctimetuple()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pandas/_libs/tslibs/timestamps.pyx", line 1332, in pandas._libs.tslibs.timestamps.Timestamp.__new__
TypeError: an integer is required

https://github.com/pandas-dev/pandas/blob/v1.3.5/pandas/_libs/tslibs/timestamps.pyx#L1331-L1332

sorry, I haven't been able to dig any deeper but hopefully this helps others narrow it down

Comment From: FObersteiner

as a side note, I'm not sure why you would need utctimetuple at all for aware UTC Timestamps;

ts = pd.Timestamp.now('UTC')
print(ts.timetuple())
# time.struct_time(tm_year=2022, tm_mon=1, tm_mday=11, tm_hour=13, tm_min=29, tm_sec=56, tm_wday=1, tm_yday=11, tm_isdst=0)
print(ts.to_pydatetime().utctimetuple())
# time.struct_time(tm_year=2022, tm_mon=1, tm_mday=11, tm_hour=13, tm_min=29, tm_sec=56, tm_wday=1, tm_yday=11, tm_isdst=0)
  • the output is exactly the same.

Also quoting from the Python datetime docs:

Warning Because naive datetime objects are treated by many datetime methods as local times, it is preferred to use aware datetimes to represent times in UTC; as a result, using utcfromtimetuple may give misleading results. If you have a naive datetime representing UTC, use datetime.replace(tzinfo=timezone.utc) to make it aware, at which point you can use datetime.timetuple().

...so basically the one-liner

tt = ts.utctimetuple() if ts.tzinfo is None else ts.timetuple()

would avoid the error and give the expected result (as with native Python datetime), although it would be only correct if the naive Timestamp (input) would also represent UTC. In which case you could have used a UTC-aware Timestamp in the first place! ;-)

Comment From: charlietsai

I'm not sure why you would need utctimetuple at all for aware Timestamps

I agree! Thanks for summarizing all the additional caveats as well.

In my case, I am not calling utctimetuple in my own code but rather it's being done somewhere in a library that I use (e.g., python protobuf's FromDateTime conversion). I just happened to be using pd.Timestamps and end up with an unexpected runtime error.

I have no problem working around this in my own code, but was curious to see that this is an issue that seems to only happen with python >=3.8.

Comment From: st-bender

as a side note, I'm not sure why you would need utctimetuple at all for aware Timestamps;

I concour and would say that this is exactly the point, to get a timetuple in UTC regardless of the timezone the original dateime is in.

As the OP (almost 2 years ago), I suggest you have a look at the commit I pushed, that references the issue. And also as indicated by @charlietsai, it is called by a package that I used, not my own code. Maybe they changed that in the meantime in that package.

The idea basically is that I'd like to keep all the datetimes as UTC (aware), and in that case I passed that to a package that I have no control over, and I guess they also want to make sure that the time is in UTC. So utctimetuple() should result in the correct UTC timetuple regardless of the timezone used.

This used to work with python < 3.8, but now raises an error, which is a regression imo.

Cheers.