Timestamp stores the value in a datetime64[ns] internal representation. For most purposes this is fine, but I'd love if I could choose the internal precision, for example to use datetime64[s] instead.
When I do timeseries manipulations that span more than a day, I generally do not need sub-second resolution and I could definitely use the extended range instead. But, more conveniently, this would also allow to pretty-print the result without a ton of trailing zeros without setting some arbitrary threshold.
Incidentally, this would remove the limit of ~500 years, but it's more of an ergonomic change than an actual need. For example, when I need to perform direct manipulations of the values in a DatetimeIndex, I often have to scale up to seconds if I want to perform meaningful fitting and avoid numerical instability. When plotting, it's an extra indirection I need to care about.
Comment From: TomAugspurger
From what I recall, Timestamps always being nanosecond resolution was an intentional decision that simplified the implementation. We provide Period objects (see here) for representing times outside the 500 years that DatetimeIndex can represent.
When plotting, it's an extra indirection I need to care about.
Is it bar plots that cause you issues? We already nicely format line plots, and I want to rip out that code and pass it up to matplotlib. In the process I hope to fix barplot formatting as well.
Comment From: jreback
there are a bunch of issues w.r.t. support for non-ns M8 resolutions, e.g. https://github.com/pydata/pandas/issues/7307, #6741
its a bit non-trivial, but on the roadmap, or course a pull-request would be the best!
Comment From: wavexx
On Sat, Feb 27 2016, Tom Augspurger notifications@github.com wrote:
From what I recall, Timestamps always being nanosecond resolution was an intentional decision that simplified the implementation. We provide Period objects (see here) for representing times outside the 500 years that DatetimeIndex can represent.
I read them about before and I see how they can be convenient, but somehow they feel overkill for what should essentially be a fixed precision multiplier.
It also doesn't map as nicely with numpy's datetime64[x] types, which pop up more frequently. But maybe Periods could be improved to use them?
When plotting, it's an extra indirection I need to care about.Is it bar plots that cause you issues? We already nicely format line plots, and I want to rip out that code and pass it up to matplotlib. In the process I hope to fix barplot formatting as well.
No, I didn't stumble in plotting issues directly. What I mean is that I often need to get the values directly out of the series, and in doing so I generally have to transform from/back to seconds for various reasons.
Incidentally, when I work with experiments from microbiology, I always use relative times (as in ms from experiment start). Because of this, I never had much use of the subsecond resolution in timestamps so far.
Comment From: jreback
@wavexx you know Timedeltas are designed exactly for that purpose.
Comment From: wavexx
On Sat, Feb 27 2016, Jeff Reback notifications@github.com wrote:
@wavexx you know Timedeltas are designed exactly for that purpose.
I really don't feel the need for timedeltas when working in microbiology. If you work with a fixed unit, you want to use that unit through-out, including in the internal representation.
I had a few uses of timedeltas before, but it generally involved timespans longer than a few minutes, where the added formatting made sense.
Comment From: jreback
duplicate of issues mentioned above.