Code Sample, a copy-pastable example if possible
In [40]: a = pd.DataFrame({"A": pd.array(['2000'], dtype='datetime64[ns]')})
In [41]: b = pd.DataFrame({"A": pd.array([1.0], dtype='float64')})
In [42]: pd.concat([a.iloc[:0], b.iloc[:0]])['A'].dtype
Out[42]: dtype('<M8[ns]')
Problem description
The return dtype of concatentating a float64 and datetime64[ns] should be object dtype. That's typically the case when both are non-empty
In [45]: pd.concat([a, b])['A'].dtype
Out[45]: dtype('O')
But when either is empty, we get incorrect results
In [42]: pd.concat([a.iloc[:0], b.iloc[:0]])['A'].dtype
Out[42]: dtype('<M8[ns]')
In [43]: pd.concat([a.iloc[:0], b])['A'].dtype
Out[43]: dtype('float64')
In [44]: pd.concat([a, b.iloc[:0]])['A'].dtype
Out[44]: dtype('<M8[ns]')
All of those should be object dtype.
Comment From: jbrockmendel
@jorisvandenbossche is this fixed by one of your recent or upcoming PRs?
Comment From: jorisvandenbossche
This is not fixed on master.
While certainly a bug, I think it is not directly related to the recent "skipping empties or not" discussion. It seems that this empty float array already gets casted to datetime64 in JoinUnit.get_reindexed_values, so before those values are passed as to_concat
arrays to the contact machinery.
I opened a general issue at https://github.com/pandas-dev/pandas/issues/39122, where I mentioned this case as example bug for DataFrame.
Comment From: mroeschke
These almost looked fixed. Could add a few tests for the cases that work and xfail the one that doesnt
In [23]: In [40]: a = pd.DataFrame({"A": pd.array(['2000'], dtype='datetime64[ns]')})
...:
...: In [41]: b = pd.DataFrame({"A": pd.array([1.0], dtype='float64')})
In [24]: In [42]: pd.concat([a.iloc[:0], b.iloc[:0]])['A'].dtype
Out[24]: dtype('O')
In [25]: In [42]: pd.concat([a.iloc[:0], b.iloc[:0]])['A'].dtype
Out[25]: dtype('O')
In [26]: In [43]: pd.concat([a.iloc[:0], b])['A'].dtype
Out[26]: dtype('O')
In [27]: In [44]: pd.concat([a, b.iloc[:0]])['A'].dtype
Out[27]: dtype('<M8[ns]')
Comment From: GustavoGB
Hello guys! Can I take this one ?
Comment From: AwaishK
take
Comment From: liang3zy22
Take. PR already submitted as above.