Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[ ] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
df1 = pd.DataFrame(index=pd.DatetimeIndex(['2022-1-1 1:00','2022-1-2 1:00','2022-1-2 1:00']),
data={'category':['A','A','B']})
res1 = df1.groupby(['category']).resample('1D').size()
print(res1)
df2 = pd.DataFrame(index=pd.DatetimeIndex(['2022-1-1 1:00','2022-1-2 1:00','2022-1-2 1:00',
'2022-1-1 1:00']),data={'category':['A','A','B','B']})
res2 = df2.groupby(['category']).resample('1D').size()
print(res2)
Issue Description
The two dataframes above, df1 and df2, are identical except for one additional data point in df2.
In df1, category A has samples on the first and second day and category B has a sample only on the second day. When I groupby-resample-count I get a Series with multiindex on category and time:
category
A 2022-01-01 1
2022-01-02 1
B 2022-01-02 1
df2 has one additional data point on an existing category and day, so that category B now has a sample on day 1, corresponding to the first time bin in the resample. The return value is now a dataframe with single-index in category and columns corresponding to the time bins:
2022-01-01 2022-01-02
category
A 1 1
B 1 1
Expected Behavior
I expect res1 and res2 to have the same type: either a multiindex series like res1 or a dataframe like res2.
Installed Versions
Comment From: dicristina
This is present on main. The result should be a series with a MultiIndex.
Comment From: dicristina
The difference between the two examples is how the results are wrapped. Eventually we get to DataFrameGroupBy._wrap_applied_output_series:
https://github.com/pandas-dev/pandas/blob/ca3e0c875fe0dd862afa88b905e8aa4a44815231/pandas/core/groupby/generic.py#L1339-L1351
Here we test whether the indices are all the same. If they are not the same (as in the first example) _concat_objects is called; if they are the same (as in the second example) the result is constructed with np.vstack.
A workaround could be to use pd.Grouper(freq="...") instead of resample. Like so:
df2 = pd.DataFrame(index=pd.DatetimeIndex(['2022-1-1 1:00','2022-1-2 1:00','2022-1-2 1:00',
'2022-1-1 1:00']),data={'category':['A','A','B','B']})
res2 = df2.groupby(['category', pd.Grouper(freq='1D')]).size()
print(res2)
I'm not sure if using pd.Grouper in this way should be the same as df.groupby(...).resample(...) for all cases but it has been the same for the examples that I have tried; except for this issue of course.
As far as I can tell this only applies to the use of df.groupby(...).resample(...).size(). Someone please edit the title so that it reads "groupby-resample-size" instead of "groupby-resample-count".