Pandas version checks
-
[X] I have checked that this issue has not already been reported.
-
[X] I have confirmed this bug exists on the latest version of pandas.
-
[X] I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
import pandas as pd
import plotly.express as px
df = px.data.iris()
#Works
px.scatter(df, x="sepal_width", y="sepal_length", color="species")
#doesn't work as expected in 1.5.x
df.plot.scatter(x="sepal_width", y="sepal_length", color="species")
Issue Description
Plotly (and perhaps other backends) take a color argument that is a column name to use to make distinct colors for each variable (or even continuous color.) This is very convenient compared to the c
variable in the pandas backend that takes a list of colors.
In Pandas 1.5, color
and c
in the back end code become the same, or rather color
is an alias of c
:
In addition, the code pops out the color
argument so that it no longer exists in kwargs
so the color information (expected to be a list of columns) is now absent. Changing the line to
color = kwargs.get('color')
would leave the variable in place, but then other plotting back ends would get color and presumably crash on an expected keyword, and would have to use c
instead.
I think it would be best to not alias color
to c
for scatter plots and let backends like plotly
continue to work as they did prior to 1.5.x
Expected Behavior
Plotly color
kwarg should behave as expected prior to 1.5.x and pass through a column name to the plotly backend, not remove the kwarg and alias it to the c
kwarg.
Installed Versions
Comment From: MarcoGorelli
That's a good point, thanks @astrowonk
It may be best to revert the PR that introduced the alias
Comment From: nicolaskruchten
Thanks for reporting this @astrowonk and for looking into it @MarcoGorelli!
As the author of the plotly
backend, I would definitely expect that the external API here be very stable... "removing" kwargs like this definitely breaks our backend, but so would adding special handling to previously-ignored kwargs. For example right now with the plotly
backend, df.plot.scatter()
accepts all of the kwargs that plotly.express.scatter()
does, so any "upstream" modifications to the kwargs would break things: https://plotly.com/python-api-reference/generated/plotly.express.scatter.html
I had to add some code to adapt the kwarg lists for various df.plot
methods here https://github.com/plotly/plotly.py/blob/master/packages/python/plotly/plotly/init.py#L99 (s
, c
, by
etc) and I guess I could continue to do so but I sort of built this backend under the assumption that this stuff wouldn't change very much :) Maybe I misunderstood the contract between Pandas and plotting backends though!
Comment From: MarcoGorelli
Hey @nicolaskruchten
Maybe I misunderstood the contract between Pandas and plotting backends though!
No no, this is totally my fault for having approved https://github.com/pandas-dev/pandas/pull/44856/files . In fact, I should've rejected the issue instead of encouraging a PR. I wasn't thinking about the plotly backend because I don't use it (I use plotly directly by importing plotly and plotly.express)
Going forwards, something I've had on the back of my mind is to suggest to the rest of the pandas dev team is to declare pandas.plotting as being in "maintenance mode" only (my personal preference would be to remove most of it completely, as I think plotly.express
and seaborn
do the same thing but better. There's no comparison...). I'll open an issue about that soon.
Either way, you shouldn't need to worry about breaking changes in pandas.plotting.
Regarding what to do about this specific issue, I'll look into solutions more carefully later this week
Comment From: astrowonk
Going forwards, something I've had on the back of my mind is to suggest to the rest of the pandas dev team is to declare pandas.plotting as being in "maintenance mode" only (my personal preference would be to remove most of it completely, as I think
plotly.express
andseaborn
do the same thing but better. There's no comparison...). I'll open an issue about that soon.
@MarcoGorelli I would just say that I have made extensive use of the pandas plotting back end syntax (albeit lately with plotly and the backend). I think it's important to keep so existing code keeps working with existing backends.
Were it to vanish, so many scripts and modules I have written would break, so whatever happens, please keep the df.plot
syntax working!
Comment From: MarcoGorelli
Thanks @astrowonk , that's a useful perspective
I was thinking more about all the custom matplotlib code in pandas, as that's what causes the most issues - it could be greatly simplified, like with an entrypoint in seaborn similar to the one in plotly. I'll flesh that out in a separate issue anyway
Comment From: rhshadrach
declare pandas.plotting as being in "maintenance mode" only (my personal preference would be to remove most of it completely, as I think
plotly.express
andseaborn
do the same thing but better.
I find it very useful in ad hoc data analysis to be able to do df['my_column'].plot.hist(bins=30)
etc. I would be -1 on removal (wasn't entirely sure what "most of it completely" meant).
Edit:
I was thinking more about all the custom matplotlib code in pandas, as that's what causes the most issues - it could be greatly simplified, like with an entrypoint in seaborn similar to the one in plotly. I'll flesh that out in a separate issue anyway
Ah, I should have read further before posting. +1 here.
Comment From: nicolaskruchten
Yeah my feeling is that the existing functionality can never go away... Maintenance mode sounds good to me though!
Comment From: MarcoGorelli
I was thinking more about all the custom matplotlib code in pandas, as that's what causes the most issues - it could be greatly simplified, like with an entrypoint in seaborn similar to the one in plotly. I'll flesh that out in a separate issue anyway
Finally got round to it - here's my proposal: https://github.com/pandas-dev/pandas/issues/50059