Pandas Proposal: Allow operator overloading for custom classes to return custom objects

Code Sample, a copy-pastable example if possible

import numpy
import pandas

class Pipe:
    def __init__(self, function):
        self.function = function

    def __ror__(self, other):
        return self.function(other)

    def __array_ufunc__(self, func, method, *args, **kwargs):
        if func == numpy.bitwise_or:
            return self.function(args[0])
        return NotImplemented

>>> numpy.arange(4) | Pipe(lambda x: x[x > 1]) | Pipe(print)
[2, 3]
>>> pandas.Series([1,2,3]) | Pipe(lambda x: x[x > 1]) | Pipe(print)
Traceback (most recent call last):
...
ValueError: Length of passed values is 2, index implies 3

Problem description

Numpy allows custom classes to implement __array_ufunc__ and customize the behavior for operators that apply on numpy array instances and custom classes. For example, I have implemented a Pipe class that calls function on its operand by | operator. You see that numpy instance detects __array_ufunc__ method of its operand and delegates to my implementation, but pandas doesn't.

I have monkeypatched this operator in sspipe library for now, but allowsing custom classes to implement custom behaviors makes opportunity for third-party libraries to integrate better with pandas.

Comment From: mohammad7t

I just found out that pandas calls __array_ufunc__ too, but it seems that I have to return an array with same length as input. Is it possible to let them return arbitrary objects?

Comment From: jorisvandenbossche

I think in general for such TypeErrors, a way to go would be to more strictly return NotImplemented for objects we don't recognize instead of raising a TypeError.

Certainly for numeric dtypes that should be possible (for object dtype that's harder though ..)

Comment From: mohammad7t

@jorisvandenbossche please take a look at https://github.com/pandas-dev/pandas/pull/26652#issuecomment-499123938

Comment From: mroeschke

Thanks for the suggestion but given the initial reaction in https://github.com/pandas-dev/pandas/pull/26652 and lack of interest over the years, seems like this is not worth pursuing at the moment so closing for now. We can reopen if there's more interest