Pandas version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pandas.

  • [ ] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import torch
pd.api.types.is_scalar(torch.tensor([0, 1, 2]))  # returns True

Issue Description

is_scalar reports True for torch tensors (of any dimensionality as far as I can tell).

Expected Behavior

I would expect this function to return False for torch tensors.

Installed Versions

In [4]: pd.__version__ Out[4]: '2.0.0' In [5]: pd.show_versions() /nvme/0/pgali/envs/cudfdev/lib/python3.10/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.") INSTALLED VERSIONS ------------------ commit : 478d340667831908b5b4bf09a2787a11a14560c9 python : 3.10.10.final.0 python-bits : 64 OS : Linux OS-release : 4.15.0-76-generic Version : #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8 pandas : 2.0.0 numpy : 1.23.5 pytz : 2023.3 dateutil : 2.8.2 setuptools : 67.6.1 pip : 23.1 Cython : 0.29.34 pytest : 7.3.1 hypothesis : 6.72.0 sphinx : 5.3.0 blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 3.1.2 IPython : 8.12.0 pandas_datareader: None bs4 : 4.12.2 bottleneck : None brotli : fastparquet : None fsspec : 2023.4.0 gcsfs : None matplotlib : None numba : 0.56.4 numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : 11.0.0 pyreadstat : None pyxlsb : None s3fs : 2023.4.0 scipy : 1.10.1 snappy : sqlalchemy : 1.4.46 tables : None tabulate : 0.9.0 xarray : None xlrd : None zstandard : None tzdata : 2023.3 qtpy : None pyqt5 : None

Comment From: mroeschke

Here are the "scalar" checks

https://github.com/pandas-dev/pandas/blob/433f34b118f3d1a3fc439553edcb5c0ac1ecb3c0/pandas/_libs/lib.pyx#L221-L243

Debugging locally, it's interesting/odd that PySequence_Check returns False and PyNumber_Check returns True for a torch.Tensor. I would expect this to be the opposite for both checks

Comment From: mroeschke

Opened up https://github.com/pytorch/pytorch/issues/99646

Comment From: akashdasp

is this is the issue with the pandas library it has the issue with the torch?

Comment From: mroeschke

So it seems that from https://github.com/pytorch/pytorch/issues/99646 that this is intended/won't fix behavior, and I don't think pandas can special case torch objects in this function as torch isn't a related dependency in pandas.

At best, maybe pandas could document that this is function is only reliable for python stdlib, numpy and pandas objects only?