Pandas version checks

  • [X] I have checked that this issue has not already been reported.

  • [X] I have confirmed this bug exists on the latest version of pandas.

  • [X] I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

from io import StringIO
import pandas as pd

data = """a,a,a,b,c,c
q,r,s,t,u,v
1,2,3,4,5,6
7,8,9,10,11,12"""
pd.read_csv(StringIO(data), header=[1, 0])

Issue Description

pandas accepts non-increasing MultiIndex header arguments, but they don't work. For instance, the snippet above produces

  NaN                    
    a a.1 a.2   b   c c.1
0   a   a   a   b   c   c
1   q   r   s   t   u   v
2   1   2   3   4   5   6
3   7   8   9  10  11  12

i.e., a DataFrame whose columns are

MultiIndex([(nan,   'a'),
            (nan, 'a.1'),
            (nan, 'a.2'),
            (nan,   'b'),
            (nan,   'c'),
            (nan, 'c.1')],
           )

Parsing the data with header=[0, 0] is also accepted, and behaves sensibly:

   a           b   c    
   a a.1 a.2   b   c c.1
0  q   r   s   t   u   v
1  1   2   3   4   5   6
2  7   8   9  10  11  12

but I can't see a reason to support redundant header levels.

Expected Behavior

I propose that non-increasing header arguments raise a ValueError('header elements must be increasing') or something to that effect.

Installed Versions

INSTALLED VERSIONS ------------------ commit : 7c913d6b750f5fe0f3679dc66441011aa90158f6 python : 3.8.13.final.0 python-bits : 64 OS : Linux OS-release : 5.15.0-27-generic Version : #28-Ubuntu SMP Thu Apr 14 04:55:28 UTC 2022 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_CA.UTF-8 LOCALE : en_CA.UTF-8 pandas : 1.5.0.dev0+786.g7c913d6b75 numpy : 1.21.5 pytz : 2022.1 dateutil : 2.8.2 pip : 22.0.4 setuptools : 62.0.0 Cython : 0.29.28 pytest : 7.1.1 hypothesis : 6.41.0 sphinx : 4.5.0 blosc : None feather : None xlsxwriter : 3.0.3 lxml.etree : 4.8.0 html5lib : 1.1 pymysql : None psycopg2 : None jinja2 : 3.0.3 IPython : 8.2.0 pandas_datareader: None bs4 : 4.11.0 bottleneck : 1.3.4 brotli : fastparquet : 0.8.0 fsspec : 2021.11.0 gcsfs : 2021.11.0 markupsafe : 2.1.1 matplotlib : 3.5.1 numba : 0.55.1 numexpr : 2.8.0 odfpy : None openpyxl : 3.0.9 pandas_gbq : None pyarrow : 7.0.0 pyreadstat : 1.1.4 pyxlsb : 1.0.9 s3fs : 2021.11.0 scipy : 1.8.0 snappy : sqlalchemy : 1.4.35 tables : 3.7.0 tabulate : 0.8.9 xarray : 0.18.2 xlrd : 2.0.1 xlwt : 1.3.0 zstandard : None

Comment From: simonjayhawkins

Thanks @ahawryluk for the report.

I propose that non-increasing header arguments raise a ValueError('header elements must be increasing') or something to that effect.

makes sense and keep this issue labelled as a bug for now. The documentation maybe should be updated too.

The alternative could be an enhancement to allow this, but maybe unnecessary to add code complexity when a simple swaplevel would give the expected result in this case.

pd.read_csv(StringIO(data), header=[0, 1]).swaplevel(axis=1)

   q  r  s   t   u   v
   a  a  a   b   c   c
0  1  2  3   4   5   6
1  7  8  9  10  11  12

but I can't see a reason to support redundant header levels.

agreed. This maybe should raise a ValueError too, although this would be a breaking change if any users currently depend on this behavior.

contributions, PR and further investigation/suggestions welcome.

Comment From: ahawryluk

take