• [x] I have checked that this issue has not already been reported.

  • [x] I have confirmed this bug exists on the latest version of pandas.

  • [x] (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

Index.union treats nan differently than MultiIndex.union. Same goes for intersection. 

idx1 = Index([1, np.nan])
idx2 = Index([1, np.nan, 3])

idx1.union(idx2)
idx1.intersection(idx2)

mi1 = MultiIndex.from_arrays([[1, np.nan]])
mi2 = MultiIndex.from_arrays([[1, np.nan, 3]])
mi1.union(mi2)
mi1.intersection(mi2)

Output:

Float64Index([1.0, 3.0, nan], dtype='float64')
Float64Index([1.0, nan], dtype='float64')
MultiIndex([(1.0,),
            (nan,),
            (nan,),
            (3.0,)],
           )
MultiIndex([(1.0,)],
           )

Problem description

Inconsistent behavior is a problem. Additionally if both MultiIndexes are equal nan is only returned once. When not matching nan is the desired output, we have to adjust for theses cases before checking for equality.

Expected Output

I am not quite sure what to expect, but taking the current behavior of merges into account, I think it would be more reasonable to return

MultiIndex([(1.0,),
            (nan,),
            (3.0,)],
           )
MultiIndex([(1.0,),
            (nan,)],
           )

for the MultiIndex operations. Thoughts @jbrockmendel ?

related to #38439

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : c6b05c3ad771da775011647f66687a398e11c533 python : 3.8.6.final.0 python-bits : 64 OS : Linux OS-release : 5.4.0-58-generic Version : #64-Ubuntu SMP Wed Dec 9 08:16:25 UTC 2020 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8 pandas : 1.3.0.dev0+129.gc6b05c3ad7 numpy : 1.19.2 pytz : 2020.1 dateutil : 2.8.1 pip : 20.2.3 setuptools : 49.6.0.post20201009 Cython : 0.29.21 pytest : 6.1.1 hypothesis : 5.37.1 sphinx : 3.2.1 blosc : None feather : None xlsxwriter : 1.3.7 lxml.etree : 4.5.2 html5lib : 1.1 pymysql : None psycopg2 : None jinja2 : 2.11.2 IPython : 7.18.1 pandas_datareader: None bs4 : 4.9.3 bottleneck : 1.3.2 fsspec : 0.8.3 fastparquet : 0.4.1 gcsfs : 0.7.1 matplotlib : 3.3.2 numexpr : 2.7.1 odfpy : None openpyxl : 3.0.5 pandas_gbq : None pyarrow : 2.0.0 pyxlsb : None s3fs : 0.4.2 scipy : 1.5.2 sqlalchemy : 1.3.20 tables : 3.6.1 tabulate : 0.8.7 xarray : 0.16.1 xlrd : 1.2.0 xlwt : 1.3.0 numba : 0.51.2 None Process finished with exit code 0

Comment From: jbrockmendel

agreed