in tests.indexes.interval.test_constructors test_constructor_dtype if we disable the tm.assert_produces_warning and instead run pytest with -W error:

pytest pandas/tests/indexes/interval/test_constructors.py -k test_constructor_dtype -W error

FAILED pandas/tests/indexes/interval/test_constructors.py::TestFromArrays::test_constructor_dtype[breaks4-int64] - FutureWarning: casting datetime64[ns] values to int6...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestFromArrays::test_constructor_dtype[breaks5-int64] - FutureWarning: casting timedelta64[ns] values to int...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestFromBreaks::test_constructor_dtype[breaks4-int64] - FutureWarning: casting datetime64[ns] values to int6...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestFromBreaks::test_constructor_dtype[breaks5-int64] - FutureWarning: casting timedelta64[ns] values to int...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestFromTuples::test_constructor_dtype[breaks4-int64] - FutureWarning: casting datetime64[ns] values to int6...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestFromTuples::test_constructor_dtype[breaks5-int64] - FutureWarning: casting timedelta64[ns] values to int...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestClassConstructors::test_constructor_dtype[IntervalIndex-breaks4-int64] - FutureWarning: casting datetime...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestClassConstructors::test_constructor_dtype[IntervalIndex-breaks5-int64] - FutureWarning: casting timedelt...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestClassConstructors::test_constructor_dtype[Index-breaks4-int64] - FutureWarning: casting datetime64[ns] v...
FAILED pandas/tests/indexes/interval/test_constructors.py::TestClassConstructors::test_constructor_dtype[Index-breaks5-int64] - FutureWarning: casting timedelta64[ns] ...

we're passing kwargs to pd.Index.__new__ which is deprecated.

Comment From: mzeitlin11

@jbrockmendel can take a look at this but can't isolate a failing example. All the above failures hit this branch https://github.com/pandas-dev/pandas/blob/feb16399da2b56bdf549b676384bfab6b7fd671e/pandas/tests/indexes/interval/test_constructors.py#L76-L78 so the call is tm.assert_produces_warnings(FutureWarning) so it doesn't raise

Comment From: jbrockmendel

Try pandas/tests/groupby/transform/test_transform.py test_transform_axis_1 case with "tshift". Set warn = FutureWarning for "tshift" and None otherwise. do with tm.assert_produces_warning(warn): around the two .transform calls. For this passes locally. Then if I change warn to be None unconditionally, it still passes.

Comment From: mzeitlin11

Thanks, can reproduce that locally. But I think it only works in both cases because the "tshift" case is xfailed. The line https://github.com/pandas-dev/pandas/blob/efbd098880a274ac19e9bc44d2dd82bac0b324c3/pandas/tests/groupby/transform/test_transform.py#L173 raises regardless, so the xfail is satisfied. If the xfail is removed and the line that raises moved out of the tm.assert... block, it does correctly assert an unexpected FutureWarning

Comment From: jbrockmendel

i think if you add raises=ValueError to the pytest.mark.xfail that'll take care of this issue

Comment From: jbrockmendel

@mroeschke i think some of your recentish work might have handled this?

Comment From: mroeschke

Yeah not getting the failures on main anymore so closing