On 26/05/2025 13:44, Patrick Steinhardt wrote:
I don't think it's inherently a bad thing to fail on unexpected passes. After all, it shows that our assumption that the test fails is broken, and that we should have a look why that is. But I can see arguments both ways.
Personally I'd be very happy if our test suite failed on an unexpected pass. Currently it is easy to miss, especially if the unexpected pass occurs in a CI run. Missing an unexpected pass means we don't change 'test_expect_failure' to 'test_expect_pass' and a future regression that causes the test to fail again will go unnoticed.
Best Wishes Phillip
