Use Test::Most for Quick Debugging

A couple of weeks ago I had to revise the data model in a client application. (Rant: when did the word "refactoring" get watered down into "changing code"? Refactoring used to mean changing the design of code without changing its behavior or external interface. If tests break, you did something wrong.) Unfortunately, this data model was at the core of one important piece of the system, and a lot of tests broke.

This was good, in the sense that we had tests for the entire feature and that they gave us a lot of coverage that we had it right.

This was bad, in that digging through a lot of failing test output is unpleasant work where you fix one thing and hope it fixes a lot of other things.

I've been working with Ovid again on this project, and one of the first things he brought up as a technical matter is the use of Test::Most over plain Test::More and the grab bag of other test modules that eventually creep in. Sure, he wrote Test::Most and has a lot of Vim macros and development habits to make his work easier, so it's not a burden to switch the code over gradually.

I'd never taken to Test::Most, mostly because I never took the time to use it seriously.

The other day, when facing a huge pile of failing tests I needed to make pass again, I decided to try it for one reason: BAIL_ON_FAIL. When you enable that feature of Test::Most, the first failing test will abort the test run. The only failure output you get is that first failing test.

I normally don't want this, because I want the whole gory details of my failure if I break something, but with the stepwise refinement necessary to finish my data model changes, it was just what I needed.

While I was editing the test files, I changed use Test::More; to use Test::Most;. That's easy enough. I didn't want to enable BAIL_ON_FAIL unilaterally because I'm not sure I want its behavior all the time. That's fine; you can run your tests with the environment variable set:

$ BAIL_ON_FAIL=1 prove -l t/web/ui_tests/some_feature.t

The responsibility for enabling this behavior is yours; it's not internal to the test file.

What if you take that one step further? What if you don't necessarily want to use Test::Most inside your test file yet? You can still get the BAIL_ON_FAIL behavior if you load Test::Most through prove:

$ BAIL_ON_FAIL=1 prove -MTest::Most t/web/ui_tests/some_feature.t

Even though the test file itself has no knowledge of Test::Most, you still get its ability to bail out of the test file at the first failure. (You do currently get a warning about a plan mismatch, but for me that's easy enough to overlook, as the rest of the output is much easier to use to diagnose failures.)