Running Named Perl Tests from Prove

Organizing Perl Test Files showed the basic framework I use for individual test files written to work with Perl's testing tools. While I gladly take advantage of frameworks such as Test::Class and Test::Routine, sometimes I need something a little simpler.

The discipline of the organization method I explained in the previous article offers the benefit of simplicity and some discoverability. It also allows my team to run only a portion of the test suite as needed.

One of our products has a web interface. We have quite a few tests for this, but because they run through the whole web stack, they're quite a bit slower than tests for the business model API directly. (We want to make sure the web site always works, so we have some exhaustive tests.)

We've divided many of these tests up into discrete files based on controllers and subsets of controllers. The administrative section has a few test files. The data sharing section has a few test files. The public section has a few test files.

Some of these tests take a while to run—multiple seconds. If you're working on a bug or a feature in one specific action, waiting more than a couple of seconds for test results is way too long.

That's where the named functions I use to group related tests come in. Assume you have a test file testing the admin features, and one of those named functions is test_admin_console_list_expired_users(). You could edit the main() function to comment out all of the other tests. Alternately, add a simple code block to main():

sub main {
    my @args  = @_;

    if (@args) {
        for my $name (@args) {
            die "No test method test_$name\n"
                unless my $func = __PACKAGE__->can( 'test_' . $name );
            $func->();
        }

        done_testing;
        return 0;
    }

    # ... run all tests here

    done_testing;
    return 0;
}

... which will interpret any arguments passed to this test file as names of test functions to run. To run only the test function test_admin_console_list_expired_users(), use the prove command line:

$ prove -l t/web/admin_console.t :: admin_console_list_expired_users

The double-colon tells prove to stop looking for its own arguments and to pass the following arguments to the test file. With that invocation, only the requested test will run.

For this strategy to work, your test functions must be independent within your test files. The data the expired users list test needs to run must already be in place, untouched by other tests, or the test will fail. That's good test discipline anyway, though.

This strategy only saves us several seconds every time we use it, but saving several seconds from asking the question "Does this work?" to getting the answer is a huge benefit in the moment.

There may be easier ways to handle this, but so far this has met a real need without forcing us to divide our tests into even finer granularity (more files to manage and remember) but surely allowing us to run tests at the level we need most. For any test file which takes longer than two or three seconds to run in parallel, this has been a huge benefit.