Plack::Test and Great Test:: Interfaces

Plack is great. Most of it is wonderful. I'm less enamored with the interface of Plack::Test, however.

In writing the Little Plack Book, I spent a couple of days writing tests for Plack and applications at the PSGI level. Plack::Test occupies a strange level in the ecosystem. It's incredibly useful for what it does in selecting between a no-HTTP backend or any other PSGI-compatible handler, and it offers some degree of abstraction for making requests and getting results, but it's far too low level to write tests for complex applications.

When writing tests for a small Dancer application, I spent more time getting the Dancer environment set up well for testing than I did writing the tests. If I'd used Dancer::Test, I suspect I'd have made more progress more quickly.

I've had similar experiences with Catalyst and Catalyst::Test.

None of this surprises me—Catalyst and Dancer are one layer up the stack from Plack. Most of the interesting things you can test about a Catalyst or Dancer or Mojolicious or whatever application are properties expressed at the framework and application layers, not the plumbing layer of Plack.

Plack::Test seems best for testing middleware and other components which occupy the layer between PSGI and applications. Even so, something about its interface kept bothering me as I wrote tests and prose about the tests:

test_psgi $app, sub
{
    my $cb  = shift;

    my $res = $cb->( GET '/?have=tea;want=tea' );
    ok $res->is_success, 'Request should succeed when values match';
    is $res->decoded_content, 'ok',
        '... with descriptive success message';

    ...
};

The pattern of a special test function which takes a block of code is semi-common in the Test::* world; you can see it in Test::Exception and a lesser extent in Test::Fatal. The best example I've seen of this is Test::Routine, which uses this block syntax to help organize tests into named groups. A disciplined tester can use this to great effect to clarify what might otherwise become a big ball of muddy tests.

I like that Plack::Test does the hard work of redirecting requests to the app I want to test on whichever backend I want, so that I don't have to worry about setting up a testing deployment target. That part's great. The confusing part is:

my $cb  = shift;
my $res = $cb->( GET '/some_url' );

test_psgi takes a PSGI application and an anonymous function as its parameters, then invokes the anonymous function, passing a callback bound tothe context of the application. Inside the anonymous function (the block, not the callback), you invoke the callback and pass an HTTP::Request object (constructed manually or with a helper such as HTTP::Request::Common) and receive an HTTP::Response object.

Put that way, it's a lot more confusing than it is, if you're comfortable with the idea of first-class functions and closures and semi-fluent interfaces in Perl 5.

Even so, my $res = $cb->( $req ) just looks weird to me. It sticks out. It's visually different from all of the rest of the code. Everything else outside the test is the semi-fluent interface of anonymous subs or boring old method calls on objects.

In a discussion with Zbigniew Łukasiak, I suggested that I'd want an interface more like:

use Plack::Test;

plack_test 'Test description here' => sub
{
    my $app = shift;
    my $res = $app->get( '/', { have => 'tea', want => 'tea' } );

    ...
};

You can see the influence of Test::Routine. I don't know exactly how $app gets into the block, but this gives labeled subtests and the concomitant organization, it obviates the need to create HTTP::Request and HTTP::Response objects (or their Plack:: equivalents) manually, and everything in the block uses visually similar mechanisms.

The only hairy part is figuring out how to connect plack_test to the app while not multiplying hidden global variables or disallowing customization and decoration with other middleware. Compatibility with Plack::Builder is important, but I don't especially want to pass in $app myself manually.

Besides, to me the only obvious benefits over Test::WWW::Mechanize::PSGI are the subtest groupings. Mech has a huge advantage of providing many useful test assertion methods which grovel inside responses so I don't have to.

Maybe this is less my discomfort with one part of the useful-at-the-appropriate-level Plack::Test and more a plea to distinguish more clearly between various elements of Test::* modules, as several distinct categories of behavior exist:

• Setting up a testing environment
• Organizing test assertions
• Providing test assertions

We did a huge amount of work a long time ago with Test::Builder making sure that everything which wanted to provide test assertions could interact nicely, and we succeeded. Maybe it's time to consider ways to enable that composition at other layers of the Test::* stack.