May 2013 Archives

The Perl 5 Porters list is discussing a proposal to remove CGI.pm from the core distribution. Per the discussion, this proposal is likely to succeed, and Perl 5.20 (released around May 2014) will no longer include CGI.pm as a core library.

The arguments in favor of this are:

• It's not great code: it does far too much (conflating HTML generation with parameter processing and cookie handling), its interface is confusing (the kind of halfhearted pseudo-OO that Perl 5 circa 1994 encouraged), and its implementation is weird beyond that (the last time I looked, it had a strange load-on-demand system to avoid the performance penalty of compiling a couple of thousand lines of Perl 5 code).
• It operates at too low a level: you need to understand far too much about how the CGI protocol works to use it effectively.
• Its execution model represents the lowest common denominator of web deployment; it hasn't moved much on beyond 1998 (with some code to support FastCGI and mod_perl, but not much else).
• It's harder to deploy that it seems, if you count the system administration work it takes to configure a web server for the CGI protocol.
• In almost every case, at least one of several plausible Perl web frameworks is a better choice.

The biggest criticism is actually a Perl success story. PSGI and Plack make developing and deploying Perl web programs so much easier that there's no credible reason to use raw CGI anymore, at least if you can install Plack or use a PSGI-compatible library. If you're tied to the CGI approach, Plack::Handler::CGI is a better alternative in every way, even for local development and testing.

The cons of ejecting CGI are:

• It's been in the core approximately forever.
• Sometimes it's the only thing available, if you're using a shared hosting account where you can't install CPAN modules and a bearded sysadmin somewhere refuses to install them for you.

In other words, the discussion on p5p seems to be that the burden of maintaining code that no one uses, no one likes, and no one really wants to maintain in the core is greater than the benefit of providing a mediocre out of the box CPAN-free experience to people who don't maintain their own Perl installations.

In other words, the dominant philosophy seems to be "CPAN is integral to the desired experience of Perl 5".

(I suspect that the CPAN-hostile environment which still promotes CGI.pm as the state of the art in Perl overlaps strongly with the older-is-better-stability-over-all enterprise Linux distribution group, of which Red Hat for example won't provide Perl 5.20 for several years, so the dominant web programming model of 1997 has yet another stay of execution. Hooray. Please feel free to join us in the 21st century sometime.)

The YAPC North America Perl Conferences of the past couple of years have held an event I like quite a bit. It's the first time attendees mixer.

As of a couple of years ago, YAPC::NA organizers and attendees realized that about half of the attendees of each conference had never attended a YAPC before. That's between one and two hundred people whose main face to face involvement with the larger Perl community may have been limited to a local Perl Mongers meeting. Yes, these attendees have almost certainly used the CPAN, very likely participated in a discussion on a Perl web site, mostly used a Perl mailing list (and not just the YAPC mailing list), and have probably been on a Perl IRC channel, but they probably aren't the people you think of when you think "Who are the best connected people in the Perl community?"

As far back as I can remember (which, admittedly, is one of the YAPC::NAs in Chicago), an early morning talk has served as an introduction to YAPC, specifically intended to help new attendees understand the conference and its quirks and norms. That talk invites these attendees to the novice welcome meeting.

The organizers also grab as many of the well connected people in the community—pumpkings, core developers, CPAN contributors, authors, project leaders, anyone whose name you might recognize—and ask them to show up and be willing to talk to people. That's it.

What I like about this system is that it welcomes people in two ways. First, it acknowledges that it's okay to be new to YAPC or the Perl community in person. If that's you, you're not alone. Half of everyone you're going to see at the conference is like you in that sense.

Not only that, but you have permission to participate. You're welcome to attend this little meetup that has an explicit place in the schedule—it's an official part of the conference—and you're encouraged to talk to people you might know only by reputation. They're there to meet and talk to you. They're not there to hang out in little groups by themselves. They're there to talk to you.

I've heard good things about this event. I've enjoyed it every time I've gone. (As an introvert myself, I like having permission to talk to people with a limit of a couple of hours.)

I feel the same way about a YAPC Code of Conduct. I don't see it as a warning that "straying from a straight and narrow path of arbitrariness will not be tolerated, so if you're not sure if you might accidentally say or do something someone else doesn't like, stay away!" I see it as giving people who aren't necessarily well versed in the norms and ideals and messy politics of dealing with the Perl community every day virtually and in person permission to believe that they should feel welcome and important in the community.

It's about empathy.

I understand people disagree about the wording and even need for a code of conduct, and I don't mean to suggest that such concerns come from robots who lack empathy. By no means.

Yet put yourself in the shoes of someone who feels like he might not quite fit in in a talk, because it's full of inside jokes and jargon and the kinds of comfortable banter you only get after you've idled on a handful of Perl IRC channels for months or years. (Imagine that person's an introvert, or at least not as stubborn as I am.) Now imagine the speaker or someone else says or does something that reminds that person that he doesn't belong there, that he doesn't fit in, that he's different.

That's not necessarily assault. That's not necessarily a criminal act. But it's probably unnecessary and hopefully unintentional.

(The best silly example I can come up with is a speaker saying "... and of course, if you're a Windows user, no one cares about you until you man up and get a real operating system." and half of the audience laughs.)

Sure, there are good legal reasons to have a code of conduct that suggests that criminal activities such as assault, battery, and sexual assault are intolerable. There's no gray area about groping or rape or physical violence.

I agree with a lot of Open source is not a war zone, and I agree fully that the Perl community is all the richer for contributions of people like Wendy and Liz and Karen and Su-Shee et cetera. I'm glad they participate, just like I'm glad people like Tim Bunce and Schwern and brian d foy and Dave Rolsky and Rik et cetera participate.

I just can't quite agree that a code of conduct has a chilling effect that will exclude people. I don't see it.

Maybe that's because I see the code of conduct as explicitly saying "If you feel like you don't quite fit in, that's okay. You're welcome here. We take you seriously. If you have one of these big problems—even if it's with a speaker or writer or developer with a famous name—we'll take that seriously. The rules apply to everyone."

Sure, there's more to helping newcomers feel welcome than enforcing a policy of civil conduct, but that's the minimum I want to see, and I'm glad that YAPCs have done other things (less controversial, I'm sure) to that end.

Organizing Perl Test Files showed the basic framework I use for individual test files written to work with Perl's testing tools. While I gladly take advantage of frameworks such as Test::Class and Test::Routine, sometimes I need something a little simpler.

The discipline of the organization method I explained in the previous article offers the benefit of simplicity and some discoverability. It also allows my team to run only a portion of the test suite as needed.

One of our products has a web interface. We have quite a few tests for this, but because they run through the whole web stack, they're quite a bit slower than tests for the business model API directly. (We want to make sure the web site always works, so we have some exhaustive tests.)

We've divided many of these tests up into discrete files based on controllers and subsets of controllers. The administrative section has a few test files. The data sharing section has a few test files. The public section has a few test files.

Some of these tests take a while to run—multiple seconds. If you're working on a bug or a feature in one specific action, waiting more than a couple of seconds for test results is way too long.

That's where the named functions I use to group related tests come in. Assume you have a test file testing the admin features, and one of those named functions is test_admin_console_list_expired_users(). You could edit the main() function to comment out all of the other tests. Alternately, add a simple code block to main():

sub main {
    my @args  = @_;

    if (@args) {
        for my $name (@args) {
            die "No test method test_$name\n"
                unless my $func = __PACKAGE__->can( 'test_' . $name );
            $func->();
        }

        done_testing;
        return 0;
    }

    # ... run all tests here

    done_testing;
    return 0;
}

... which will interpret any arguments passed to this test file as names of test functions to run. To run only the test function test_admin_console_list_expired_users(), use the prove command line:

$ prove -l t/web/admin_console.t :: admin_console_list_expired_users

The double-colon tells prove to stop looking for its own arguments and to pass the following arguments to the test file. With that invocation, only the requested test will run.

For this strategy to work, your test functions must be independent within your test files. The data the expired users list test needs to run must already be in place, untouched by other tests, or the test will fail. That's good test discipline anyway, though.

This strategy only saves us several seconds every time we use it, but saving several seconds from asking the question "Does this work?" to getting the answer is a huge benefit in the moment.

There may be easier ways to handle this, but so far this has met a real need without forcing us to divide our tests into even finer granularity (more files to manage and remember) but surely allowing us to run tests at the level we need most. For any test file which takes longer than two or three seconds to run in parallel, this has been a huge benefit.

As the Testing section of the Modern Perl book tries to explain, Perl tests are just Perl code. Sure, the libraries built on top of Test::Builder extend your testing toolbox by providing new functions to help you write tests, but tests are just code.

The first implication of that statement is that, if you know how to write code, you can write tests.

The second implication of that statement—one which takes longer to realize—is that test code has maintenance costs just like code code does.

I like using Test::Class and Test::Routine when they make sense, but some tests are easier to write when they start with a little less ceremony. (I also like to refine and reorganize my tests when better organization suggests itself.) In those cases, I start with something simpler:

#!/usr/bin/env perl

use Modern::Perl '2013';

use Test::More;

use lib 't/lib';

exit main( @ARGV );

sub main {
    my @args  = @_;

    test_this();
    test_that();
    test_the_other();

    done_testing;
    return 0;
}

Every set of test assertions I can collect into a named group is a function. (You can even use Test::More subtests if you like.) When a function gets too long—when I can come up with a new name for a subset of the behavior under test—it's time for a new function.

(I write exit main( @ARGV ); as the first non-pragma line of the program so that there are no order-of-initialization concerns from Perl's compile-time/run-time distinctions. If that means nothing to you, carry on; it's a subtle thing that almost never causes problems.)

Grouping everything under functions also has the benefit of reminding other developers that it's okay to write more functions. For example, if every test for every action in a controller ought to verify access permissions, it's easy to write a helper function to set up an environment with and without access permissions to test both ways. (Some of these test functions get promoted to project-specific test libraries.)

I suspect that the discipline of writing test files as if they were code, not just scripts, is that it provides a subtle pressure to continue to write tests as if it were code, with all of the discipline and factoring and maintenance work that implies. It's easier to take something a little messy and make it a lot messier, but for me at least it's nice to start with something clean and leave it at least that clean. (This is probably why dishes pile up in my office when the dishwasher needs to be emptied.)

Please note that these test names do not necessarily correspond to individual methods or functional units within the code under test. That is, just because I might have a method called analyze_country_performance_metrics(), I don't have a single test function to test that method. I might have several. I might have none (if other code tests it adequately). Each test function tests a single, coherent, nameable feature, like "editing a user account with invalid data" or "exporting person search to CSV".

This is subtle, but it's important. Think in terms of features you can discuss with end users. (Perhaps that means I find Test::Class and its friends more useful for unit-style testing.) This helps you navigate your way through the test suite and your issue tracker by giving everyone canonical names to describe tests.

A little structure is valuable. It's easy to set up and to maintain. Yes, there's a little bit of duplication in calling functions and declaring them, but the benefit of navigation and nomenclature outweigh that many times over. The specific structure of your test files doesn't matter as much as that structure exists and you take advantage of it.

(It also lets you run test functions individually as you're fixing bugs or adding features, but the mechanics of that are the subject of another post.)