June 2011 Archives

At YAPC::NA 2011, the hotel made the understandable mistake of directing attendees to "The PERL Foundation" event. As you might expect someone corrected the sign by prepending ucfirst lc.

Any self-respecting Perl hacker should be able to devise several more creative (if less straightforward) ways to correct such a mishap.

While JAPHs and obfuscations are fun, why not correct the problem at the root? What if it were impossible to write PERL by accident? Enter Acme::PERL::Autocorrect, an extension to the Perl optimizer which automatically corrects PERL to Perl in literal strings in Perl source code.

The initial version only fixed strings containing PERL. The second version improves that to detect PERL as a substring. That improvement was more difficult than it might seem. My first approach made the tests themselves fail:

#   Failed test 'use Acme::Perl::Autocorrect;'
#   at t/autocorrect.t line 7.
#     Tried to use 'Acme::Perl::Autocorrect'.
#     Error:  Can't locate Acme/Perl/Autocorrect.pm in @INC...

... for the test line:

BEGIN { use_ok( 'Acme::PERL::Autocorrect' ) }

... because the string in that statement gets rewritten as part of the optimization process. The naï approach is to use a negative-width lookahead regular expression to find PERL:: and exclude that from the rewriting, but that doesn't work. By the time the custom operator override gets to this string, Perl's optree contains instead Acme/PERL/Autocorrect.pm.

You may have to install a development version of optimizer to make this work, but work it does—with pure Perl. You need a decent understanding of the Perl optree to write an optimizer stage, but even this silly Acme:: module demonstrates that such things are possible.

Perl's in kind of a bad way, mindshare wise, but look sometime, really look, at the Perl 6 RFCs.

How many of those problems are still real problems?

Forget the CPAN. It's not that the CPAN is bad. (It's great!) It's not that you can't find great code on the CPAN to solve many of those problems. (You can!)

I'm glad Perl borrowed and implemented Python's lousy object system in such a minimal way only in so far as it allows for things like Moose which replace and improve upon and supersede it in such a dramatic way that I'm not sure Moose would have existed if Perl's default object system were better.

Are you still wearing your "I'm not a Perl programmer!" shoes? Step back one more step.

Does it make sense to you that if you want to write object oriented Perl in 2011, almost ten years after the P6 RFCs identified 361 pain points in Perl, you still have to download an extension because the Perl community can't or won't agree on one good way to write classes and objects by default and doesn't want to manage the task of adding an object system to the Perl core because it might change, and even if the Perl porters took on that burden, you still probably wouldn't be able to declare a class with the keyword class, because package is (and here is where you step forward several steps) pretty much just as good and really the same thing if you squint?

Does that make sense to you?

By no means do I fault the implementors. Modifying Perl is difficult. I don't blame anyone for not wanting to dive into its guts to refactor things to make it easier to add new features or to fix bugs or to improve performance or even to see if something is vestigial code which can go away without breaking backwards compatibility. It's not fun, and people like Nick and Rafael and Dave and Zefram and Florian and Karl and too many other people to mention who've done far more than I have have my full respect and deserve your respect as well.

Even so, take another step in those "I don't know much about Perl, so tell me about it?" shoes.

Renaming Perl 6 won't change two facts.

• Perl has no Larry. That Larry doesn't have to be the Larry, but a sufficient quantity of Perl hackers must respect the new Larry as a suitable Larry.
• At almost every point where the design of Perl requires a hard choice between improving (or maintaining the status quo of) the language for people who've been using Perl for years or decades or improving the language for people who haven't, the default choice has been to circle the wagons and keep the status quo.

Renaming Perl won't change that. All of your marketing material can rigorously refer to "hydrogen" as "Elevo Aeris", but you haven't changed the physical properties of a single atom. Oh, the Perlmanity. If you believe that Perl's popularity depends on a feature set, you need somehow to provide:

• A better object system
• Function signatures
• A better reference syntax
• Continued internals overhaul to improve Unicode handling
• Better defaults
• An extension system which doesn't require learning a new language which is half-C, half Perl macros, and half reformed Cimmerian
• A distribution mechanism that solves the Enterprise in Mothballs problem
• A gradual typing system
• A parser reusable outside of the act of writing code
• The possibility of running on other VMs
• The possibility of JIT or other optimizations
• A parallelism and concurrency solution better than heavyweight ithreads
• Improved uniformity of syntax and semantics along the lines of autobox
• An exception system which does the right thing by default

... and I've probably left out a few things.

You can work around many of these problems with the CPAN (in one shot with things like Task::Kensho and perl5i. Thank Larry and countless volunteers for that...

... but are you still wearing your "I've heard about this Perl thing, but why would I use it?" shoes? Take a Perl project of modest complexity. I have a few with several thousand lines of code apiece. They're not huge projects, but I've done the rodeo circuit enough times that I get a lot of use out of CPAN distributions.

Install all of the dependencies of one of these projects in a fresh Perl sometime. Track all of them. Run Devel::TraceUse on the top-level program sometime. Go through that list. Group those dependencies into categories of similar behavior. Go ahead. I'll wait.

The problem is that your average Perl project of moderate size which uses the CPAN needs to load multiple, competing modules to provide behavior that arguably should have been core language behavior five years ago, if not ten or twenty.

The problem is also that the voluminous documentation to which IRC continually points frustrated novices (telling them that the F is for Friendly) assumes in many places a working knowledge of C, shell, and Unix, as well as an existing overview of how Perl itself fits together.

What will help the frustrated novice who sees a dozen competing projects for exception handling and doesn't know where to start. Yes, yes, I know, Task::Kensho, but do you really want to say "To throw an exception in Perl, you must first configure a CPAN client. Oh, you don't have root access? Convince your system administrator to install and configure local::lib, but some people prefer App::perlbrew, and I hope you're not on Windows, ha ha... wait, where are you going? All you have to do is pipe a shell program downloaded from a web page into...."

I know, I know. It's frustrating to think that Perl is in a holding pattern and can't evolve while Perl 6 has been just around the corner for a decade. But that doesn't change anything. Perl will not evolve into something more popular for new projects as long as it huddles within the safe, comfortable fortress walls of "Well, we've always done it this way, and it would be scary to change it."

(I don't believe that most contributors have that opinion, but given the difficulty of making changes to Perl and the potential disaster if part of that change is wrong and the implication that the world could be stuck with something awful for years if things go wrong, there's very strong pressure not to do anything. I also believe that the last eighteen months in the world of p5p demonstrate potential for huge, enormous, world-changing improvements in what Perl is and can be, and I'm excited for the future of Perl. I gladly use CPAN. I look forward to Perl.16 and 5.18 and 5.20. I have great respect for every contributor to Perl and the CPAN and don't blame any of them for the current situation. It just happened. Good things are happening. The tides are turning. This is no accident. Now how do keep that momentum and direct it effectively?)

Features, of course, are only part of the problem. PHP is a mess of a language in many ways, but it's popular because it does a couple of things right enough. Ruby is a flawed language, but it has buzz because Rails did a couple of things right. Python and Perl share many of the same flaws (and Python adds a couple and takes away a couple), but its marketing message (however nonsensical) is at least consistent. Lua is incredibly minimal in features and frustrating in what it lacks, but its designers have a laser-like focus on what the language is and will be, and it succeeds in its niche because of that. JavaScript is Perl 4 awful in writing large projects, but it spins the pork pie hat of every skinny-jeans hipster in the tech world because it's there.

It's easy to blame "Perl 6" for Perl's conservatism, but it's wrong. Perl has consistently demonstrated the relentless desire not to lose existing users instead of the relentless and focused desire to attract new users.

(Don't hold it against Perl, however. P6 has demonstrated a different problem: the relentless and focused desire not to produce working software which might possibly attract users. You can tell because of the monthly whining noise which comes from Rakudo marketdroids begging for attention to justify their hobby.)

Want to make Perl more appealing to the person who loaned you that pair of shoes? (Start by giving back the shoes.) Let's talk about fixing Perl instead of fixating on how not having a major version number means we can never fix Perl.

By all means complain about the overloading of the name "Perl" to mean two similar things which are not the same thing. It's an easy target. Just don't delude yourself that a name change will happen or that it will accomplish anything meaningful. The version number 6 has been doing its damage since about 2001, and it's hard to imagine that it can do any more.

Let's change the name of Perl 6. Why not change the name of Perl 5 too?

The benefits are obvious. Perl would be free to break backwards compatibility. More than that, it could stop halfway borrowing features from P6 that are impossible to introduce properly to Perl because Perl's infrastructure just can't support them.

It could remove features and vestigial libraries from the core, like dumpvar.pl (look it up) and h2xs. It could clean up the bareword lexing mess and the dative syntax mess. It might even allow the autopromotion of bare blocks into first-class functions in places other than only the first argument of a predeclared and prototyped function.

All of the creaky old ported-straight-from-Fortran tutorials would obviously no longer apply. Awful, awful examples and books would no longer work. Search engine result ranks would be open to a shiny new world of good code and great documentation from the start.

We also wouldn't have to use those stupid branding terms like "Modern Perl" or "Enlightened Perl" or "Perl Renaissance" or "Kensho" or "Foundation" or "Equites Synarchic".

So many problems would simply go away by changing at least two of four letters in the name "Perl". Maybe three. Or maybe adding a letter, or changing one of them to a number, or maybe a snowman glyph. Something that would look nice on a tattoo, or alliterates better than "Desperate Perl Hacker". The word "puppy" is nice. Can you work that in there somewhere?

Don't worry about losing mindshare (the 6 in P6 has been taking care of that for years). Don't worry about confusing people. After all, the name "Perl" means exactly what you think it means right now, whether you first used Perl 1 in 1987 or last used Perl in 1987. If Perl, the concrete thing, is to change from what you think it means right now at this moment—this perpetual now, where there is no future and there is no past, just a bundle of electrical impulses sailing an eternal and constant voyage between wads of synapses—to something else, it must have a different name. There is no ideal. There is only what we see and experience in the immediate. (You kind of have to accept this anyway if there's no specification against which you can accurately measure a concrete thing's Perlishness.)

Yes, renaming Perl will immediately heal ten years of a great divide in the snap of my fingers. It will save the Perl community from the tyrannical march of version numbers (the horror, the horror). It will squelch the smoldering debates over whether the second item in a set of dotted decimal triplets is a sub-version or a major version or a minor version (and the families of the victims might finally know peace).

Just don't argue with me over the version numbering scheme. It's 2011.06.24.14.00.38 as all good hearted people will obviously agree.

I spent all day thinking about marketing. Blame VM Brasseur.

I think a lot about marketing my business. Today I think also about marketing Perl. (Sometimes what's good for the latter helps the former.) I worry sometimes that Perl misses opportunities to find great new developers and wonderful new users because it's not always clear how or where or even why to start.

I worry sometimes that the mean time between beginning something and getting to that first "I am awesome!" moment is too long. (I once read a book proposal about teaching programming to kids using games which promised that within a week they'd be able to perform ASCII character animations.)

I worry that the excellent tools of the modern or enlightened or CPANified or renaissance or whatever you want to call it Perl of 2011 have a learning curve too steep. After all, when someone like Schwern says "If I have to learn how to use a completely new build system just to send you a patch, I just might give up!" you should pay attention—not that it's wrong to create and use great new tools, but that adopting them has a non-negligible opportunity cost.

Then I have a conversation with Michael Buffington who tells me about this idea he and Rael had to solve hard problems in multi-person calendaring by making a Battleship-style game for arbitrary other users to play in order to find the maximal good scheduling solution. ("Battleship isn't the right model," I told Michael. "But it's almost right." I told him my better idea, and it just might work.)

Games, hmm? If the wisdom of crowds can help solve grotty OCR problems, can you take other computationally difficult problems and solve them by turning them into games?

If so—if you can convince people that they're playing while they're actually doing real work of some sense or another—can you use a similar mechanism to help them appreciate that as they practice doing real work, they're achieving small "Hey, I just did something awesome!" moments?

Could these problems solve themselves? Could gamification help market Perl? Ben Trott and I once had a silly contest far too many years ago to see who could answer the most questions on PerlMonks to reach an artificial milestone.

Maybe new contributors deserve stickers.

Maybe finally getting your PAUSE ID should earn you a little badge, or filing a unique perlbug could post to your Facebook wall, or convincing ten of your colleagues to download and read the Modern Perl book might merit a new background for your Identi.ca or Twitter page.

Then I realize that I grew up in the Puritan-infused America and specifically the Manifest Destiny rugged individualist fierce self-reliant western United States, and then I start to think that maybe gamification is what you do when your busy work isn't compelling enough on its own for people to enjoy themselves in what they're doing.

After all, sometimes I still feel like I'm on top of the world when I hear about a great new distribution which will change a little piece of my world, and I have it installed and working thanks to the great CPAN infrastructure and the SYNOPSIS section and CPAN testers and all of this wonderful ecosystem and, in ten minutes or less, I can do more with less code, and, like I said, I started far more many years ago than I want to admit.

Maybe somehow we can help show other people that if the shortest distance between their problem and a good solution is Perl—and it often is—they can share that same experience. No shiny tricks. No slight of hand. No slimy psychology putting us all in Skinner boxes. No empty calories of gamification or hypermasculine chest-beating or unprovable appeals to the artificial standards of pseudocode readable by anyone. Instead, it's just good code you can be proud of having written because it solves real problems and stays out of your way.

I can live with that.

If you look at the darkpan-inject program in my CPAN::Dark experiment, you'll see something I found curious:

use CPAN::Dark;
CPAN::Dark->new->inject_files( @ARGV );

That's all. If you already have configured your CPAN::Mini installation, my little wrapper around CPAN::Mini::Inject does everything else for you, including setting up a sparse CPAN mirror. It'd configure CPAN clients for you too, if there were an easy way to do so.

Why? Why not! I'm lazy. I'm happy providing sane defaults that will work on all machines to which I deploy, if that means I can avoid tedious, time-consuming, fiddly steps I'm likely to get wrong in various interesting ways.

More than that, I'd rather write a module than a program, because I arrange my programs like modules anyway, and the infrastructure for testing and documenting modules is much better than the infrastructure for testing and documenting programs.

I realized this as I pondered the build system we use for Modern Perl and several other books. (More and more people ask if they can use the build system for their own projects, and I'm thrilled to tell them yes.) I used to think I'd eventually extract the build system into its own Github project which people could clone if they wanted to write a new book, but eventually my good sense as a software developer finally kicked in.

The Onyx Neon book tools don't belong as, primarily, a Github project. They belong on the CPAN. Creating a new book shouldn't require forking a project and copying some files around—especially as those files are programs longer than a dozen lines. Creating a new book could be as easy as creating a new CPAN distribution with Dist::Zilla or a new web project with Dancer (or Catalyst or Mojolicious or whatever).

Certainly you'll have to fill in a configuration file with some specific information, but most of the infrastructure required to build a PDF or an ePub file or nicely-formatted HTML is already on the CPAN. Why not go all the way?

Making a program into a module and excising all but two lines of the program won't solve every problem, but the module is still available for people to use to write their own programs. It's easy—and the common things, if the common configurations are correct—are simple. (Creating a documentation-only tutorial is also easy, and it puts the documentation for layouts and workflow and even syntax and usage recommendations in the hands of potential authors and editors.)

Come to the Book Author BOF at YAPC::NA 2011 or catch me in the hallway track or talk to me at Open Source Bridge for more details.

What if everything we thought about types in dynamic languages were wrong?

(Dear Internet forum readers, Thank you for reading all the way into the second paragraph. I shouldn't have to answer a rhetorical question with the obvious answer of "It isn't," but now you have both the answer and the moral authority to mock everyone with a narcissistic streak large enough and an attention span short enough to rush immediately to a favored Internet forum and post a lengthy "Look at how smart I am and how dumb someone else is!" screed. Have a nice afternoon!)

I've updated Test::MockObject, UNIVERSAL::isa, and UNIVERSAL::can recently. I've also spent a lot of time working with Moose-powered code to eliminate duplication, improve genericity and polymorphism, reduce the possibility of errors, and increase testability and test coverage.

Then I read the documentation for Data::Thunk and something interesting struck me.

I've long argued that checking types by name and mechanism in Perl 5 is problematic. So is giving up by pretending everything is a duck. The conflict between specifying your requirements too strictly and removing the possibilities for extension or genericity you can't imagine and being too lax and allowing the possibility for error is difficult to navigate.

Yet sometimes I think about this in the wrong way.

A thunk in the Data::Thunk sense is a way to delay an expensive calculation until you need it. (I quite like lazy synthetic attributes in Moose for similar reasons.) While synthetic object attributes are promises encapsulated in an object behind accessors—and you can pass that object around without ever triggering the lazy generation until you need it—a thunk exposed as a non-object, perhaps an array or an other primitive first class value, doesn't have the same level of encapsulation.

In other words, it's far too easy to tell that @immediate_values is different from $thunk_to_calculate_values.

... except that perhaps we think about things incorrectly.

Suppose you want to generate a list of prime numbers for cryptographic purposes. The first few prime numbers are cryptographically worthless. The numbers get ever more expensive to calculate as you go on. Your code needs to find a balance between calculating too many or too few, but you don't necessarily know which pair will suffice for your purposes until you calculate them. (Also, you probably don't have to calculate all of the intermediate primes between 1 and 2 and n and m, if you have a good algorithm to pick a few potential primes and continue from there.)

I see three possibilities to represent the data structure containing this list of primes:

• A plain array
• An iterator or generator (whether with internal language support or an object)
• Something lazy as a combination of both

You might be fortunate enough to use a language such as Haskell with this laziness as a fundamental language feature. Good for you! You may use Python, in which case a generator expression might be the best approach. Hooray, I suppose. You may use Perl 5, and so you have plenty of options for syntax. That flexibility can be handy.

How much are you going to let your internal representation of a storage mechanism tie your hands from a design perspective?

If you choose the array, you've tied your code to a specific mechanism and a specific syntax. The same applies to an object or generator, though the object gives you slightly more options, in the polymorphic sense, to retain an interface but provide a different implementation. Even so, you can't swap a lazy array for an object without bridging the difference in interface, unless your language explicitly supports this.

Therein lies my question about type checking in dynamic languages.

I've been a good programmer. I rewarded myself with a couple of cookies for separating the concern of generating a list of random numbers from the code which uses that list of random numbers. It's easier to test, to maintain, to document, to maintain, to do everything I might need to do to it.

Yet the two pieces of code I've worked so hard to decouple in form are still tightly coupled via the types of a parameter used to communicate between them, because they both have a dependency on being some sort of array, some sort of generator or iterator, or some sort of object which provides a generator or iterator, when all I really want to be able to say is "These two separate pieces of my system communicate when one of them provides a promise to provide multiple prime numbers".

I used the word tied in a previous paragraph. If you caught the pun, great. (If you're not a Perl 5 programmer and you wonder about the pun, don't. It's really not that clever.) That approach is good in some ways and awful in others, because it does allow uniformity of interface (with an awkward and slow implementation) but it doesn't allow me to decorate the arrayish variable with the type information that "This thingie you can treat as an array is a promise to provide random numbers when you want them. Don't ask how. Leave efficiency concerns to the implementation. You worry about what you get out of it."

(One also sometimes wonders why Python simultaneously prides itself on a rigid orthogonality and parsimony of syntax such that toddlers often speak in valid Python programs before they learn the vagaries and inconsistencies of English while borrowing and uglifying generator and filter syntax from other languages when it would have been simpler to say that generators are specific enhancements and refinements of lists. Then again, I as an American from the frontier states lack certain clever Continental irony.)

In other other words, maybe us dynamic language get types so wrong because we're so caught up in thinking "What primitive does this resemble?" or "What global string name does this or any class in its inheritance hierarchy match?" that we too rarely stop to ask ourselves the important question of "What does this data represent?"

Would that our languages allowed us to express that meaning instead of merely the mechanism.

I have my own DarkPAN. Several of my projects share code that's not yet ready for the public CPAN. Some of that code may never belong on the CPAN—it's far, far too specific for the problems I'm solving.

Experienced Perl hackers often joke that 90% of their applications comes directly from the CPAN. A few thousand lines of code on my part might make use of tens of thousands of lines of code written, debugged, tuned, improved, documented, tested, and maintained by countless other people. Even better, installing and maintaining and depending on that code requires only the use of a few simple tools, including App::perlbrew, Dist::Zilla, cpanminus, and cpan-outdated.

Yet I want to share code between my own applications without having to upload it to the public CPAN. I want to use the same tools to manage and deploy this code as I do to code from the public CPAN.

In short, I want my own private CPAN containing only the code I put there.

That's why I wrote something I'm calling CPAN::Dark for now. It's a thin wrapper around CPAN::Mini::Inject which creates an empty, private CPAN into which you can add any or all distribution you like.

If you do this, you can configure your CPAN tools to use your private DarkPAN as one of multiple mirrors, so that CPAN operations will find your private code. You can also (as I do) use the dzil ScpDeploy plugin to inject your code into your DarkPAN as easily as you would release it to the CPAN.

(It would be nice if cpanplus and cpan-outdated and other CPAN tools had a single point of configuration to manage mirror lists, but that lack is reasonably easy to automate away for now.)

With all of my projects safely ensconced in a DarkPAN, it's even superlatively easy to test my code against any release of Perl 5 that perlbrew can manage. All I have to do is install the new version of Perl 5, then let cpanminus do its thing until it finds an error or completes successfully.

I've never had code deployment this easy before.

Your job, if you choose to accept it, is to give me feedback on CPAN::Dark. Is the name right? Does this solve a real problem? Is the documentation clear? (Does it need a mechanism to remove distributions?) Should it include configuration instructions for cpanminus/CPANPLUS/CPAN.pm?

I have a handful of modest Catalyst applications and will undoubtedly have more. One of the reasons I enjoy working with Perl 5 so much is that it's reasonably easy to solve a problem experienced in multiple places, extract that solution, and generalize and publish it for other people. The modern Perl world only makes that easier with Moose, CPAN, and even Github.

Sometimes that ease moves the problem around, which reinforces the notion that the only two hard problems in computer science are naming and caching. To abuse that truism, once I've found a workable solution, the hard part is figuring out what to call it and how to make it available to other people.

Take the flash feature of Catalyst's Catalyst::Plugin::Session. It resembles the stash, in that you can store and retrieve information in and from the flash, but that data is only available until the first request which uses it. This is incredibly handy for user notifications, such as "You missed a required entry field!" or "Your interaction succeeded!", especially when combined with the Humane JS library.

My controller actions have a pattern:

sub edit :Chained( 'get_widget' ) :PathPart('edit') :Args(0)
{
    my ($self, $c) = @_;

    return unless lc $c->req->method eq 'post';

    my $params = $c->req->params;
    my $widget   = $c->stash->{widget};

    try
    {
        $widget->$_( $params->{$_} ) for $widget->editable_fields;
        $widget->update;
        push @{ $c->flash->{messages} }, 'Updated widget!';
    }
    catch { push @{ $c->flash->{errors} }, $_ };

    return $c->res->redirect(
        $c->uri_for( $c->controller( 'Widgets' )
          ->action_for( 'index' ) )
    );
}

Yet every time I write that code, it feels slightly wrong, as if I'm violating encapsulation by updating the variable behind the flash directly. It's also prone to typos and has the whiff of violating the Law of Demeter.

I prefer instead to write:

    try
    {
        $widget->$_( $params->{$_} ) for $widget->editable_fields;
        $widget->update;
        $c->add_message( 'Updated widget!' );
    }
    catch { $c->add_error( $_ ) };

... where my actions don't have to know about the internals of how the flash stores its data, where I don't have the possibility of stomping all over the array references accidentally, and where I can replace syntax (the grotty Perl 5 reference syntax!) with a method which describes my intent.

The simplest way to make this work is:

package Catalyst::Plugin::Session::FlashMethods;
# ABSTRACT: add flash-manipulation methods to Catalyst::Plugin::Session

use parent 'Catalyst::Plugin::Session';

use Modern::Perl;

sub add_message
{
    my $self = shift;
    push @{ $self->flash->{messages} }, @_;
}

sub add_error
{
    my $self = shift;
    push @{ $self->flash->{errors} }, @_;
}

1;

... and load it in my applications with:

use Catalyst::Plugin::ConfigLoader;
use Catalyst::Plugin::Static::Simple;
use Catalyst::Plugin::Session::FlashMethods;
use Catalyst::Plugin::Session::State::Cookie;
use Catalyst::Plugin::Session::Store::FastMmap;

use Catalyst qw/
    ConfigLoader
    Static::Simple
    Session::FlashMethods
    Session::State::Cookie
    Session::Store::FastMmap
/;

This approach has (at least) two problems. First, while I'm happy to make this code available to other developers, I don't want to add any difficulties to use other plugins which may themselves extend or modify Catalyst::Plugin::Session.

Second, this code is awfully specific to my own uses. I believe those uses could help other people, but why solve a problem so specific when genericity and wider applicability are possible?

Rephrasing the problem might help.

I have a hash. I want named methods to store application-specific data in that hash. I would like the ability to customize that set of methods methods without hard-coding them, so as to produce a component that multiple people can reuse in multiple applications.

You can see why a big bag of untyped stuff (a hash) is useful.

What I really need is a way to say "My application will use these and only these specific elements of the flash, so please produce methods for them." Fortunately, I have one in MooseX::Role::Parameterized. The consumer of this code looks like:

package MyApp::Catalyst::Plugin::FlashMethods;
# ABSTRACT: role to add flash-manipulation methods to Catalyst app

use Moose;

extends 'Catalyst::Plugin::Session';
with 'Catalyst::Plugin::Role::FlashMethods'
    => { flash_elements => [qw( message error )] };

1;

... and it gets enabled in my Catalyst application with:

use MyApp::Catalyst::Plugin::FlashMethods;
use Catalyst::Plugin::Session::State::Cookie;
use Catalyst::Plugin::Session::Store::FastMmap;

use Catalyst qw/
    -Debug
    ConfigLoader
    Static::Simple
    +MyApp::Catalyst::Plugin::FlashMethods
    Session::State::Cookie
    Session::Store::FastMmap
/;

... which consumes the parametric role:

package Catalyst::Plugin::Role::FlashMethods;

use MooseX::Role::Parameterized;

parameter 'flash_elements', isa => 'ArrayRef[Str]', required => 1;

role
{
    my $p = shift;

    for my $element (@{ $p->flash_elements })
    {
        method "add_${element}" => sub
        {
            my $self = shift;
            push @{ $self->flash->{ $element . 's' } }, @_;
        };
    }
};

1;

This code does have some naïve in it. In particular, it ought to use a smarter pluralization scheme. As well, the interface bothers me in two places. I'm not sure that the module's name is correct. I also don't particularly like that using this role means creating a new class of ~10 lines to make the code available as a Catalyst plugin (though how one might expand Catalyst's import() to understand how to reify a parametric role given the appropriate parameters is beyond my language design skills today).

Then again, my ~10 line plugin which consumes this role does work across my multiple Catalyst applications, at the cost of one more file and nine more lines of code altogether. The ratio of code per method decreases substantially with each new method I add, though.

I had originally intended to improve stash handling in this fashion, but but that's much more work. Neither Catalyst's core nor any other plugin I currently use rely on the presence of flash, so the scope of this work was much smaller. Even if they did, this approach does not prevent anything else from poking into the flash for good or ill.

Managing Unicode properly isn't exactly easy even in 2011.

Perl 5.14 makes Unicode somewhat easier with the optional unicode_strings feature, but you have to enable it explicitly, and you can only handle external data correctly if you know the intent of that external data.

(One of the small details I like in the book Gravitas, published by my company, is the documentation of the main character's struggles in one chapter with a Unicode bug exacerbated by one too many assumptions about characters versus bytes in his project's ORM. Art imitates life as satire.)

Tom Christiansen's Why Does Modern Perl avoid UTF-8 By Default missive is classic tchrist—clever, articulate, detailed, and a wave of text which crashes over the unsuspecting like a sneaker wave with a sinister undertow. If you're not careful, it'll lead you in a direction you never suspected.

You can see this when a smart person such as Nelson Minear claims that "Perl 5 can't handle Unicode properly". Aristotle caught his attention and Nelson offered a respectful retraction...

... but be careful not to miss the main point.

Handling Unicode appropriately is difficult, even in 2011. Many of Tom's very valid points are repeated reinforcements of the notion that your software, my software, everyone's software makes several assumptions about what incoming and outgoing data means. When those assumptions are wrong, you get bugs.

If 14 May 2010—the release date of Perl 5.14—had been Perl 5's Unicode flag day, such that perl assumed that all incoming data and all outgoing data were Unicode unless explicitly marked otherwise, Perl 5 programmers and users alike would discover exactly how many assumptions we've made. Some of them we can fix easily. Some of them we can't. Some of them require further fixes to the Perl 5 core itself, and some of them require operating system vendors and distributors to fix their own software.

This job isn't easy and it won't be quick.

I'm all for making progress and for making painful changes to improve the present and future for preset and future programmers, but the benefits have to outweigh the costs. Right now, they don't. Hopefully that day will come soon.

If you would like to enable UTF-8 everywhere in your Perl 5 programs, see Mike Doherty's utf8::all.

Good news!

We at Onyx Neon have decided to follow up to our successful Modern Perl: the Book with (at least) four more Perl books in the next year.

Chris Nehren is writing a guide to the modern CPAN based on Task::Kensho. These are the modules every practical Perl programmer should know.

Jess Robinson has agreed to write the definitive guide to DBIx::Class.

Kartik Thakore and the other SDL Perl hackers have a guide to multimedia programming and game development with Perl underway. We will have part of this book available for your perusal at YAPC::NA 2011.

I just started a Github repository for the Little Plack Book, an explanation of Plack and its ecosystem.

As is our habit, we're writing and editing these books in public. You're welcome to read along as we produce them, suggest new areas, fix bugs, and spread the word. We'll continue our very liberal redistribution policy as well when we reach the printed page.

(Moose fans do not despair! Lock some combination of Stevan Little, Chris Prather, Dave Rolsky, and myself in a room for an hour with the appropriate libations and we'll see what we can do.)

I just now rewrote this method:

sub save_with_fetchable
{
    my $self  = shift;
    my @dirty = map { $_ => $_->fetchable->is_changed ? $_->fetchable : () } @_
    $self->txn_do(sub { $_->insert_or_update() for @dirty });
}

sub save_with_fetchable
{
    my ($self, @items) = @_;

    $self->txn_do(sub
        {
            for my $item (@items)
            {
                $item->insert_or_update;
                my $fetchable = $item->fetchable;
                $fetchable->insert_or_update if $fetchable->is_changed;
            }
        }
    );
}

... after Devel::Cover complained that my tests didn't exercise the $_->fetchable->is_changed branch condition in all of its forms. The tests indeed do exercise that, but I rewrote the code anyway.

Novice testers introduced to test code coverage often go through wild gyrations to make sure that every part of their code gets 100% coverage. Complete coverage is a good rule of thumb if and only if your tests exercise the intent of the code completely. It's relatively easy to exercise every statement and branch and conditional while letting trivial bugs go untested and unfixed.

With that said, code that's easy to test tends to be easier to verify and to maintain.

In this case, rewriting compact code—clever code—into something simple enough for D::C to verify makes the code easier to read. The design of the code is slightly better. Add to that the fact that the module containing this code now has 100% test coverage, so that any subsequent changes of coverage will have an obvious tie to subsequent changes of the code, and this was an easy decision to make.

It's nice when improving test coverage fixes bugs, but it's even nicer when improving test coverage leads to improving code and its design.

Sometimes getting test coverage where it ought to be can be a little tedious, at least with a highly dynamic language. I have an object containing constant data and synthetic attributes calculated from the constant data. It has about a dozen of these synthetic attributes:

has [ @plusses, @minuses ], is => 'ro', lazy_build => 1;

I use this skeleton for my testing code:

#! perl

use Modern::Perl;
use Test::More;
use lib 't/lib';

__PACKAGE__->main( 'Some::Module::To::Test' );

sub main
{
    my ($test, $module) = @_;
    use_ok( $module ) or exit;

    $test->test_some_feature( $module );
    $test->test_another_feature( $module );
    $test->test_still_another_feature( $module );
    $test->test_one_remaining_feature( $module );

    done_testing();
}

sub default_args
{
    my $self = shift;

    return
    (
        default => 'Value',
        for     => 'example',
        @_
    );
}

sub test_some_feature
{
    my ($test, $module) = @_;
    my $obj             = $module->new( $test->default_args );

    ...
}

Ultimately I should port these tests to Test::Class, but I don't quite need it yet. That does leave some repetition in typing the boilerplate code I need for each test method. If I were using an IDE which could resolve these method calls without having to run code, I'd let autocomplete generate this code for me.

It's only a couple of extra steps in Perl though:

sub AUTOLOAD
{
    (my $name = our $AUTOLOAD) =~ s/main:://;
    print STDERR <<END_HERE;
sub $name
{
    my (\$test, \$module) = \@_;
    my \$score           = \$module->new( \$test->default_args );
}

END_HERE
}

With that AUTOLOAD in place, I can run the test, redirect STDERR to a file, then insert the contents of that file in the test file itself. It's silly and it's simple, but it works.

Have you ever read the documentation for Perl 5's wantarray? (It's perldoc -f wantarray.) I don't remember the first time I read it, but as far back as I can remember, it's said something like:

This function should have been named wantlist() instead.

The second edition of Programming Perl, published in 1996, says something similar.

At least fifteen years later, novices still get tripped up on the difference between arrays and lists. That's fifteen years of confusion and fifteen years of further entrenching the damage of having a core keyword with the wrong name, making the problem worse and the pain of fixing things worse.

(What would have been the problem of making an alias named wantlist and deprecating but not necessarily removing wantarray?)