The "Big Bag of Arbitrary Metadata Pairs" Pattern

The singular trick of any extensible programming language is designing the capability for people to do arbitrary things the language designer never thought of, without tempting them into the Turing tarpit. (I know how to use continuation-passing style in assembly and how to write any loop in terms of compare and jump. Ask me if I ever want to do that again.)

Some languages do this better than others. Lisp's answer is "All code looks basically the same and code operates on code, so get used to manipulating trees." Forth's answer is "Push and pop are complements. Get to work." Tcl's answer is "Everything is a string, and we're happy to parse all the time." Smalltalk's answer is "Even conditionals are passed messages." Java's answer is "Look! The delegation pattern!"

Ruby's answer is "Every part of speech—including pronouns, articles, and gerunds if you're really cool—is a chained method call. Also, have you heard of blocks?"

I make no secret that I consider Moose a transformative technology in modern Perl. (The Modern Perl book even explains object oriented Perl using Moose before mentioning that you can bless references if you really want to.) Moose is really two and a half things:

• A set of much better defaults for an object system that encourage you to do the right thing by default
• A well designed metaobject protocol with great opportunity for core flexibility and extended customization
• A syntactic pattern to enable the first two items.

You can think of this design as three layers, and you're right. In practice, code written with Moose looks something like:

package MyApp::App::Role::HasRS;
# ABSTRACT: App role to represent ResultSets used in other App roles

use Modern::Perl;
use Moose::Role;

requires 'schema';

has [qw( update_rs analyze_rs )], is => 'ro', lazy_build => 1;

sub _build_update_rs  { shift->schema->resultset( 'Stock' ) }
sub _build_analyze_rs { shift->schema->resultset( 'Stock' ) }

around 'update_rs'  => \&reset_rs;
around 'analyze_rs' => \&reset_rs;

...

... where requires() marks an attribute as required during role composition, has() introduces attributes, and around() wraps existing methods.

Moose novices often get confused by the punctuation of this code. Moose itself exports requires, has(), around(), and a few other helper functions. These functions take a list of key/value pairs which influence what happens.

That's it. That's the magic. The syntax you see is merely convention which takes advantage of a couple of pieces of core Perl syntax. There's nothing special or magical going on here.

Okay, that's not true. There is magic, but it's not in the syntax. I've recently started to use HTML::FormHandler to great effect. HTML::FormHandler::Moose takes the Moose syntax one step further:

package WineCatalog::Form::Wine;
# ABSTRACT: the wine editing form

use Modern::Perl;
use HTML::FormHandler::Moose;
use namespace::autoclean;

extends 'WineCatalog::Form::Base';

my $start_year   = 1900;
my $current_year = (localtime)[5] + 1901;

has '+item_class', default => 'Wine';

has_field 'link', type => 'Text', label => 'link to the wine on your site';

has_field 'year', type           => 'Select',
                  label          => 'Year',
                  required       => 1,
                  options_method => \&year_options,
                  empty_select   => '-- Select a Year --';

has_field 'image';
has_field 'description', type     => 'Text',
                         label    => 'The long description of the wine',
                         required => 1;

has_field 'winetype', type         => 'Select',
                      required     => 1,
                      empty_select => '-- Select a Type --',
                      label        => 'Wine type',
                      messages     =>
                      {
                          select_invalid_value => 'Invalid wine type selected'
                      };

has_field 'submit', type => 'Submit', value => 'Add Wine!';

sub form_name { 'wine_edit_form' }

sub year_options
{
    [
        map { { value => $_, label => $_ } }
            reverse $start_year .. $current_year
    ];
}

__PACKAGE__->meta->make_immutable;

The result is a Moose object which allows you to be very specific about what happens with each field of the form: it maps to a database SELECT statement or accepts only a certain type or range of values or has a specific initial value or something else. You denote this with a mostly declarative syntax which should look very familiar to Moose users.

Mostly declarative?

Here's the drawback. has() and has_field() are function calls. Perl doesn't know anything about them. The attributes passed to these function calls are just values. Perl doesn't know anything else special about them. You're passing a list of values to functions. If you're a Perl programmer, you do this all day.

This shouldn't work as well as it does.

What happens if you mistype the name of a key in that big bag of arbitrary metadata pairs? Hopefully something somewhere validates it when the program starts. What happens if you accidentally duplicate a key? That depends on how the pair processor processes those pairs. What happens if you leave off a key or a value and you no longer have a list of pairs? Again, that depends on the implementation.

If you're extending Moose or something built on Moose, how do you register that you support a couple of new attributes? How do you register that you require the presence of a couple of attributes? How do you do so without conflicting with attributes of the same name that another extension uses?

If you're reading the code, how do you know which attributes are available? How do you know what they mean? How do you know if they're being used correctly?

If you're using an IDE or something with syntax highlighting, how do you know which attributes are available? How do you know which are correct? How do you know the difference between an arbitrary list of key/value pairs that's present only because it's expedient and one that's custom as in the case of Moose? (You can hardcode the popular ones, but that doesn't scale.)

If you're using an optimizer or a code scanner which can analyze your code and suggest improvements or perform automated refactorings or look for error conditions, where do you even start?

Again, this works well in practice. It works very well almost all of the time (Moose's proclivity to verbose stack traces notwithstanding). It probably shouldn't work as well as it does, but it does, and that's good.

The opportunity exists, however, to consider ways to improve this syntax even further. Perhaps allowing a file of parser hints would work. Perhaps finding ways to extend syntax or to perform partial precompilation would help. I'm not sure.

The fact that this technique works as well as it does is testament to several core features of Perl 5 which deserve respect, however: the arbitrary arity of @_, the importing of new symbols, and the identifier quoting of the lvalue of the infix fat comma operator. That's one way Perl 5 has stayed fresh into its second decade (and soon its third).