Adding a Method Keyword to Perl 5

On p5p, David Golden suggested adding use feature 'method'; as a simple extension to Perl 5 which adds a method keyword to declare a method, then shift off the invocant and make it available within the body of the method.

I wrote a patch to enable method in feature.pm. The patch looks longer than it is. The important user-side example is in the simple test suite I added:

package SomeClass;

use strict;
use warnings;

use feature 'method';

sub new {
    my ($class, $value) = @_;
    bless \$value, $class;
}

method get_value {
    return $$self;
}

method dup_value
{
    return    $$self
    . reverse $$self;
}

method set_value :lvalue
{
    $$self;
}

method ctor
{
    my $value = shift;
    bless \$value, $self;
}

package main;

my $sc = SomeClass->new( 'instance variable' );

can_ok($sc, 'get_value');
can_ok('SomeClass', 'set_value');
is($sc->get_value, 'instance variable', 'simple method should work');
is($sc->dup_value, 'instance variableelbairav ecnatsni',
    'method using additional $self should work');
$sc->set_value = 'foo';
is($sc->get_value, 'foo', ':lvalue attribute should work');

$sc = SomeClass->ctor( 'method constructor' );
is($sc->get_value, 'method constructor',
    'method should work for constructor too');

done_testing();

As you can see, with use feature 'method'; in scope, the method keyword is a synonym for sub which implicitly adds a single line at the start of the body of the method: my $self = shift;.

Making this work meant adding method to the tokenizer as a keyword recognized only in the scope with use feature 'method'; enabled. That explains the patches to lib/feature.pm, keywords.h, perl_keyword.pl, and regen/keywords.pl, as well as the the second half of the changes to toke.c.

The first half of the changes to toke.c are a copy and paste and specialization of the special handling of the sub keyword. I had to do this for two reasons. First, the sub handling always returns a token representing SUB to the parser, and I needed to return something special. I chose the token METH instead. Second, it makes little sense to support prototypes on methods, as Perl 5 cannot resolve prototypes across method dispatch. I removed the portion of the sub tokenizing related to that. (This batch of code isn't an example of screechingly obviousness; it could become cleaner as I see more unnecessary code to remove.)

With all of that in place, the tokenizer provides a stream of tokens to the acutal parser in perly.y. The patch adds an alternation to one production and two supplementary productions.

The alternation to statement processing starts when the parser encounters the METH token. A method starts with meth, runs the special startsubparse production to set up the new subroutine scope, has a name, has an optional attribute list, and then has a method body. The latter is a new production.

A method body has curly braces delimiting it. It then runs a special new production called addimplicitshift (did I mention the phrase "screechingly obvious"?), runs an existing production called remember to start a new lexical scope, then gathers a list of statements, and finally encounters the ending curly brace.

The addimplicitshift production is the most interesting part of this whole process. It uses the core lex_stuff_pvs() function to add a single line of Perl 5 code: my $self = shift;. Perl will parse that line as if it had always been the first line of the body of the method, even though the parser added it itself.

This means that if you wrote:

use feature 'method';

method some_method
{
    my $self = shift;
    ...
}

... you'll get the warning you expect about a redeclaration of a lexical, with the correct the line number.

Does this work well with CPAN modules which declare their own method keywords? It's not incompatible with them, in the sense that the only code in the world which gets any use out of use feature 'method'; right now is the test in my patch. Yet something like MooseX::Declare may (I haven't tested it) be incompatible if and only if you use MooseX::Declare and use feature 'method' in the same scope.

A larger question is whether this is a valuable feature. Certainly it offers two modest advantages, by marking the intent of the developer to distinguish between a method and a function and by removing a line of boilerplate from every method. (Given that most of my methods are between three and ten lines long, saving double-digit percentages of their length is a tremendous savings of code.)

Will this feature get in Perl 5.14? I don't know. I have my doubts—the implementation could be cleaner, the ultimate syntax could change to use something other than my $self = shift;, and there's undoubtedly a long line of people who believe that the only way to prototype a proposed change to the Perl 5 core is to write a CPAN module (at which point a long line of people is happy to say "Why would you want that in core? There's already a CPAN module for it!").

Regardless of all of that, I can think of a fair few advantages to doing something like this in the core. I'll explain those tomorrow.