Why Perl 5 Needs a Metaobject Protocol

In Features Perl 5 Needs in 2012, I suggested five mostly-not-syntax features that Perl 5 could use right now.

The caveat to all of these features is that 1) someone must code them and 2) someone must maintain them. While p5p has traditionally done a lot with a little, remember that any suggestions or discussions will only make it into a Perl 5 release if enough effort goes into implementation to make them happen. If you want any of these features to happen, contact TPF about donations or volunteer your time and skills to help (even if all you feel comfortable doing is helping to refine the design of a feature, that's worthwhile).

I promised to explain these features in more detail. First up is a metaobject protocol for Perl 5. Stevan Little and Jesse Luehrs are working on a proof of concept MOP for Perl 5, so this feature is well underway.

If you're not familiar with a metaobject protocol, the book The Art of the Metaobject Protocol explains everything in extensive detail. (It's also a pretty good book about Common Lisp, so if you're interested in a semi-practical theoretical exploration of why computer sciencey things like the lambda calculus and well-designed extension mechanisms matter, this is a great book about the design and implementation of a language.)

If you don't want to read ~350 pages right now, the short definition of a MOP is an API for interacting with your object system. Classes (if your MOP supports them) are objects. You can define a new class by making method calls. You can create objects by making method calls. You can define an entirely new style of class by creating your own new metaclass with different behaviors. This is the reason Class::MOP predated Moose.

A good MOP provides a great theoretical foundation for a wonderful object system. Where many of us have defended Perl 5's object system as "minimal but effective and flexible", a good MOP in the core would let us get rid of bless in our code... and have more good features.

Performance Improvements

Moose can be slow to start, if you have lots of little classes, all on disk in their individual modules. Not only are you doing lots of IO to find and load those classes, but you have to build up all of the classes you've defined in memory through Moose's import mechanism (has() and extends() and with() are all function calls) which creates and populates a metaclass behind the scenes.

This happens every time you start your program.

Every MooseX extension you load changes things.

Though some of this happens in XS, where memory use can be smaller (less Perl data structure overhead) and speed can be faster (the same algorithm done in C instead of something written atop C), much of this happens at the Perl 5 level.

A great MOP could have its own custom data structures and code in the core to represent classes and methods and attributes, rather than Perl 5 hashes and arrays. A great MOP could have its own ops to handle things like introspection and dispatch. A great MOP could have parser support to avoid the overhead of calling Perl 5 functions (or crossing the XS boundary) to perform basic setup.

Simplicity of Alternate Object Systems

Perl 5's default object system is very flexible for strange people like me who've programmed in far too many languages (and implemented a few) who like various features of multiple languages and think "Hey, I know, I'll implement this in Perl 5." You bless a reference. You invoke a method. That's it. Everything else you can build yourself.

The downside is that you have to figure out how to build it yourself. Sometimes that means even diving into the source code of Perl 5 itself to see how things happen.

A good MOP provides a foundation on which you can provide your own custom behavior. As long as you hew to the protocol it defines, you can do anything you want: make a custom metaclass that allows simple and protected shared memory between workers, provide automatic auditing around specific methods, proxy transparently to remote resources, anything.

Better yet...

Interoperability Between Different Object Systems

One of the projects I maintain uses Moose heavily. One of its dependencies pulls in Moo. The authors of both object systems have gone to great lengths to make sure that both work together.

With a good MOP in place in the core, you have to go to great lengths not to interoperate well with other object systems. The default is interoperability.

(Try that in Perl 5 now, where some of the web programming or IO classes bless typeglobs. Oh, you expected a blessed hash? Too bad!)

Syntactic Improvements

As I alluded to earlier, a good MOP opens the door for declarative syntaxes for defining classes, methods, and attributes. We might finally get a class keyword and a method keyword. This is very nice because it'll be faster, but also...

Tool Improvements

... language keywords the language itself knows make their way into syntax highlighters and static analysis tools and reformatters and refactorers. It's easy to get a list of all of the methods in all of the classes in a file when it's unambiguous to pick out all of the names of classes in a file because you know they all have the class keyword in front of them, for example.

Compare that to today, where anything that looks like a coderef in a namespace is potentially a method or potentially a function and the only way to know is to invoke it, and why bother even trying?

Current Status

It's difficult to argue against putting a MOP in the core. Stevan and Jesse are working hard on it, but they could use more eyeballs on the tests and design and certainly more testing on various platforms. If you've been looking for an excuse to contribute to Perl 5.18 or Perl 5.20, this is a great place to start.