CPAN, Convergence, and Core

In Take Advantage of Modern Perl at YAPC::NA 2009, I mentioned a two-word mantra I keep in mind:

Seek convergence.

I'm glad to see Devel::Declare; it allows clever Perl 5 developers to add specific syntax to Perl 5 in ways that subroutine prototypes don't permit (MooseX::Declare may be my favorite) and it is safer and more debuggable than source filters.

It's not just for adding new features. It allows fixing existing features. In particular, Ash Berlin's TryCatch packages up almost all of the tricky little details required to handle exceptions safely and properly in Perl 5 in a syntax that's clear, concise, and difficult to get wrong.

Those are nice properties for a programming language feature.

CPAN

If you have access to the CPAN, you can use these modules today. You can mangle and manipulate and modify the syntax of Perl 5 you use on your project to add new features and to reduce the possibility of misusing existing features.

In one sense you can think of a language itself as an API. Certainly that's one way to look at Lisp, Scheme, Forth, Tcl, and Smalltalk.

In another sense, languages without the property of homoiconicity -- or languages not built up from a tiny set of primitives which allow you to define your own first-class citizens which appear to interact the same way as the built-in primitives -- make a clear distinction between language and library.

That's not necessarily a bad thing. The "DSL's Everywhere!" crowd will soon learn (individually) that language design isn't easy. Syntax and semantics and even partial formalisms require thought and experimentation and willingness to make changes and often fanatical attention to detail tempered by well-understood user requirements.

This is a roundabout way of saying that CPAN is not just a place to find libraries for building your own Jabber server or parsing FASTA data or calculating tidal cycles based on the Incan calendar. Some CPAN distributions help you write and maintain code (Perl::Critic, for example). These are tools.

Some CPAN distributions mangle the language itself. The SUPER module works around a pervasive feature in Perl 5's core method dispatch that often does the wrong thing in circumstances Perl 5's default object system makes easy. Moose gives Perl 5 a powerful object system based around a well-defined metaobject protocol. Want enhances caller() to report many other types of contexts. Coro adds a new type of control flow.

You know this; this is not a new insight. I mention them only to point out that there is a class of CPAN distribution which provides features that might apply to many, many programs. Their concerns have language scope: better exception handling, workarounds for core misfeatures, replacements for language-level features.

Hold that thought.

Convergence

Two of the most useful community developments I've seen in Perl 5 -- ever -- are DateTime and the Perl Email Project.

Both began out of frustration of a fragmented, wildly divergent problem area. Both required a lot of work to standardize on one good way to do things. Both required strong leadership and time. Both have produced powerful, usable, easy-to-recommend code.

(Both inflicted some birthing pains. You can't avoid that.)

A simpilar project exists today. The Extended Perl Core is an Enlightened Perl initiative intended to identify bundles of CPAN modules which provide features not currently in the Perl 5 core. These modules should represent the best code CPAN has to offer -- code which, while not perfect for every need, is suitable for at least 80% of the common cases.

This will be a lot of work. Not everyone will agree, where multiple alternatives exist. That's fine; that's healthy. Your needs are not the same as my needs. Alternatives will still exist.

Yet we can seek convergence on what's common and which of myriad alternatives provides the best default to start.

Keep that in mind too.

Core

Consider the cross-cutting concerns of certain CPAN code. Consider convergence. Consider the core.

I can't hide one of my desires for Perl 5: I'd like to identify some of these language-level concerns distributed on the CPAN and used widely and consider bringing them in to the language itself.

Maybe that's not the entire feature; maybe the core doesn't need all of Moose's MOP. Maybe it can't currently handle all of Coro. P5NCI isn't nearly stable enough, nor is it obviously the right approach for an equivalent to Python's wonderful ctypes.

Yet I believe it would be a shame for a language as flexible, as malleable, as pragmatic, and as open to experimentation as Perl 5 to encourage all of this evolution on the CPAN and in the wild without a language designer occasionally identifying pain points and bringing ideas and implementations back into the core language.

I'm not saying it's easy. I'm not saying it never happens. I'm saying that I believe this is a lovely goal for Perl development -- 5 and 6.