Designing for the Unanticipated

In 1987, not even Larry Wall could have predicted the Web. Perl's first wave of adoption was system administration. Perl's second wave of adoption was as a language for CGI programming.

My goal for Perl 6 is a language for general-purpose programming.

One of the design goals for Perl 5 was to remove arbitrary distinctions. Perl 4 was embeddable, but extending it for Oracle or Sybase or access to other database systems meant recompiling the Perl 4 binary. This is why you still occasionally see references to oraperl or sybperl.

Perl 5's greatest benefit is the development of the CPAN. That's not necessarily something Larry could have predicted in 1993 -- and certainly not in 1987 -- but designing Perl 5 for extensibility made the CPAN possible.

Fifteen years after the release of Perl 5.000, the Perl 5 community is still discovering ways to improve CPAN and the Perl 5 extension process.

That's normal. That's expectable. As a language designer, the best you can hope for is to encourage this kind of evolution and revolution. Mostly you have to hope that none of your design decisions actively prevent it. You also have to accept that you won't get it all right the first time.

I've written before that rapid, feedback-aware iteration is an effective way to understand a problem and find working solutions. Feedback from real users with real uses is invaluable (see also why Pugs and, later, Rakudo multiplied the velocity of Perl 6). You won't understand the problem fully until you've had to balance multiple, sometimes contradictory, constraints. Just as no battle plan survives contact with the enemy, no software design survives contact with users unscathed. You may find that you're painting yourself into a corner.

There's a balance: you don't want to cause users unnecessary pain by rapid, arbitrary change, but legacy code and features can hamstring a project.

(Sebastian Riedel's Version Numbers and Backwards Compatibility demonstrates a conundrum: how do you get this feedback early in a project's life when the CPAN is such a compelling distribution mechanism? The CPAN ecosystem provides few tools to manage rapid yet backwards-incompatible releases.)

That balance is especially difficult in language design, where consistency of expression and abstraction and concept is important. 361 contradictory Perl 6 RFCs demonstrate that Perl 5 has real flaws, but a language revision which adopted those RFCs in whole would be a mess.

A vision for the present and future of a software project is important, but that vision must also seek uniformity (where possible) and emphasize differences (where possible). Similar things should look similar and different things should look different. You see this in debates over operator overloading: the C programming language's polymorphic addition operator manages to add integers and floats. You can overload it in C++ to add matrices and irrational numbers. Yet some people rightly complain when someone else overloads it to append to files across a network.

I don't believe anyone really knows how to design a programming language that's sufficiently internally consistent that it's easy to begin, possible to master with minimal pain, well-optimized for its current domain strengths, and ready to take over the Next Big Niche.

I don't know many (successful) general purpose languages which haven't switched their focus more than once. I think "success" for a programming language means that people use it for purposes the designers never anticipated.

That's why discussions about Perl 5 deprecations, for example, frustrate me. I'm sympathetic to the desire not to break existing code (though people always have the option not to upgrade), but I want to remove arbitrary restrictions and smooth off rough edges and make programming tomorrow a little more joyful than programming today.

In other words, bring on Perl 5.11 and 5.12. 5.10.1 isn't the end of the line. I'm not stopping there. Bring on Rakudo Star and Parrot 2.0 and all of the intervening releases. I don't care if the software I rely on isn't "done", whatever that word means in this context. I care if it's useful, usable, and if it continually gets a little bit better based on feedback from real world users.

Sure, there's change involved. Certainly change produces pain sometimes. Yet without change, there's no progress -- and I don't believe we've arrived at the perfect language in which to write perfect software yet.