December 2010 Archives

At the end of the year, writers and editors want to take time off, so they throw together silly top n lists to fill pages quickly. I planned to write The Year 2010 in Perl, but garu's 2010 Filled with Perls did a great job for Perl in general, and Moritz's Perl 6 in 2010 covered Perl 6 very well.

Instead, here are the three most important stories in Perl from 2010, from my perspective:

• Perl 5.12 released on time, with little fuss or mess. The change to monthly development releases and yearly stable releases settled down to a well-understood project heartbeat. Perl 5.14 is a few months away, which should allow the graceful ending of support for all Perl 5 versions older than 5.10.1. Welcome to the age of modern Perl, everyone.
• Moose reached version 1.0. After several years of rapid development, all of the pieces were in place to replace Perl 5's default object system with something more powerful, more concise, more correct, and easier to use and to explain. Moose isn't quite the de facto object system in Perl 5 yet, but with ancillary projects adopting Moose-compatibility while providing additional features (quicker startup, less memory usage), it's clearly the right way to write modern OO Perl 5.
• The first Rakudo Perl 6 Star release came out. While various Perl 6 implementations had been available since 2003, Rakudo Star was the first distribution release intended for end users to play with. Development continues on the Perl 6 specification and Rakudo, but the monthly releases of Rakudo demonstrate that Perl 6 is useful and usable today and improving regularly.

There are plenty more great projects to mention—cpanminus, perlbrew, plack, Catamoose, CPAN testers upgrades, and a couple of good books—but these three are the most important in terms of the long term health and vitality of Perl. With them, the Perl community has a firm foundation for a great 2011.

One of the (many) reasons Modern Perl is free to redistribute in electronic forms is that I want to subvert hacker culture in a very Perlish way.

I believe that:

• Perl 5 is a powerful, modern language.
• Perl 5 scales from novices writing baby Perl to workaday programmers maintaining real programs to adepts creating new and amazing things.
• Perl has a design philosophy which expresses itself throughout syntax and semantics.
• Understanding that philosophy will help every Perl programmer — from novice to adept — improve.

Most of you believe that too.

I believe the latter so strongly that I devoted the first chapter of the book, the first 12 pages, to explaining everything you need to know to understand Perl after you've written Hello, world! I'm betting the book's reputation on that.

If you're struggling to learn Perl, or if you know people who are, or if you know people who don't quite understand to put it together, or if you know people who say "You know, I tried it, but it couldn't just stick in my head, all of that context and sigils and operators, why is it so confusing?" then try an experiment. Give them a copy of the book. Say "Read the philosophy chapter, right at the start. Really read it. Let it sink in."

My audacious, culture-subverting bet is this: someone who understands the two philosophical linguistic principles behind Perl's design can understand the rest of Perl.

Here's where you come in. Perhaps you buy another copy to give to a friend. Perhaps you convince your boss to buy a box of books, one for every developer. Perhaps you get Modern Perl in your local library. Perhaps you write an email which says "Hey, you could learn a lot from this book. Download Modern Perl for free."

I want reviewers, on blogs, on bookstore sites, on Perl monger mailing lists, on tech websites, wherever.

More than that, I want readers. I want to show off the great Renaissance of Perl we've perpetuated over the past decade, and I want Perl programmers working in isolation to join the community for mutual benefit, and I want novices to become adepts and to write great code with the rest of us.

Please, refer this book to a friend or five or fifty. It's a small thing, not like the gift of world peace or anything huge, but it's my gift to the Perl community and the world. Please help spread it far and wide.

Counting Perl Modules is half serious and half satire. I should know better than to publish satire. (Someday someone will invent an Internet populated only by people who know that Jonathan Swift was not the Western world's best known host of Donner dinner parties, but that hasn't happened yet.)

Getting an accurate count of the number of distinct modules on the CPAN is useful, but it's insufficient to compare the awesomeness of Perl to the fauxhawked machismo of Ruby to the humorless pseudocode-spewing Cylonification of Python to the cubist performance art of PHP.

Why?

Because what is a module and what does it represent?

In Perl 5, a module is a namespace. It may or may not contain code. It may or may not contain data. It may or may not exist in its own file. It may or may not be part of a bundle of other modules. It may or may not be useful on its own.

Count all of those up on the CPAN and compare them to the count of classes or packages or gems or eggs or copy and paste snippets for other languages, and you can learn... well, not nothing but not much useful.

You may as well compare the utility of (say) code written in Java to the same code written in Forth, despite the fact that Java's capability for abstraction begins and ends with the idea that everything in the world is a class or an instance of a class (except for those nasty primitives, and what are you, some sort of anti-business Smalltalk hippie anyway!) and that Forth's capability for abstraction is really good Forth programmers who create their own little languages that look and act like Forth but better, at least for their specific problem domains.

Oh, and a Perl 5 module may or may not be a class and it may or may not export a procedural interface and it may or may not be primarily documentation.

You may as well compare the goodness of two comparable modules chosen randomly from two languages with regard to use of standard language features, standard library features, and re-use of other modules, because ... well, it all depends, doesn't it? Sometimes the standard library is weak in one area, or one language lacks a feature of abstraction critical to another language, or there's clearly one really good way to do something in one language (Perl's DBI) and someday a standard might evolve and slowly spread through the ecosystem including a morass of hosting providers (PHP's database access layers).

... or what if one language has one more-or-less blessed way of doing something (Python's Django) and a small handful of alternate approaches or one obvious first choice to consider to do things (Perl's Catalyst) and a larger handful of worthy approaches or is in fact itself the primary way people do things (PHP) or overshadows other projects in the language so dramatically (Rails)?

... or what if one repository is primarily applications and one is reusable components, or one is a vast swath of untested abandonware ripped out of three-commit Git projects and another has an annual refresh rate of 30% and millions of test reports every year?

If you still insist on measuring your value as a person by comparing incomparables between your favorite language and the barbarian mutterings of those unwashed Philistines who {like sigils|hate sigils|love blocks|couldn't indent their code with a tab key and a flashlight|smell of body spray|know how to read documentation|haven't upgraded to a newer version yet|deprecated an insecure feature you relied on anyway|do it like Java|don't do it like Java|have a mustache|aren't Danish|aren't Dutch}, you might despair and consider returning to the time-honored practice of counting the number of mentions on the Internet of a random moniker sometimes and somehow related to the name of your languages—but resist that temptation, for That Is Obviously Bad Epistemology.

You might as well count the lines of code written in each language to find the bestest, most gratifyingest language in which only real, true programmers program.

I hope to see you all at Yet Another COBOL Conference 2011.

A new website modulecounts.com compares the number of extension modules for Perl, PHP, Python, and Ruby. Unfortunately, it's wrong.

That's not necessarily the fault of the website, but the graph and the extrapolation on the graph make Ruby's momentum with gems look all but unbeatable. That's wrong too.

CPAN's number (as of today) is 18936 modules, which corresponds closely with the number on www.cpan.org. You can forgive someone not closely associated with the Perl 5 community for thinking that number represents all of CPAN.

In truth, it represents the CPAN modules list—specifically the registered modules list. The important notice on that page says:

This Perl 5 Registered Module List document is currently not being maintained and is several years out of date.

search.cpan.org provides a much better set of numbers: 21585 distributions and 88698 modules. Almost 80% of the modules available on CPAN are not on the registered list.

What's the registered list? Back in the day, when you uploaded a new distribution on the CPAN, community standards suggested that you should choose a location in the module naming hierarchy and that you should describe your upload with a short code of metadata to indicate the support level, the type of interface, and the like. This was/is the DSLIP code.

Some experienced CPAN authors might even remember DSLIP codes; they're used even less often than the initial hierarchy of the CPAN index. Registering modules is much less useful than merely uploading them and letting people search for them by name and description with search.cpan.org.

To summarize, the module count on www.cpan.org represents a fraction of the available modules available on the CPAN because it only counts modules which the uploaders have bothered to register with the registered module list. You can write a lot of great modern Perl code while using unregistered modules. There's no correlation between appearance on the module list and quality or utility, except that most of the registered modules are likely older projects first uploaded when the registered module list and DSLIP codes were used more widely.

Any count of CPAN modules should use the numbers from search.cpan.org instead of cpan.org. I've submitted a pull request to update the sources of the CPAN module counts for modulecounts.com.

A client project stymied me for a little bit this week. One facet of the application revolves around user account creation and authentication. I'm using Catalyst for the web side and DBIC for the database side.

For now, I want to be able to create a user account, sanitize the information, and then immediate log in the user and redirect to another URI which requires user authentication. I use CatalystX::SimpleLogin to provide login/logout actions, as well as the DBIC password and session cookie options for Catalyst::Plugin::Authentication. With minimal changes to my database as originally planned, user passwords get hashed with Digest::SHA1 for an additional layer of paranoia.

CatalystX::SimpleLogin provides chained actions which will enforce login/logout to protect other actions which require authentication. I have a two base actions in the User controller which perform setup of the relevant Resultset, among other things:

sub base :Chained('/') :PathPart('users') :CaptureArgs(0)
{
    my ( $self, $c ) = @_;
    $c->stash( users_rs => $c->model( 'DB::User' ) );
}

sub authbase :Chained('/login/required') :PathPart('users') :CaptureArgs(0)
{
    my ( $self, $c ) = @_;
    $c->stash( users_rs => $c->model( 'DB::User' ) );
}

The action to create a user chains off of base, so it does not require authentication:

sub add :Chained('base') :PathPart('add') :Args(0)
{
    ...
}

After validating various parameters, it finally attempts to create the new user in the database:

    my $newuser = try
    {
        $users_rs->create( \%user_params )
    }
    catch
    {
        $c->stash(
            user   => \%user_params,
            errors => $_,
        );
        return;
    };

    return unless $newuser;

At this point, $newuser represents the new user. Here comes the fun part:

    # remove any credentials and force new authentication
    $c->logout();
    $c->authenticate({
        username => $params->{username},
        password => $params->{password},
    });

If there's an existing authorized user session, this code invalidates it, then authenticates with the new user's data. (An earlier version of this code used user data from $newuser itself, but I couldn't explain to myself why the SHA-1 password from $newuser->password should work.) There shouldn't be an existing user session, but $c->logout() is a cheap way to avoid trouble.

    # force simple auth to reauthenticate
    $c->change_session_id;
    $c->visit('/login/login');

If there's an existing session, the id should change to reflect the new authentication properties. Again, this is a cheap way to avoid trouble. The really interesting part of the code is the second line, which performs the login action, using the current-but-updated credentials, and immediately returns to the current action. This makes CatalystX::SimpleLogin do the right thing on the next request without the user knowing anything about a login form.

    # and display the profile page
    $c->res->redirect( $c->uri_for( 'profile', $newuser->id ) );
    $c->detach();
    return;

To avoid the double-POST problem, this action finally redirects to the "Hooray, you've created a new user!" action. This simple HTTP redirect ends up in the profile action:

sub profile :Chained('authbase') :Path :Args(1)
{
    ...
}

Because of the chain to authbase, anyone who visits the /usrs/profile/id URI must log in, whether manually through the CatalystX::SimpleLogin form or automatically after having created a new account.

At some point in the future I may need to change the application to perform email validation or admin approval of new accounts, but for now it seemed easiest to assume that users who create new accounts want to use those new accounts automatically.

(or They Get it Right in Version 5)

ECMAScript 5 finally has strict mode. JavaScript's strict mode forces static function scoping and variable declaration, enforces read-only properties, forbids often-confusing octal literal declarations, and variable name collisions.

In other words, strict mode will help identify common coding mishaps which can cause unintended consequences and painfully unnecessary debugging sessions. Disciplined and careful coders can probably get away with writing sloppy code, but strict mode helps everyone write slightly better code.

If this reminds you of Perl 5's strict pragma, perhaps in 16 years (Perl 5.000 released on 17 October 1994) ECMAScript's strict mode will be mostly uncontroversial among JavaScript programmers.

Christopher Bottoms sent me a nice note with a great idea the other day. He asked his local library to stock a copy of Modern Perl: The Book. They did.

Now anyone browsing the programming section has the opportunity to see Perl 5 in its full glory circa 2010, not the Perl 4-on-the-web mess that too many other programming books promote.

Here's where you come in.

What if everyone reading this put in a purchase request to the local public or school library for a copy of Modern Perl and Effective Perl Programming? (Enterprising readers and bibliophiles might even donate copies; Onyx Neon does that for the local library system.)

Certainly spreading around the electronic versions of these books far and wide (especially when available for free as is Modern Perl) helps the current generation of Perl novices, but we also need to reach people who aren't in touch with the wider Perl community and help them see how to write code well in 2010 and beyond.

Create something beautiful or amazing or useful or wonderful with Perl. You can create something for other programmers (Perl::Critic is one example), but you'll get better reception if you build something that non-programmers can appreciate.

For example, Onyx Neon's publishing process relies on a lot of Perl to produce our books. We use a lot of free software to make camera-ready copy to send to the printer. The average person who picks up one of our novels in the bookstore might never realize this, but it's a business advantage for us and an ethical thing.

The flexibility and power of Perl and the CPAN have only helped our workflow. (We even have Perl 6 code in a couple of places.) Perhaps we could publish great books without Perl, but using Perl helps us do our work on time and on budget with great results.

With Modern Perl available (ePub version coming soon) and my attention back on Using Perl 6, it's time to plan for the next wave of Perl books. Here's what I want:

• Modern Perl and the CPAN. The title needs work, but think of it as the successor to Modern Perl. Where I didn't cover all of the great CPAN modules that real programs written by great Perl 5 programmers use all the time, this book has room for them.
• Modern Perl OO with Moose. I've reached for where this book should be on my shelf far too often lately. You know you want this too. I have just the authors in mind too.
• DBIx::Class. It's time.
• Modern Perl Web Development with Plack. This book could be short, very short, but no less useful for its size. It almost writes itself, given a good outline.
• wxPerl, because I've never made sense of the C++ documentation.Game Programming with SDL and Perl. Here's a long shot, but Kartik and company have done far more with SDL_Perl than I ever have, making multimedia development far more convenient and far prettier than ever before.
• Building Little Languages on Parrot with NQP, because NQP needs documentation and Parrot's world domination plan requires it.

I have ISBNs at the ready and contracts to send. I believe I have audiences. (Please confirm!) Now I need authors.

That's my list. What's yours?

Perl 5's recent improvements to project management (migration to a public and distributed source code management system, monthly development releases, and yearly stable releases) have spurred on other improvements. For example, I've decided that I'm looking forward to implicit dereferencing from prototyped operators in Perl 5.14.

Some of the other improvements are less obvious and less dramatic. Chas. Owens has been revising perldoc perlop to improve its accuracy and organization. Much of the core documentation has accreted over the years—not just sundry and (in the gestalt) jarring additions, but various programming styles as well.

I've submitted patches at various times to replace certain specific deprecated code styles with improvements. I'm not the only one. Granted, it takes little work to convince a committer that removing examples of pseudohashes, 5.005 threads, or indirect constructor calls can improve the documentation for people using 5.12, but even such patches are not without controversy.

A recent thread on p5p started with a patch which went further. One spot of controversy was the replacement of global bareword filehandles with lexical filehandles. Another point of contention is the value of declaring all variables as lexicals.

One philosophical stand suggests that the documentation for any given version of Perl 5 should reflect the best practices as understood for that version of Perl 5. In other words, if Perl 5 users should prefer lexical filehandles over local-scoped globs in 2010, the documentation in 2010 should do so also.

One objection to this idea is that the documentation will be in a constant state of churn as some, many, or most (pick how fat you stuff your straw man) best practices are mere fashion. You can imagine, some argue, strewing references to inside-out objects throughout the core documentation in 2006 or 2007. How silly would that look now?

The other philosophical position suggests that the consistency of a single coding standard throughout the thousands of pages of documentation is a foolish hobgoblin's goal, that any example (whether written in the Perl 4 or 1 days) that remains working in bleadperl does not need patching, thank you very much, and that novices benefit from the very expression of TIMTOWTDI in the documentation.

The strawman objection to that argument is that that argument is basically "Don't tell me I'm coding wrong!"

Me, I'm somewhere in between. Critics often complain that coding Perl 5 well means understanding every possible way that someone might code something. That's obviously false on its face, but TIMTOWTDI is a pragmatic admission that no single WTDI is obviously superior in every case. (Want to start a bar fight? Ask three different people when it's okay to use unless.)

Even so, I think you also have to be foolish (if not outright malicious) to suggest that novices should prefer things like global filehandles to lexical filehandles. Everyone knows that whatever's in the documentation gets copied and pasted and tweaked until it works, and if the original source has bugs or malfeasances or security errors or subtle tricks that will only hurt the coder several months down the road... well, perhaps the documentation should aim a little higher than that.

You won't catch me producing a bleadperl-wide vertical alignment patch, nor will you see me slapping my on all variables in even a single POD file. Yet improving the example Perl 5 code which will get repeated ad infinitum in the world is a good goal, and no piece of existing documentation is so sacrosanct that it does not bear the critical eye of someone asking "Is this the best way to write this example with Perl 5.12 or Perl 5.14?"