March 2011 Archives

Every few weeks, someone asks me about the status of translations of Modern Perl: the book into languages other than English. Our official stance (publisher, editor, and author) is:

Please translate this book into your native language!

We ask three things. First, please coordinate with other native speakers, as far as that is possible. (If I need to add a page to the Onyx Neon book site to organize this, I will.) Second, please only add to the existing credits (you are very welcome to add yourselves). Third, please include a link to the existing book site in English.

Beyond that, we are very happy to increase the amount of good Perl documentation and tutorial material available to every programmer and potential programmer, regardless of native language.

One of my side projects is a small, subscription-based business built around a website. Obviously authentication and authorization are concerns. I've built the application using Plack, so I have middleware to separate the interesting concerns of authentication, authorization, and database selection from the mechanics of the application itself.

(I can troll people like Aristotle and Stevan and even Miyagawa by suggesting that Plack middleware and inversion of control can have much more in common with each other than you might imagine.)

While thinking about authentication today, I decided that the login system could use a standard HTTP authentication mechanism where, instead of typing a username and password into a little form on a web page, the client should pop up some sort of authentication box as a prompt.

My business partner scowled and said "People find that scary."

The difference between returning the appropriate HTTP headers to prompt a client for authorization and returning an HTML page with a form to fill out is minimal on the development side. How about the user experience?

Is prompting for HTTP authorization less friendly? A login form—not a popup—exists in the context of a site. It can have links to register or to recover a forgotten password.

The cleanliness (the HTTPfulness!) of the authorization header still appeals to me, however, even as I recognize that it may put off a lot of people. Alternately, deep linking into the application could use authorization, while visiting the front page could use the login form...

... but perhaps this is a case where cleverness or theoretical purity is much less important than ease of use and clarity to customers.

A business partner convinced me to work on a short-term Java web programming contract with him, and the details were right. (My most recent Java programming experience was an experimental project for a very large middleware company a couple of years ago that gave them a lot of information at the cost of a lot of frustration.)

The experience has been enlightening.

One bright spot is Java's notion of web deployment. It has its obvious flaws, but it has obvious benefits. You gather all of the artifacts necessary for your project: XML files, JAR files (bundles of Java libraries), web templates, the classes you've written, and other resources, all in the appropriate subdirectories governed partly by convention and partly by a manifest file. Your deployment process bundles everything up in to a WAR file (a web archive). Then you dump this WAR file into an application server.

Compare that to another project I delivered recently where I sent a CPAN-style distribution tarball to a (much more technically savvy) client, who set about installing dependencies from the CPAN (where Module::Install couldn't get the job done) and from yum, where you don't want CPAN installing, for example, PostgreSQL for you.

Granted, I could have used a tool such as Shipwright to bundle my code, but I didn't. For better or worse, CPAN-style distribution is awfully common.

Imagine instead if I could have bundled everything up into a single WAR-style file which a hypothetical Perl web application server could deploy.

Java achieves this in at least two ways. First, Java wants to own the world—Java does have bindings to native code, but the Java desire to have pure-Java everywhere tends to encourage people to write pure-Java code where Perl hackers might write XS. (XS means you have to compile native code on the target deployment platform.) Someday perhaps Perl 5 can reduce its reliance on XS.

Second, Java application servers also have the advantage of a stronger memory separation between different applications running on a single JVM. Some of this is due to the static nature of the Java language. When you can resolve all references within an application at load time, you don't have to look things up by strings in a global directory. (Of course, you do have to be clever in your VM design to allow runtime reflection, so the implementation of the VM is also important.)

Plack offers a way around this. Imagine a Perl app server based on Plack to which you can deploy multiple applications. The app server could start multiple independent backends—even so far as having multiple installations or at least library directories for each application, as necessary—to use Unix process separation to achieve this memory separation.

Such an approach would still likely use less memory than Java.

Make the Plack app server a reverse proxy capable of serving static files and SSL and redirecting invalid requests and you might have something really exciting. (Make it cloud aware and give me a 10% equity stake.)

These are idle musings, and the Java ghetto doesn't tempt me to ditch the world of Perl for a work life full of beans and mappers and inversion of control governed by there's-no-angle-bracket-shortage-in-this-sea-of-XML—but even in that mess, there are still some good ideas worth borrowing, buffing, and bettering.

Plack is one of the most important projects in the Perl world in the last two years. It's a good idea borrowed from Python's WSGI and Ruby's Rack but made Perlish; it's a simple formalizing of a pattern of web application development, where the entry point into the application is a function reference and the exit point is a tuple of header information and a response body.

That's it. That's as simple as it can be, and that simplicity deceives a lot of people who want to learn it.

Test::WWW::Mechanize::PSGI is one of many reasons why it matters.

Given a Plack application, you don't have to deploy to a web server—even locally—to test your application as if it were deployed. TWMP executes your application as if it were running via Plack (because it is!) and allows you to perform Test::WWW::Mechanize tests against it.

Certainly you need to test that you've deployed your application correctly when you do deploy it, but that's a single test.

Plack and TWMP (and Plack::Test) use the well-defined Plack pattern to make something which was previously difficult into something amazingly easy. They're not the first and they won't be the last, but they do demonstrate the value of Plack.

Joel Roth asks is violating encapsulation really that bad?.

In two words, it depends.

Encapsulation is merely a technique of structuring your program. The same goes for structured programming and design patterns and object orientation and functional programming and minimizing mutable state. These techniques exist (and good programmers practice them) to help programmers reason about the intent and activity of code.

If you had ten minutes to write a short program to save you a million dollars and you never had to touch that program again, never ever, you might make different design decisions than if you worked on a year long project with a dozen other programmers. If you're a capable and informed intelligent economic actor, this is because the reward of being able to reason about a system at every level from small to large governs the risk of unexpected interactions between unrelated components.

Ask yourself this: would you violate the encapsulation of your objects? Would you use global variables? Would you eschew lexical scoping in favor of global variables? Would you avoid function parameters and return values in favor of setting and reading global variables? Would you avoid functions altogether?

These design strategies exist along a continuum (just like scalability and performance strategies). You pick what you need when it's appropriate. If you're willing to bear the risk that your violation of encapsulation here or there makes maintaining your code more difficult in the future, at least make that choice with good information. (Again, if you don't need to maintain your software, do what's fastest.)

With that said, in the context of writing tutorials—especially for novice programmers—it's irresponsible to encourage people to write quick and dirty code with known risks. The experts writing tutorials have gone through the pain of maintaining and debugging and fixing plenty of code where the initial developer's gamble went awry. Tutorials should encourage novices away from well-known pitfalls. When they're no longer novices—when they really, truly understand the risks—they can do things the risky way.

A good friend of mine runs an orphanage in the Philippines. They live far, far from Manila on an outlying island, so their Internet access is a flaky 14.4kbps dialup line. Mike's an American citizen and most of his family lives here, so when he wants to share pictures or movies of his kids with their grandparents or aunts and uncles and cousins, he's stuck:

• Email (relatively) huge attachments to multiple family members
• Upload to a shared hosting space
• Upload to a social network
• Email attachments to a single family member who will redistribute them to others

Mike's situation is reasonably common among humanitarians outside of the digital world. The large documents they transfer might not only be photos of family; they might be agricultural manuals or recordings of native music or languages. In all cases, the distribution mechanism is a broadcast mechanism, from one source to multiple recipients.

Mike asked me the other day about using BitTorrent—where he could upload files to something which generates a private tracker which notifies his desired recipient list. From there, they use their own bandwidth to distribute the files. He has to upload them once over his slow connection (though the BitTorrent protocol could easily handle partial and resumed transfers).

All of the pieces of this system are available. Has anyone put them together? Sometimes it's important to build tools to help the world outside of the coastal urban clusters communicate effectively.

(Obligatory Perl: this is one reason I distribute the electronic versions of Modern Perl for free: $30 may be small barrier to a programmer in North America or Western Europe, but even $2 might be a large barrier to a programmer in a less developed country. They deserve great tutorials and references as well.)

In the discussion of A Design for Class Attributes in Perl 5.16 or Perl 5.18 (along with other discussions such as Modern Perl: the book), a meta-object protocol is, effectively, a design for an object system that it itself object oriented.

In other words, you have an object which represents the class of an object. To create a new class, you might write something like:

my $class = MetaClass->new( name => 'My::Class' );

You can add methods by calling add_method() and add attributes by calling add_attribute(). This is, in fact, what Class::MOP does in Perl 5, and what Parrot provides (not well, at the moment) to Rakudo Perl 6 and other hosted languages, and what a new MOP built for Rakudo Perl 6 does very well.

A good MOP has at least two important properties. First, it's a consistent and coherent underlying mechanism on which to build language features. Second, it's flexible and extensible.

The latter is even more important than it sounds at first.

Moose encourages the use of well-encapsulated, opaque objects by default, while blessed hash references might as well just be hash references, as anyone can poke and prod in them anytime. (Certainly you can do the same with Moose objects, but the documentation and design of Moose discourage you from even knowing that such a thing is possible.)

If the language syntax (and I've not mentioned them before now) allow you to specify a different metaclass—something other than Class—you can create objects with different behaviors and different storage of methods and attributes and other data trivially.

If the rest of the language, from the internals to all other extension mechanisms, interoperate in terms of the protocol of the MOP, these objects and classes and metaclasses will all behave and interoperate effectively.

In other words, if I were to propose a blue sky feature for Perl 5.16 without worrying about development resources or testing or feasibility of implementation or likelihood of merging into bleadperl, I would say this:

Beg, borrow, and steal a good MOP—perhaps the MOP from Rakudo Perl 6—and reimplement default Perl 5 OO in terms of that MOP.

Then let us discuss adding a class keyword and a method keyword and syntax for declaring attributes. Let's solve the biggest problem first: there's no reason to have multiple object systems in the core, not if they're slightly incompatible. As a bonus, we get much better introspection, dynamic programming, and reflection for free.

(If you're not sure how reimplementing blessed hashes in terms of a MOP makes things easier, consider this: anything which has the same duck-typed or interface-checked or, as is my wont, role-based design nature as the DefaultClass metaclass will be substitutable in any place you'd normally use DefaultClass, whether you use a MooseClass metaclass or a XSFriendlyClass metaclass or a SerializableClass metaclass or a ThreadSafeClass metaclass or what have you. The possibilities are... powerful.)

As the debate over revising the OO tutorials in the Perl 5 core continues, one point has come up as salient:

I am disappointed that after all this time we have no consensus on how to say in Perl "Class X has attribute Y" when so many other languages have solutions that have freed their users' minds up to move on to higher-level problems.

— Peter Scott, Re: Revising Perl's OO docs - a new OO tutorial.

Fortunately, Jesse Vincent opened the door to a discussion of something other than what's appropriate in core documentation:

I'd love to see proposals (soon) for how we might do this in the core for 5.16 or 5.18. I'm not promising I'll accept one of them, but I'd love to see us actually discuss such a thing.

— Jesse Vincent, Re: Revising Perl's OO docs - a new OO tutorial

This is a great opportunity for armchair language designers to address a notable feature gap in core Perl 5. Here are a few constraints I think any design must address:

• New syntax is necessary, though it should be optional (as much as I dislike the idea of feature.pm, it hasn't yet been deprecated, so you might as well use it).
• New syntax should be as compatible both with Perl 6 and Moose as possible.
• New syntax can be simple (that is to say, there's no need for 5.16 or 5.18 to include a full type system such as Moose provides) but should allow for extension (it's okay not to support method signatures as long as it's possible to add them in the future).
• New syntax should allow enhancement with something like Method::Signatures.
• The new features should encourage programmers to do the right thing by default without forbidding them to do other things when necessary.

You can get by without creating a MOP, but the real power comes from doing something more than fields—if you can declare a class, declare attributes, and declare methods without having to bless references or write your own constructor, accessors, and mutators, so much the better.

In the context of a (to me, rather silly) debate over improving Perl 5's object orientation tutorial, I staked out my opinion a year ago. The Modern Perl book assumes no knowledge of object orientation and introduces OO Perl 5 with Moose, favoring encapsulation, genericity, polymorphism, and roles and delegation and composition over inheritance. The subsequent chapter demonstrates the default Perl 5 mechanism of blessed references and explains that "You may have to maintain legacy code or write new code where CPAN is not an option, so here are the gory details."

In other words, I saw no good reason to expose novices to the core mechanism of object orientation.

With that said, you don't have to use Moose to write great Perl 5. You can write great Perl 5 with the default object system, though you'll have to write far too much code yourself and need great care and discipline to do so well.

To me, given the existence of one very, very good object system—and I double up the superlatives because it's so complete, from its theoretical underpinnings to its effective and almost seamless syncretism as well as working reflection which is horribly ugly where it's not straight up impossible with the default Perl 5 object system—and a handful of other good object systems on the CPAN, it would be irresponsible and even unethical of me to explain something else when the only reason to use the default object system is when you have to maintain legacy code.

Modern Perl is not about training a new generation of monkeys who can only maintain increasingly irrelevant legacy code. We can set our sights a fair bit higher.

Perhaps someday the Perl 5 core will have a better default object system (goodness knows it's difficult enough even to propose syntax extensions on which to build a better object system), but I sleep much better at night knowing that people who study the Modern Perl Moose chapter can write good code without tripping themselves up on the visibility and scoping of @ISA or worry about the differences between base and parent or paint themselves into a corner with single-argument bless or make refactoring difficult by eschewing encapsulation for direct attribute access or even writing lots and lots and lots of repetative boilerplate code, because I showed them a better option first.

They still have options if they need them, but at least they don't have to make ten years of mistakes on their own.

If I didn't have better things to do with my time, I'd write new documentation demonstrating how to write your own XML parser and propose its addition to the Perl 5 core documentation as pod/perlxml.pod. Sure, you shouldn't parse XML with regular expressions, but as Perl 5 doesn't include an XML parser and people want to parse XML with Perl 5, why not show them one awkward, fragile, and ill-advised way to do things without recommending to, as the world's most effective and capable Perl 5 programmers do, install a real XML parser from the CPAN.

(See also the debate over a proposal to revise the assemble-it-yourself, aren't-you-so-clever documentation for the minimal default Perl 5 object system borrowed wholeheartedly from Python. The mind boggles at the reticence to make things easier for novices and more consistent and clearer for the rest of us.)