Context Awareness in a Powerful Perl IDE

Some people complain that the motivation behind modern Perl is "to turn Perl into Java!" I don't understand that; any reasonably sized and well-written Perl program takes advantage of Perl idioms that the Java language cannot (and will likely never be able to) express.

They're right in one respect though: I'm happy to admit that like all mature poets, I steal good ideas wherever I can find them. Today's idea comes from Smalltalk.

A Digression on Context

The idea of context -- not just automatic data coersion between stringy and numeric and boolean and reference and (if you've disabled strict 'refs') executable values -- sets Perl apart from almost every other programming language. I explained that there are static and dynamic views of contexts in Tiny, Void Context Core Optimizations. These form the other axis of context-awareness: what do you expect to do with the result of an evaluated expression?

Expression context confuses novices. You can see it in the documentation of the return builtin and all best-practices discussions that suggest a bare return; over return undef;.

You also see it when people call functions in hash initializer lists:

my %big_bundle_of_data =
(
    name => $name,
    # oops!  don't do this
    id   => $q->param( 'search_id' ),
);

The problem is that the list used to initialize the hash enforces list context to the method call. If that method returns a list of items in list context, the hash may not contain what you expect.

Reflection and Introspection

One of Smalltalk's greatest strengths is the use of images. (That's also one of Smalltalk's greatest vulnerabilities; owning the world is good when you can own the world, but if something else starts the own the world, you should try to play nicely too.) You write Smalltalk programs in the Smalltalk browser (not a web browser, but an IDE) written in Smalltalk itself. Some Lisps do this too. You can consider Emacs a Lisp implementation which happens to run an editor to write more Lisp code, for example.

When you write Smalltalk this way, you edit parts of your program as it runs. The IDE is part of your program (or vice versa).

This offers tremendous power. If you want to list all of the objects or classes or methods in your program, you can do that. They're all available. If you want to perform exploratory programming, you can do that: ask the debugger to halt when it reaches a method you haven't yet implemented. Write a test, write a bit of code, run the test, see if you need that stub method.

There's a reason test-driven development came from the Smalltalk world.

Static versus Dynamic

The noisy discussion earlier this year about parsing Perl 5 was all about whether you can determine exactly what a Perl 5 program will do, parse-wise, without executing any of its code. In the Smalltalk model, you've already executed the code so you already know how it parses. You can even change that. (You can even rewrite Smalltalk's garbage collector in Smalltalk, if you want to do that. See also Parrot Lorito.

The B::Concise output in the discussion of my context patch demonstrated that the Perl 5 optree contains information about the context of expressions (in op form) before runtime begins. That information is available through the B modules while a program runs. You can find the context of any op by careful use of the B::* modules: pass a subref, get its ops, walk the COPs and look for the right file and line number, and follow the sibling pointers until you reach the right op. Then look at its flag.

This is all possible. It's tedious, but not difficult. Yet few people know it's possible -- and to my knowledge, no one has done it.

Yet imagine the power and the ease of learning and the potential for finding more error conditions if a powerful Perl IDE had the option of identifying the static context for any Perl expression. The code snippet I demonstrated earlier could pop up a warning window explaining exactly what might go wrong and suggesting the judicious use of scalar.

Perl::Critic can identify many of these cases. It's a great tool. Use it! Yet it can't identify all cases.

One of the important goals for Perl 6 is to make tooling like this possible. Separating the execution model from the tree model used to identify valid programs -- and making those trees available to tools -- will help. Gerard Goosen's TPF grant proposal for building ASTs for Perl 5 has similar possibilities. Hopefully it can get funding next time; it's a good project.

We may not get powerful Perl tools like this any time soon for Perl 5, but isn't it nice to imagine the possibilities? Maybe that'll make Perl 5 like Java, in the sense that Java has some powerful tools. Would that be such a bad thing?