Are you a good Perl 5 hacker?
Do you know how to access the symbol table yourself, without resorting to string eval? Do you know how to find all of the variables within a package or all functions within a package?
This metaprogramming—or reflection—is a fundamental part of higher programming languages. While all symbolic languages intend to operate on parametric symbols instead of raw data (that's why you can write equations in programming languages instead of using your computer as a glorified adding machine), higher languages operate on language constructs as well.
You can see this in lower languages such as Java, where the only good way to make flexible and extensible programs is to generate big wads of XML which control your AbstractFactoryInterface instantiations such that the classloader can inject the right code to make the language do what you need without forcing you to write even bigger wads of repetitive boilerplate Java code.
(During a consulting job several years ago I had to generate 60+ unique reports with Crystal Reports. The reports differed by way of two column names, but Crystal Reports wasn't flexible enough to let me define one report once, then pass in the two parameters I needed. This is why 4GLs failed.)
Metaprogramming and parameterization and flexibility beyond "This is how you write programs in this language, darn it, and you shall never think of doing anything different that the language designer didn't have in mind." may be difficult to implement well and it may not completely satisfy everyone, but it's useful and necessary.
Consider Package::Stash. If you couldn't remember all of the answers to all of the questions in the first paragraph, you'll probably find P::S much easier. (Certainly it's less buggy and much less annoying than munging the symbol table by hand.) Use it when you need it. It's nice. It's Perlish. It doesn't look like the dog's breakfast of random punctuation that you need to manipulate symbol tables (which are even uglier than the standard dereferencing syntax, because they're special hashes).
Now that I've explaind a bit of programming language design philosophy, explained a dead end in programming language design, and pointed you to a useful tool from the CPAN, let me get to the point:
There's no excuse for something like Package::Stash not
being core behavior of Perl 5.
Think about that for a moment.
How can you defend Perl 5 as a practical language, one which makes easy and hard things possible, when standard higher language behaviors such as symbol manipulation are so eye-gougingly ugly and esoteric by syntax that they are out of reach of even adept Perl hackers? These are not black magic concepts (else MIT wouldn't have taught freshmen Scheme as a first language). They are fundamental to a proper understanding of symbolic computing and practical programming.
I know the arguments against subsuming something like P::S in the core though, and they're all nonsense.
- You can already do all of these things with the existing syntax. You can also store relational data in flat files instead of an RDBMS. That doesn't justify anything.
- Not everyone will like this syntax. The core should come up with something everybody like. Nonsense; the core should provide one really good way to do something (and the "They're just hash references with special keys and some other bits glued on!" approach isn't it) without precluding alternate approaches. Refusing to do something without complete consensus is as much a recipe for failure as adding every feature proposed without considering its necessity, utility, implementation, and coherence in the gestalt.
- What about all of the existing code that doesn't use it? Irrelevant; it still works. That code's already written. The cost of writing that code is a sunk cost; you'll never get it back, no matter how much you penalize code under current maintenance and new code written from this point forward.
There are plenty of dependencies of popular CPAN distributions which do not represent failures of the Perl 5 core to provide useful features in a usable way, but consider this: every time you have to install or load one of several competing exception or function signature or list handling or library management or build systems or FFI workalikes or object system modules in a large chain of dependencies (of which a transient bug in any costs you debugging time), you're paying a tax the core developers have levied on all users in lieu of subsuming useful and necessary language features into the core of Perl 5 itself.
Things have improved in the past ten years (and especially the last 18 months), but this is why non-Perl programmers look at Perl 5 and say "Modern? That's what you call modern?" All of the rounded corners and hip slogans and fauxhawks and body spray in the world won't change that.
Of course, it's difficult to criticize p5p for its conservatism: no one has stepped forward with a coherent and singular vision for the continued and managed improvement of the design and implementation of Perl 5 as Perl 5—nor is it clear that p5p would accept anyone other than Larry doing so.
Is there a solution?
Note that a solution wasn't found, but we had a very similar discussion on this at Frozen Perl on Saturday. :)
can we make 'use boolean;' core first... or better yet remove the need for it at all by adding true/false keywords to the language... but since that won't happen...
Well, I can do that in Java without all those big xml filess. There are annotations, there is Guice that allows me to get rid of the SuperAbstractFactory.
I really don't get what you're writing about. For me properly used dependency injection in Java looks much more clearer than playing with the magical symbols. I had to play with that once, it wasn't nice. But I will try the module you write about, maybe it will be nicer this time.
You're right; I shouldn't say that's the only way.
The fact remains, though--without addons, Java provides no core mechanism of dealing with its lack of metaprogramming abstraction.
I personally think that modules belonging in 'core' ought to serve only one purpose: making it easier and more automatic for users of Perl to extend their environment and set up their applications. In this regard, CPAN.pm and Net::FTP and lwp are all clearly necessary to that end. As are the necessary build and test frameworks (MB/EUMM, Test::*, etc). I also consider local::lib an exceedingly worthy addition.
My rationale is simple and fairly obvious: If you have just enough to download and install code from the CPAN without hassle, then everything else becomes trivial (yes, FSVO trivial) - or political. (some may argue that those are the same thing!)
Certainly there's great value in having consensus on modules and extensions that improve the language - for example, picking what should be covered in books, or recommended on IRC or StackOverflow - but must that consensus come about by bundling it all up in core? How about putting it all on the CPAN? Like Task::Kensho, perhaps Task::Core - there could even be different releases for different versions of perl T::C::5_10_1, etc...
BTW, P::S looks like great stuff. I didn't know about it until this article, but now that I've seen it, it has earned a place in my toolbox. Hopefully my co-workers will no longer write and read code that messes with the symbol table and make assumptions about its purpose - or have to bug me to explain! ;-)
@stephen -- See my recent article: Perl 5.14 will include HTTP::Tiny and CPAN.pm will be able to bootstrap local::lib direct from CPAN.
@chromatic -- generally, I agree with your sentiments, but I think you're painting this one with too broad a brush. Every module added to core adds a tax on the core maintainers, too. Putting modules in core just moves the cross-platform dependency management upstream.
Half your argument seems to boil down to the classic debate between a "tiny Perl core" vs a "kitchen sink Perl core". Last I read, Jesse's point of view seems to be that we should aim for that to be a configurable option eventually. Strawberry Perl is a good example of what "Perl core plus useful stuff" can be.
The secondary argument seems to be about which ugly things made easier with modules should really be made easy as part of the core language (as opposed to just shipped as modules with the core). There, I tend to agree with you for low-level stuff, but question whether it's really the right thing for higher-level abstractions.
For example, if Perl had some sort of default object system built in, would we ever have gotten Moose? The lack of one way to do it does allow a degree of innovation and experimentation that can lead to better things over time, and it's hard to change something that is built into the core language. In that sense, having things on CPAN makes dependency management easier, because it's independent of any version of Perl itself.
I would like to see the Perl core make metaprogramming easier and to provide better low-level features for things that are really sub-par (e.g. exception handling) plus make it really, really easy to install things from CPAN (which I think we'll have as of 5.14). I think that's the best tradeoff between "all in core" and "all on CPAN".
-- dagolden
We can both agree that adding Package::Stash as a core library is inappropriate.
As for adding the equivalent feature to the core, consider the fragmentation of Lisp prior to Common Lisp (and even cross-platform features of CL implementations post-CL) as well as the pre-8.0 fragmentation of Tcl as warning signs.
Sometimes you have to provide one good way to do most things without excluding the possibility of doing other things. The art of language design is picking the correct set of core axioms. With a good core MOP, the Perl 5 core wouldn't have to provide all of Moose but could make the basic, most-used features of Moose available and the more exotic features of Moose possible.
@dagolden++ many times over for your contributions! I was indeed thinking of those when I commented!
@chromatic++ indeed - I also think the thoughtful addition of new language features (like new functions/keywords for more easily manipulating the symbol table) are indeed important, and can help perl keep its edge. I also heartily agree that finding and adding the most primitive pieces necessary to support an evolving MOP would also be something great.
Of course, there are two things that I am loathe to sacrifice much of (if at all) - backwards compatibility and performance. However, I have enormous faith in the community and the folks in p5p that those are highly on their minds - and looking at the last two years - no longer paralyzingly so.
@dagolden++ again for the new dereferencing that is in blead - I am practically drooling at the thought that I might get to use that in production.
My current employer is somewhat conservative about upgrading perl but I truly hope that I can get them to move to 5.12.x or even 5.14 in the next two years. We're still on 5.8.x globally, and with tons of legacy code. I've already managed a minor victory in that I've now got Moose in our platform and better yet the software I have re-written using it is several times *faster* than its predecessor!