Once in a while, an innocent looking change to bleadperl (the version of Perl 5 under current development) causes changes which ripple through the CPAN. As the CPAN is a graph of dependencies, any such change which causes tests to fail could have dramatic effects on user applications.
(Once I almost released a change which would have made half of CPAN uninstallable. Then Schwern slapped my hand metaphorically.)
Sometimes the fault isn't in bleadperl.
Consider RT #106538, which laments the inconsistency between the output of the builtin die and Carp's croak(). From the bug report:
$ perl -e 'die'
Died at -e line 1.
$ perl -MCarp -e 'croak Died'
Died at -e line 1If your eyes don't immediately catch the missing period, you're in good company.
Consistency suggests that the output of both error messages should be
identical. After all, Carp exists to enhance Perl 5's core
exception mechanisms.
Yet as you might expect, changing error messages breaks buggy code that attempts to parse unstructured text too strictly. Adding a single dot to an error message makes several important CPAN modules fail their tests.
I can't blame CPAN developers for performing exact matches against string error messages—it's quick and easy and unlikely to change, and it's reasonably easy to fix... until you get a fix that looks like:
$pattern .= $Carp::VERSION gt "1.24" ? "." :"";... which knows that the period is present but persist in hard-coding specific formatting details of the output.
The right solution, of course, is to stop emitting only unstructured text (from the core side) and to stop testing the exact details of unstructured text (on the CPAN side). The interim solution is to stop testing the exact details of unstructured text on the CPAN side.
Despite all of the effort around what could have been a simple change, the entire process of developing Perl 5 is a huge improvement over its past. Making this change and identifying its effects was reasonably easy, when you consider the size of the task and its consequences. Sure, the entire Perl community has to pay off some of the technical debt for well-established choices and design decisions that turned out to have been less than perfect, but this is a good opportunity to see how much better things are than they were even five years ago and to reflect on how to improve processes and tools to make them better as early as next year.
Do keep in mind, however, that if you're performing exact string matches against the results of things the core has never promised not to change, you are writing risky code.
Elephant in the room... You can't ignore the "exact detauls" of the irony in that typo, in the context of this post.
Very good post however! Playing the devil's advocate though, where would one find a list of things that the core has promised not to change? That kind of thing would be very handy for testing test suites.
I think more than anything else right now I wish that Core perl threw exception objects and that we could get a real try, catch, finally, throw, set of synax's. No more string parsing, real exception objects.
To my knowledge, no formal promise exists. You can go a long way by looking at the test suite and documentation; undocumented behavior is always risky.
rjbs has suggested that he's very favorable to exception objects. They're doable without changing or adding syntax.