The Perl world often speaks of the DarkPAN. This is the CPAN's big brother. Where the CPAN contains millions of lines of freely-available, reusable, and tested Perl code, no one knows how big the DarkPAN is, what it contains, or if it's tested. Where there are CPAN testers, CPAN metrics, CPAN searching, comprehensive CPAN history, and CPAN code search and cross referencing, none of those features are available for the DarkPAN.
In short, we don't know where the DarkPAN is. We don't know what it contains. We don't know which versions of Perl it uses, which CPAN modules it uses, which XS functions it uses, which Perl idioms it uses, and which bugs it relies on going unfixed. It's a big wad of unknown that we suspect exists but can't measure.
The Perl 5 Porters have a fabulous resource named Andreas J. König, not coincidentally the maintainer of the CPAN module. Andreas often posts Bleadperl Breaks CPAN reports, where he identifies specific changes to Perl under development which have caused test failures in CPAN modules. At that point, the question is whether the change to bleadperl is wrong, or whether the module did something wrong. One or the other gets fixed.
That doesn't work for the DarkPAN. CPAN testing is automatable (proof: it's automated, and has been for years). DarkPAN testing is not. Every module uploaded to the CPAN and added to the index gets tested and analyzed and cross-referenced and, look at that, there's another piece of data added to the giant feedback loop which is community-driven software development. Every piece of DarkPAN code added stays in the dark, where none of this works...
... unless someone deliberately extracts relevant code from the DarkPAN and puts on the CPAN (which happens), or tracks bleadperl snapshots and testing releases and reports changes (which rarely happens), or files bugs after a release (which happens).
That's how community-driven software development works. If you're willing to work with the community in the open, we all benefit many times over.
There's one small problem, however. No one wants to break code in the DarkPAN. Note please that "break code" in the Perl 5 world is often a euphemism for establishing and following a deprecation policy. No one's advocating removing major features such as AUTOLOAD, as tempting as that might be. Yet the specter of the DarkPAN often arises in discussions where it's the final nuclear deterrant against adding a simple piece of syntax to simplify a task which is more laborious it ought to be in a language under development in 2009. "We don't know what it would break," they say, and that ends the argument.
When I write software which matters, I create a test suite. I choose dependencies carefully. I never upgrade it in production without running the tests. (Sometimes I run the test suite just for the fun of it.) The function of that code matters, so I'm cautious.
If I upgrade a dependency, I run the test suite. If I find a bug, I add it to the test suite. If I upgrade my version of Perl, I run the test suite. In short, I take responsibility for identifying and diagnosing all potentially functional changes to the behavior of my code even if they come from external resources. It's my responsibility as maintainer of that code to keep it running, and I know of no better way than to maintain a comprehensive, sane test suite to verify that it maintains the behavior I intended.
I expect the people to whom I distribute the software to do the same, so I include the test suite there. It's common sense software management. It's common sense operations.
Not so with the DarkPAN. Somehow, the Perl world has decided that this unmeasurable, unknowable conglomeration of code which may or may not exist is incapable of performing software and operations management to a minimal degree of competence, despite the fact that, for example, this is what operating system vendors do. This is what professional system administrators do.
Thus the burden of integration and deployment falls not to the people who know where and what DarkPAN software exists and how it works and what it uses, but to the least appropriate group of people: people who cannot possibly see it and can only guess at what it does, what it needs, how it does it, and if any of that is appropriate or supported.
I can imagine that if I called technical support for a product, I wouldn't get support if I refused to disclose any information about my problem -- or even if I actually had a problem.
Why should anyone expect differently from volunteers?
Lots of the DarkPAN code doesn't have comprehensive tests. Lots of the DarkPAN code was written for Perl 5.5.3 (or whatever). But you knew that ...
I think you underestimate how valuable it is the extend that perl Just Works. Getting broken code from upgrading core perl is just so obscenely rare; and that's really really nice. Imagine having a 200k line app with limited test coverage and considering upgrading to Python 3k.
Other than re-installing CPAN modules, upgrading Perl is extremely low risk. (Of course to some extend this value of low risk is eroded anyway by Perl upgrades often being done with CPAN module upgrades which aren't quite as graceful and incredibly robust)
Anyone who's written a 200k line app with limited test coverage is going to have problems upgrading anyway. How is that the core's responsibility?
Actually, for one company I worked at (with far more than 200k lines of code), we had a huge test suite and nonetheless failed to upgrade from 5.6 to 5.8. Part of the problem was that tests with XML serialization relied on Perl's deterministic sort order for hashes when creating XML attributes (whose order should not be relevant). 5.8 quite reasonably broke changed that to avoid algorithmic complexity attacks. Of course, despite the documentation warned of not relying on sort order, we had so many tests which broke because of that, when I finally fixed all of them (and a number of other issues), management never approved the upgrade. So many things appeared dependent on 5.6 that they weren't willing to take the chance of an upgrade.
No, I'm not arguing that this was the best decision, but given how many millions of dollars were riding on this, it's certainly not a surprising decision.
Ovid, I have more to write about that tomorrow. It might surprise some readers (though I believe not Ovid, who knows me better than this) to hear that I agree that this is a workable business decision.
It has to be an informed business decision, but I can imagine some circumstances in which it's the right choice.
There are many costs to that strategy, however, and I'm not sure that everyone is cognizant of them. Even in the free software world, a lot of people underestimate the costs of producing and maintaining software. I try to address that.
If you meant "anyone who has a 200k line app with limited test coverage is going to have problems anyway", I'd agree.
However, my point was that - so far anyway - the perl core has been so insanely stable and backwards compatible that breakage in reasonable code from core changes are exceedingly rare.
(Now, a core upgrade will often encourage if not require CPAN updates, too -- and the stability of those is a completely different matter).
- ask
It's not Perl, but a company I worked at was running (proprietary) software that had been unsupported for years. We hit the limits of what we could do with it and so were migrating away. I don't see why the same can't be done by users of Perl - if you need to use functionality in a newer version of Perl then you update your applications to run on it. If you're happy with your old environment then you keep the old Perl and everything keeps running as is.
The only people I think might have a problem with this are those with Perl running on servers they don't control (I'm thinking individuals with cheap web hosting) but in their case they'll just have to work things out with whoever administers the servers.