Replacing the Standard Library with Distributions

Languages evolve.

Some people want to put closures in Java. (Set aside the fact that only 10% of the people I've seen talk about closures in Java know the difference between first-class functions and closures.)

C++ will eventually get a new specification.

COBOL has OO extensions.

Despite the fact that Microsoft can't seem to support C99 in Visual Studio (hey, give them a break -- they're still working on supporting PNG and CSS 1.x, two standards even older than C99), C# gets new features.

You don't have to use the new versions. You don't have to use the new features of the new versions. You don't have to like the new features of the new versions, and you don't even have to know they exist.

You have to delve deeply into delusional solipsism to deny that they exist... and that's merely language features. Consider all of the changes in libraries available for those languages.

In The Replaceable Batteries of Your Standard Library, I argued that change is even more constant when you consider the available libraries for a language. This is true of standard libraries blessed by the implementors (try using gets() in a modern C project and see the reactions you get) as well as libraries made popular by common use (remember Class::DBI?).

Change happens.

Language designers and developers and maintainers and patchers and porters and releasers ought to consider how to deal with the inevitability of change.

Should a new Perl project use CGI today? Plack seems like a better option. Should a new Perl project start with Perl 5's basic OO or use Moose? Is there any excuse to recommend the use of File::Find, when the basic API requires the use of features that 90% of Java programmers (in my unscientific and unrepresentative experience) do not understand?

The best argument for using CGI and bare-bones Perl OO and File::Find is that they're available on whichever version of Perl 5 you're likely to have installed (even if that's Perl 5.005, which has been obsolete for the whole of the 21st century and even if they have bugs which have been fixed in the past several years). The second best argument for using them is that there's plenty of example code which demonstrates how to use them (though if you write Perl 5 code based on examples you can find online... ouch).

The third best reason is that, once something is in a language's core library, only an act of the language designer can remove it. In theory, you'll get a lengthy period of deprecation warnings (if you pay attention to that sort of thing, but convince me that the people not using Moose and sticking with Perl 5.6.1 know how to get deprecation warnings from Perl 5). In practice, you're in trouble anyway, unless your language has a specification or a comprehensive test suite and you read it to figure out if specific behavior on which your code relies will always be present in the future.

In other words, change happens.

How can a language designer or release manager or maintainer or porter walk the thin line between providing a bare-bones language with few libraries and cramming everything into a core bursting with subtly-different alternatives and growing less relevant to real-world adept uses because of too-early blessings of approaches which time has proven inopportune for the common uses?

I've argued before that the right approach is a small core which contains only those libraries absolutely necessary to bootstrap an intelligent library loading system. This works for operating systems such as Gentoo GNU/Linux, Debian in its flavors, and many of the BSDs. Parrot is exploring a similar approach.

The Perl 5 core could do very well to include only a couple of libraries necessary to build and install Perl and to download (or otherwise build from a download or checkout) a bare-bones CPAN client. Perl 6 may do something very much like this.

That leaves one question unaddressed. What happens to those unfortunate novices who download the core distribution themselves and find out that they can't do much with it? What happens to people with cruel, cruel sysadmins who refuse to install anything other than the core?

The solution comes from realizing that the people who are best at maintaining and porting and releasing a new version of a language and its core are not necessarily the same people who are best at discovering what the language's community considers most useful, most usable, or most appropriate.

You can already see the solution in something like Strawberry Perl, which unabashedly includes a compiler and is on its way to including an IDE and a GUI toolkit -- not because you can't use Perl 5 effectively without any of those, but because Perl 5 is much more powerful and much more useful with them.

That line of reasoning is very similar to the one which includes any given library in the standard library.

Why not embrace it? Let distributions come up with their own lifecycles and deprecation and suppport policies. Let them identify which libraries people should have available by default for the next eighteen to twenty-four months. Let them perform the work of integration and testing and bug reporting and working with upstream authors.

Upstream would always remain the same. If you can access CPAN, you can ignore distributions or install your own additional packages or build your own distribution. (Strawberry makes its own tools available.)

A similar process works well for operating systems -- and, arguably, operating system users are less savvy than programmers. Many distributions already package Perl 5 this way.

Could you live with such a change?