Helping Perl Packagers Package Perl

| 7 Comments

I know I often shouldn't, but I use the Perl 5 installed through Ubuntu packages for most of my local development. I could maintain a parallel installation myself, but I have better things to do. (I do have bleadperl available if I need it.)

Every time I get a new machine, or perform an OS upgrade which changes the major version of Perl 5, I have to reconfigure the CPAN client to install distributions from the CPAN appropriately.

That's ridiculous, for two reasons.

First, the CPAN.pm configuration has traditionally asked too many questions. I understand that it's nice to have configurability and the ability to run on all sorts of platforms with odd behaviors and strange utilities and baffling constraints, but I also think it's plausible to assume that most new installations of Perl 5 on a modern Unix-like system can speak HTTP, for example.

The second problem is that the Perl 5 version has come from a system package — a .deb file, in my case — and the CPAN client prefers to install tarballs from the CPAN itself.

Even though Debian and Ubuntu packagers (especially the admirable Debian Perl Group) have made plenty of CPAN distributions available as .debs, I have to configure my CPAN client myself, and it does not work with the system package manager.

There's no reason it couldn't.

Imagine that the system Perl 5 included in the default package (or included an optional package) which had a CPAN client configured appropriately. It has selected an appropriate mirror (or uses the redirector). It knows about installation paths. It understands how to use LWP or wget or curl to download tarballs. It requires a make utility and the Perl 5 development headers.

Why stop there? There could be an alternate package (or an alternate Perl 5 installation or a program to switch paths) which set up local::lib for each user to install modules without overwriting the global installation.

Go another step further. A system such as Debian or Ubuntu or Fedora or one of the BSDs may include OS packages of CPAN distributions. If you want to install WWW::Mechanize, why can't a custom CPAN subclass translate that into a request to install libwww-mechanize-perl through the packaging system if it's available?

I realize that plenty of experienced Perl 5 developers dislike the idea of giving up control over every aspect of their own installations. That's fine. They can keep that control. Improving the defaults of the Perl 5 experience does not have to mean removing customization possibilities for experts.

The Perl 5 community has to produce at least three artifacts before this is possible:

  • An agreement that it's okay for distributions to customize their CPAN client configurations at installation time.
  • A set of guidelines for how to do so safely—probably backed up by code.
  • The will to improve the experience of installing, maintaining, and upgrading modules from CPAN distributions, especially for novices.

Each goal is achievable, though the latter likely requires the active Perl 5 community to refuse to support specific vendor customizations in any official capacity.

The result—a Perl 5 that's easier to develop, easier to begin, and exactly as customizable as it is now—is better for everyone.

7 Comments

Every few months, I look into a complete custom Perl 5 build for my Debian systems, and every few months, I realize the tools just aren't yet at the level required to create the kind of portable .deb packages (with full dependency information, etc.) that I want.

CPANPLUS (cpan2dist, rather) is moving in the right direction, I think, but it's still not mature enough for a sysadmin to rely entirely on it for Perl distribution.

On the other hand, I keep my home directory dot-files and other relatively "static" things in an SVN repository, so on a new system, I can just grab my ~/.cpan/CPAN/MyConfig.pm and get going. That's not too horrible, but I'd love to have my CPAN modules integrated into my systems in a more robust and automatic way.

Debian's dh-make-perl is wonderful. Its not perfect, but I find that most of the time I can replace "cpan Foo-Bar" with "dh-make-perl --install --cpan Foo-Bar" and it just works.

As you note, we mostly just need some defined glue to hook together tools that are already there. A CPAN subclass could replace or augment the various phases of the package search/fetch/build/install process would do it. eg instead of calling "make install" to do the install, it could instead call something like "dh-make-perl --install" to create a .deb and install it through the package manager.

Thinking on it more now, this really is the right thing to do. Downloading Perl from perl.org says that Perl is already included with your Linux distro. The docs say that CPAN is the right way to install modules. Right now this combination can lead to subtle breakages if you don't know what you're doing.

I've been reading your mind. We have been working on a system in GoboLinux to integrate CPAN and other domain-specific package managers with the distribution. It's been working out pretty well in testing and is about to be released properly (there is an RC out now).

CPAN is given control of its own directory tree, so that it won't ever interfere with or be interfered with by the system, and users can install CPAN packages using the standard method they're used to. As well, packages in the distribution package manager can use CPAN packages to satisfy their dependencies.

A dependency on "CPAN:XML::Writer >= 0.604" will look for XML::Writer in the CPAN installation, and then have it installed it if it isn't found. There is no repackaging or duplication between the system packages and CPAN. The idea is to try to embrace rather than extinguish the third-party package managers, since that helps nobody.

There is an overview of the system in the linked blog post. I also have a paper on this in the Distro Summit at linux.conf.au next month that explains it in more detail. I can put up a preprint if anyone is interested in it before then.

Gentoo has a pretty good system for this, called g-cpan: http://www.gentoo.org/proj/en/perl/g-cpan.xml

If there's a standard ebuild for a particular CPAN module, g-cpan will install it. If there isn't, it will generate an ebuild in a local Portage overlay on the fly. It's greatest feat is managing updates: g-cpan -u will tell you which CPAN modules have newer versions available and install them for you, cleaning away previous versions by virtue of the package manager.

Because the packages are registered under the g-cpan package namespace, it's also trivial to list and rebuild all of your local Perl modules when upgrading to new major versions of Perl.

To throw some wood on the fire: Being a sysadmin, I'd hate to use this. This would install the latest version of a lib and I don't want the latest and greatest package when I install something on my Debian production server, I want something that's been tested and to which (security) fixes get backported. As a sysadmin, I consider 'bleeding edge' a danger to any production system. I know developers like to have the latest available, but that's just asking for trouble if you're actually running a production environment that should be as stable as you can get it.

The nice thing about Debian packages is that they are monitored and fixes are backported. If a lib contains a problem, I only want to fix that problem, not include new features X, Y and Z because I had to upgrade to the latest version of the package to get the fix. Who knows what new problems X, Y and Z may bring along with them!

I can see why developers would want this, but it would be far better to take a look at the proposal Mark Shuttleworth made to the community: Have one version of a app/lib that is considered stable and to which bugfixes get backported for an extended period of time. Have apps depend on these kind of versions. Yes, that would mean you'll have to work around missing some functionality that the latest version of the lib added and that really made your life easier. But from a sysadmin point of view, that's what I'd want.

If something like that gets integrated in this idea, I'm all for it. Would save a lot of time building my own packages that I'd need to keep track of too.

Strawberry Perl already does part of this. It distributes a pre-configured CPAN::Config.

In the medium-term, I want to see CPAN.pm be able to bootstrap itself via HTTP, auto-configure itself and do the equivalent work of local::lib when the user doesn't have permissions to write to the perl library directories.

For what it's worth, the latest development version of CPAN.pm (which happens to also be included in Perl 5.11.3) already has a less verbose configuration routine and can automatically select mirrors for you.

-- David

One thing that might ease the life of those who package perl and its modules is for perl itself to settle on a build system - having to support three build systems (MakeMaker, Module::Install, and Module::Build) somewhat confuses the otherwise predictable CPAN.

There has often been discussion around a tool that will automatically install a perl module as a deb and insert it into the debian package database. CPANPLUS does this in fact, but among other things it lags behind development of dh-make-perl so it doesn't really create 'lintian clean' debian packages. The biggest value of debian however is the quality control you get from the distribution. Debian has security tracking, integrates perl into an OS with other dependencies, and offers some pretty good Quality Assurance. Bypassing these things would be suboptimal.

Should anyone want to work on further integration of dh-make-perl and CPAN, there is an unofficial git repo here: http://git.debian.org/?p=pkg-perl/apps/dh-make-perl.git;a=summary

Modern Perl: The Book

cover image for Modern Perl: the book

The best Perl Programmers read Modern Perl: The Book.

sponsored by the How to Make a Smoothie guide

Categories

Pages

About this Entry

This page contains a single entry by chromatic published on December 23, 2009 12:58 PM.

Safety in (Version) Numbers was the previous entry in this blog.

More Perl Packaging Possibilities is the next entry in this blog.

Find recent content on the main index or look in the archives to find all content.


Powered by the Perl programming language

what is programming?