I like to cook. I hate washing dishes. Sometimes I let dishes pile up in my sink and on my countertop. When I stagger downstairs for breakfast, sometimes I have no clean dishes, and I have to rearrange the dirty dishes in the sink to find dishes to wash and to make room to wash them.
Sometimes I'm good, and I put my dirty dishes in the dishwasher immediately. My sink's always empty, so I can use it if necessary, and my cupboards are full of clean dishes and pots and pans and other cooking utensils. I don't have to set aside extra time to get my kitchen in shape before I can cook.
A few of my friends run marathons. They train every day for several months, though usually not the full twenty-six-and-a-half miles on every run. They don't show up on race day having run not at all in the past year, expecting to finish. They put in a little bit of work every day to prepare.
I write novels as a hobby. My goal is to write a thousand words every day. That takes somewhere between half an hour and an hour and a half every day. In a year, I can write two novels at this pace. It's possible to write a novel in a month, but that's some three thousand words every day. A professional writer with a great outline, a good sense of character, and the discipline that comes from months or years of writing every day could make that work, but not so anyone else.
When I write code, I run my tests every time I make a change to the code I'm testing. Sometimes that's every few seconds. I could run my tests only when preparing to release a new version, but every time I've done that, I've spent hours debugging problems that would have taken me seconds to fix if I could trace them down to the single line of code I changed since my last successful test run.
In all of these endeavors, one step at a time adds up to big progress over time. Writing one word isn't difficult. Washing one dish is easy. Fixing one failing test is trivial. Running one mile is much less daunting than running twenty six. This isn't a new thought. It's barely an interesting thought.
I wonder, though -- what is so different about upgrading to a new release of software every month that's so difficult the only way to do it is in a long-delayed, long-planned, big thud upgrade every two or three or five or ten years?
It's not as if I wash any fewer dishes one at a time than all at once, or run any fewer steps one mile at a time than twenty six at a time, or write any fewer words a thousand at a time than a hundred thousand at a time, or write any less code two and three lines at a time than two or three hundred at a time. The amount of work accomplished tends to scale linearly. Maybe upgrades seem daunting and weird and difficult because big bang all at once once in a blue moon upgrades are simply too big to be anything other than difficult. Maybe small, frequent upgrades can be so boring and dull that they're as easy as putting your spoon in the dishwasher after you finish eating.
Certainly with Parrot we've discovered that releasing a new stable version every month is much easier than releasing a new stable version only once or twice a year. It's documented. It's automated. It's repeatable. It takes some time, but it's fairly boring how uncomplicated it is.
What makes the other side of the process -- users upgrading -- any more difficult?
Users are practicing "If-it-aint-broke-don't-fix-it". An incremental release may not provide sufficient reason to upgrade, but an long-term release should provide major features to make a change worthwhile.
To promptly answer your question about user-land upgrades, it's a simple matter of looking at the masses.
What percentage of developers are good? What percentage write unit tests? What percentage of users understand what they're doing, or following tasks?
The point of these questions is quite simple: When an upgrade goes bad, what percentage of users are savvy enough to troubleshoot their way out of it themselves and not interrupt their work?
While I consider myself on the advanced side, I've had issues with upgrades that contributed to very difficult problems to sort out (most recently a change in utf-8 behavior between mysql, DBI, Catalyst, perl 5.8 and an upgrade to perl 5.10).
It took me several hours to figure out what went wrong, and to fix the bugs in my code.
Note a particular phrase, though: "figure out what went wrong". I wasn't interested in simply getting things working, changing tests to hide the failures, etc.
I truly wanted to figure out what went wrong, so that I could gain knowledge.
This is what separates out communal hackers (contributors) to users. Users want to use things, they consume a product and do something with it. Unless something affects them directly, there is no incentive to upgrade. In fact, there are reasons not to upgrade at all, since unless they're affected by a bug they are working within known constraints. They don't want to transition and rock the boat.
While one man's trash is another's treasure, one man's "fairly boring" is daunting and foreboding to someone else.
Perl 5.8 to Perl 5.10 isn't a good example, given the five and a half years between them. With that said....
People who can't run a mile also can't run a marathon.
I'm horrible about remembering to do anything, so I automate as much as I can manage. My operating system and language tools tell me when it's time to upgrade something. Basically, my spoon starts shouting at me if it hasn't been washed and put away. Ignoring the noise is more inconvenient then simply taking care of the task. It doesn't bother me when an update breaks something, mainly because I'm usually breaking things plenty on my own.
That might not apply for normal people. I think generally folks are afraid of any change to an existing system that works - or works well enough, at least. If they could be sure that an update would be seamless, there would be less fear.
Just thought of something else. There's also the problem of dependency lag. I haven't played much with Python 3.0, largely because my favorite libraries are all still catching up with 2.6. Some of them have even explicitly stated that they won't work on 3.0 compatibility until they have some demand for it.
So if there's one weak link between the upgrade and you, applying that upgrade becomes a risk.
The equivalent of baby steps with regards to upgrades would be continually git pull'ing or svn up'ing your dependencies--even then, it might be unmanageable due to the number of dependencies modern applications have.
For this to be feasible (and it is feasible; I do it with a number of libraries I have to work very closely with), you have to 1. be assured that the trunk is reasonably stable, and 2. if it breaks, you can talk to people who know how to fix it, or fix it yourself, and 3. be diligent in up'ing *all* of your dependencies (with modern applications and frameworks, this is not a small number).
Which is why most people don't do it.