In Shooting Yourself in the Foot with Customer Branches, I described a colleague's current work project -- specifically, how they multiplied the amount of code they had to maintain for every new customer profile.
That's not their only problem. His current frustration stemmed from inadequate testing (and very poor QA management) due to poor handling of code freezes.
A popular (but wrong) theory of software development argues that development progresses in linear and predictable cycles. You gather the specifications. You design the subsystems. You code features. You stabilize a release branch and close it to all but bugfixes (this is the feature freeze). Then you perform detailed QA to try to find and diagnose all of the bugs possible before the release.
That's a very popular theory -- it's almost as popular as it is completely and unarguably wrong.
Management has realized that eventually they have to ship software. There are always features to add and bugs to fix and subsystems to refactor, but the only way to release software is to draw a line on the calendar and say "We're shipping whatever's ready then." So far, so good.
Of course, you can't ship without this cool feature. You can't ship when the long-promised feature isn't ready. You can't ship with this many bugs. You can't ship because there's always something someone wants, and if they have to wait another 18 months for the next version, they'll take their money and find someone else to pay for the software. There's always a rush and a crunch at the end of a release cycle when everyone suddenly realizes that this time, they're not kidding about the release date.
Try being in QA when this happens. All of a sudden, the day before the code freeze, a dozen feature branches that weren't ready land on trunk. QA hasn't seen them. QA didn't even know they built. (Some of them don't.)
Now QA has the unenviable task of working around the damage of features that may not even meet the basic needs of customers but absolutely can't slip because the next release is months or years away. QA gets in arguments about features, because they don't work. Developers don't want to hear that. Management doesn't want to hear that. Change is bad during a freeze. Bugs in design or implementation or requirements take too long to fix, and the relentless drums of "But we promised to release this month!" keep counting down the days.
Eventually, a release candidate escapes, as an admission that no one knows for sure if the software works, and wouldn't it be swell if someone might try to test it, because everyone thinks it probably works, because the project manager has spent the last several weeknights in the bug tracker, reclassifying bug severities and reassigning them to later milestones because one of the criteria for a release candidate is "Zero blocking bugs" and another is "Zero critical severity bugs" and if the difference between getting a bonus this year or making the quarterly numbers means that customers will file bugs in a few months that everyone argued about fixing this week, well, there's a chance customers won't notice. All software has bugs anyway.
Bugs linger in the bug tracker. Their priorities and severities mysteriously change. Duplicate bugs pile up. The software gets released and everyone takes a long weekend....
... and on Tuesday morning, they're back at the grind, preparing for the next patch release cycle to fix all of the problems they knew about before but couldn't fix, because you can't make changes during a code freeze, because no one knows if the code works, because you can't create a release candidate with that bug count, because releasing software is a very big deal, because it's just too darn hard to prepare a release with all of this ceremony. At least they have daily status meetings, though.
Meanwhile, another company down the street knows what "done" means. They keep their bug count low. They can release a new version of their software stable enough for customers to use every day, if they want. Releases aren't a big deal. They're boring and common place. (How do they do it? See The Rapid Release Tautology.)
If you're the first company, the second company has already eaten your lunch, and they'll continue to do so every week, until you fail or until you figure out how to release software.
I was hoping for a bit more discussion about what the better process is! As it stands, all of the arguments you made against a date based process come down to a failure of communication. You've created a company where every group and individual is looking after their own self interests because a lack of trust, and every possible avenue for product sabotage is taken.
I don't see how this same group of paranoid individuals would do any better with the other process. Developers could check code in at the last minute before a daily release and cause a production bug. QA can never create a full test plan because bug fixes and features arrive in semi-random order. They waste time re-testing the same area of the application every few days because fixes aren't grouped in any way.
Developers, bored with the feature they're coding add extra functionality that has no requirements. QA complains but marketing says it has to go in because they've already issued a press release. The bug count is creeping up but the execs promised the board that the numbers would be up by the end of Q1 so the 3rd party partner deal gets worked on instead so that the revenue can be realized sooner. Everyone looks out for themselves.
People will always be able to subvert whatever process is in place. So I think the question isn't "how will process A be ruined by bad people" but "how will process B make work easier for normal people?" And what is that process?
Agreed with Garth. Eleven paragraphs about how not to do a software release and only one with tangential remarks about another company doing it correctly (the "another comapny down the street" paragraph).
While this basically fits the blog entry title it demands a follow up blog entry.
Paul ---