Backwards Compatibility is Technical Debt

First, some caveats:

• I use the word "maintenance" to refer to software under current development, even if that development is only for bugfixing purposes, and not adding new features. Thus almost all software undergoes maintenance.
• The risk guidelines apply only to code undergoing maintenance. If you are not and will not modify software, the risks are immaterial.
• Guidelines are guidelines, not universal precepts. They're rules of thumb. They may not apply in all cases. They meet my experience, but I know that a few projects have had different experiences. Some people who smoke never get lung cancer, but it's foolish to ignore the connection. That's why I call them risks and guidelines.
• I believe that software can and should get better over time.

Suppose you write a successful library. You had a great idea, and you implemented it, and now people use it. That experience has given you further ideas for enhancements. You'd like to make your library more powerful, or easier to use, or generally better.

Thus you experiment. You play with different API ideas. You look at solutions to similar problems. You ask some of your best users for feedback, and you release a new version of the library.

Repeat this a few times, and you'll discover that you don't always get everything right the first time. You face the awkward question of how to make improvements while not stranding your existing users.

Consider the risks. First, you risk getting an API or a design wrong by making any changes or adding any features. You can ameliorate these risks by being very cautious, but you can't eliminate this risk. You're probably not an expert on the problem area unless you've already solved the problem with code multiple times, in which case why are you starting over? Only feedback will tell you if you've done it right.

The second risk is that no one will care. You can't mitigate this. Release it anyway.

The third risk is that you did it wrong, and you'll look like a fool. Research can fix this. Not caring what random people on the Internet will think helps. Adding disclaimers helps (but only because you can tell people to read the disclaimers before arguing with you, and then they look silly).

The fourth risk is that you get it kind of right and kind of wrong, and to get it more right, you have to make changes to the wrong parts, and that will change how your users interact with the code.

That's the scariest risk. With all of these wonderful users, how can you tell them that you didn't get it all right, and they may have to suffer through an incompatible change? (I think the argument is easy to make; you need a sober assessment of their risks and responsibilities, but that's a different facet to explore another day.)

Consider the risk of not making improvements when you see them. Ahh, now you understand all the talk of risk.

Technical Debt (see also Design Debt) is a measurement of how easy it is to work with the code. An Approximate Measure of Technical Debt argues that every line of code is a good approximation of technical debt. In general, the more code, the more technical debt a system is likely to have.

You see technical debt every time you go to add a feature or fix a bug and it's more difficult than it should be. Perhaps the name of a variable or function is wrong. Perhaps a comment is misleading -- imagine that. Perhaps there's duplication, or near duplication. There may be good reasons why the code is in that state. Sometimes those shortcuts are necessary, or sometimes the right approach isn't obvious without more experience, or sometimes changes elsewhere give you the opportunity to improve abstractions and coalesce near-duplication into duplication to remove it. If software really can get better over time, we should expect this to happen.

Consider, however -- every feature you don't need represents technical debt. Code you might use in the future represents technical debt, even if only scrolling past it in your editor, or that extra second spent compiling, or a function name you have to skip over in the API documentation.

Take that argument one step further. Code that exists solely for backwards compatibility -- to allow people to continue to use broken, or old, or clunky, or wrong code -- is very nasty, very expensive technical debt.

You can't always avoid taking on technical debt, but if you're maintaining software, you'll always pay interest on that debt until you pay off the debt. Like duplication, backwards compatibility leading to huge amounts of technical debt can eventually crush a project. You need a plan to get rid of it. Refactoring helps remove duplication and improve designs. Only deprecation -- and removal -- can remove backwards compatibility debt.