Chris's Wiki :: blog/tech/BundlingHelpsSoftwareAuthors

Over on Twitter I had a hot take:

Hot take: Every distribution packager who's saying "you shouldn't bundle dependencies" is implicitly telling software authors "you should do more work for us and limit the features (and reliability) of your software".

(This was sparked by reading The modern packager’s security nightmare, via. I'm not exactly on one side or the other, but I do think distributions should be honest about what they're asking for and I don't think they're going to get it.)

This hot take is a bit too narrow. What really matters is software authors and modern software systems restricting the versions of dependencies (for both maximum and minimum versions). Explicit or implicit bundling on top of that just makes the problem slightly worse for distributions.

For software authors, restricting the versions of dependencies that they work with reduces the amount of work that they have to do, both to test against a range of versions and to either forever chase after whatever changes those dependencies like to make or to forever limit what features of dependencies they use to ones available in old versions (and sometimes both at once). In theory, both testing and chasing after changes would be dealt with by Semantic Versioning (if everyone followed it), at least for a single program. In practice, not only are people fallible but also people have a different understanding of what semantic versioning means because semantic versioning is ultimately a social thing, not a technical one. Our field's history has shown (sometimes vividly) that if software authors allow versions of dependencies to move on them, soon or later things break and the software author has to fix it.

(There's also the practical issue that not all dependencies even claim or agree to follow semantic versioning in the first place.)

For distributors, once software authors start restricting versions the distributor has both an upgrade problem and a distribution problem. On the upgrade side, dealing with an issue in a program may now require persuading it to accept a new version of a dependency. On the distribution side, it's now likely that you'll have multiple programs that have different version requirements for the same dependency. At the very least this multiplies the packages involved.

(Many distributions also have package system design problems that restricts the range of versions of things that they can have installed at the same time. Even under Semantic Versioning, this is a problem for the distribution the moment that you have two programs with conflicting version requirements that can't both be packaged and installed at the same time.)

However, there's no free lunch here. What distributors want when they ask for unbundled dependencies without version restrictions is for software authors to do the work to accept any version of their dependencies, or at least any version that falls within Semantic Versioning, and for dependencies to faithfully follow semver and also make it possible to package and install different major versions (at least) at the same time. Accepting a broad version range of your dependencies is actual work, even apart from the limitations it may impose on what your code and your software can do. Software authors and the creators of software package ecosystems (like Go and Rust) are not refusing to do this because they don't like distributions; they are refusing to do this because they have found, over and over again, that this doesn't really work and does cause problems for software authors (and often users of programs) in the long run.

(The software community that's gone through this experience the most visibly is Go, which started out with intrinsically unversioned dependencies that were used universally across all your programs by default and wound up switching to strongly versioned dependencies after many people had many problems with that initial state. Go experienced so many problems that they adopted an unusually strict and software author friendly versioning scheme.)

It's popular for people to argue that software authors should be doing this work anyway even if the distributions weren't asking for it, so them actually doing it is no big deal. This is quite convenient for the people making the argument, but it doesn't make the argument valid. Software authors don't owe anyone any work what so ever; they do whatever work serves their needs and is interesting to them. With limited time and interest, it's both rational and proper for software authors to optimize for their own development.

PS: Generally distributions also want some combination of all software to update to the latest version of their dependencies and for dependencies to explicitly support older versions. This is also extra work for software authors, especially when the distribution also wants it to happen for older versions of programs.