The long-term consequences of maintainers' actions

OpenSSL 3 has entered Alpine, and we have been switching software to use it over the past week.  While OpenSSL 1.1 is not going anywhere any time soon, it will eventually leave the distribution, once it no longer has any dependents.  I mostly bring this up because it highlights a few examples of maintainers not thinking about the big picture, let me explain.

First, the good news: in distribution-wide rebuilds, we already know that the overwhelming majority of packages in Alpine build just fine with OpenSSL 3, when individually built against it.  Roughly 85% of main builds just fine with OpenSSL 3, and 89% of community builds with it.  The rebuild effort is off to a good start.

Major upgrades to OpenSSL are not without their fallout, however.  In many cases, we cannot upgrade packages to use OpenSSL 3 because they have dependencies which themselves cannot yet be built with OpenSSL 3.  So, that 15% of main ultimately translates to 30-40% of main once you take into account dependencies like curl, which builds just fine with OpenSSL 3, but has hundreds of dependents, some of which don’t.

A major example of this is mariadb.  It has been known that OpenSSL 3 was on the horizon for over 4 years now, and that the OpenSSL 3 release would remove support for the classical OpenSSL programming approach of touching random internals.  However, they are just now beginning to update their OpenSSL support to use the modern APIs.  Because of this, we wound up having to downgrade dozens of packages which would otherwise have supported OpenSSL 3 just fine, because the maintainers of those packages did their part and followed the OpenSSL deprecation warnings as they showed up in OpenSSL releases.  MariaDB is a highly profitable company, who do business with the overwhelming majority of the Fortune 500 companies.  But yet, when OpenSSL 3 releases started to be cut, they weren’t ready, and despite having years of warning they’re still not, which accordingly limits what packages can get the OpenSSL 3 upgrade as a result.

Another casualty will be Ansible: we have already moved it to community.  You are probably wondering why Ansible, a software package which does not use OpenSSL at all, would be a casualty, so please let me explain.  Ansible uses paramiko for its SSH client, which is a great SSH library for Python, and is a totally solid decision to make.  However, paramiko uses cryptography for its cryptographic functions, again a totally solid decision to make, cryptography is a great library for developers to use.

For distributions, however, the story is different: cryptography moved to using Rust, because they wanted to leverage all of the static analysis capabilities built into the language.  This, too, is a reasonable decision, from a development perspective.  From the ecosystem perspective, however, it is problematic, as the Rust ecosystem is still rapidly evolving, and so we cannot support a single branch of the Rust compiler for an entire 2 year lifecycle, which means it exists in community.  Our solution, historically, has been to hold cryptography at the latest version that did not require Rust to build.  However, that version is not compatible with OpenSSL 3, and so it will eventually need to be upgraded to a new version which is.  And so, since cryptography has to move to community, so does paramiko and Ansible.

The ideology of moving fast and breaking things, while tolerated in the technology industry, does not translate to success in the world at large.  Outside of technology, the world prefers stability: the reason why banks still buy mainframes, and still use z/OS, is because the technology works and can be depended upon.  Similarly, the engine controller in cars, and medical devices like pacemakers and insulin pumps, are running on C.  They don’t run on C because C is a good language with all the latest features, they run on C because the risks and mitigations for issues in C programs are well-understood and documented as part of MISRA C.

Distributions exist to provide a similar set of stability and reliability guarantees.  If we cannot provide a long-term support lifecycle for a piece of technology your software depends on, then we are forced to provide a shorter support lifecycle for your software as well.  For some, that is fine, but I think many will be disappointed to see that they haven’t fully gotten OpenSSL 3, or that Ansible has had to be moved to community.