[oi-dev] phasing out openssl 1.0.2 (mostly)

Sat Feb 24 18:23:33 UTC 2024

Hi,

On Sat, Feb 24, 2024 at 06:27:30PM +0100, Goetz T. Fischer wrote:
> as you know there're still some packages in the repo that use openssl 1.0.2. so 
> far this had the unpleasant implication that all new packages had to be 
> hardcoded to newer ssl versions one way or the other, because the buildsystem's 
> ssl mediator had to remain at 1.0.

Sorry, this is not true. Proof:

$ pkg contents -mr library/python/cryptography-39 | grep ^depend.*openssl
depend fmri=pkg:/library/security/openssl-31 at 3.1.5-2024.0.0.0 type=require
$ grep -i ssl components/python/cryptography/Makefile 
REQUIRED_PACKAGES += library/security/openssl-31
$

Please note that you said "all" so single counter example is enough to
prove your statement is not true.

> obviously that wastes a lot of time and usually should be the other way around. 

It is already the other way around because the default OpenSSL version
in the build framework is 3.1:

$ grep OPENSSL_DEFAULT make-rules/shared-macros.mk 
OPENSSL_DEFAULT = 3.1
OPENSSL_VERSION ?= $(OPENSSL_DEFAULT)
$

So in other words when a component needs non-default OpenSSL then it
needs to specify the desired version via the OPENSSL_VERSION macro in
its Makefile.

And, in an ideal world this should be all that is needed re the OpenSSL
support and versions for a component (see cryptography above for an
example).

> i.e. only hardcoding the handful of packages which, for whatever reason, still 
> need 1.0.2 and having the buildsystem's ssl mediator set to whatever is 

The openssl mediator (on build machine) is orthogonal to that.  All
components builds should produce same results with any openssl mediator
setting.  Provided the OpenSSL support in a packaged component is done
properly.  Unfortunately, we do not live in an ideal world so some
components needs more than simple OPENSSL_VERSION set (or omit) to build
properly with OpenSSL.  This is usually an issue in the software we are
packaging (so it should be solved there, in upstream), or maybe in our
build framework (this should be reported and/or debugged, and eventually
fixed).

> considered the default at the time. having a significantly smaller number of 
> packages with a fixed ssl version also makes switching to a different ssl 
> version at some point much nicer. the latter of course depending on how much 
> has been modified of each package to achieve the fixed ssl dependency.

Yes, I agree here.  For good citizens (components) you should never set
anything related to GCC, Perl, Python, OpenSSL, etc.  Everything is set
by the build framework and it should simply work OOTB.

> right now 91 packages are affected. see attachment for the list. not counting 
> the ones which even need 0.9.8 :-O

A package needing old(er) OpenSSL is not a problem.  For example some
software might not support the latest OpenSSL, so if we want to package
it we need to use older OpenSSL there.

Yes, it would be great if all pacakges from your list are rebuilt for
newer OpenSSL (if they support newer OpenSSL).  Somebody just needs to
sit down and do the work (and then create a PR).  Things do not
materialize from nowhere.

> some of them should obviously be updated anyway. especially server things that 
> are reachable from the outside like proftpd or nginx would be priority targets 

What I wrote few lines above applies here.

> in any case. probably more tricky is the system stuff like wpa.

The wpa package comes from illumos-gate.  I'm not sure if it supports
newer OpenSSL than 1.0 out of the box.  If not then it should be fixed
there.  If it does, then maybe we need to adjust our
components/openindiana/illumos-gate/Makefile somehow.  Again, see above
(sit down, etc.).

> some packages will likely be stuck with ssl 1.0.2 because they can't be updated 
> for various reasons. the ones who remain[1] would be the candidates for actual 
> patching to make them use a fixed (older) ssl version.
> 
> in short, the fact that a single program, that has been retired 4 years ago, 
> (still) has such an impact on the whole buildsystem is a condition that should 
> likely be changed rather sooner than later.

Sorry, you apparently miss something.  See above.  The openssl mediator
setting at build machine should not affect the result.

The only problem are bad citizens (as I described above).  For these, we
need to have well known mediator settings so in a case they are bad
citizens and this was not catched/solved during the packaging, we need
to get the same results on the official build machine and on the
developer's/packager's build machine.  This means that all mediators
should be set to default value on all machines used for building and/or
packaging oi-userland stuff or doing illumos-gate development.

> an alternative approach:
> 
> the general goal is to keep the ssl dependency flexible. at least as far as 
> each program's code is concerned. if doing that by mediator causes too many 
> problems, using $(OPENSSL_INCDIR) and $(OPENSSL_LIBDIR) in the Makefile could 
> be an alternative for those programs/packages where that's sufficient.
> having a peek at other repos shows that e.g. the solaris userland has sort of a 
> compromise solution. they do set the ssl version explicitly. however, their 
> package names only contain the major version like "openssl-3" and the same goes 
> for the install paths like "/usr/openssl/3/". that's not as flexible as having 
> $(OPENSSL_INCDIR) and $(OPENSSL_LIBDIR) only or having it sorted by the 
> mediator but at least allows all 3.x versions without code changes.

Please do not look at solaris-userland too much.  We already diverged
significantly so very often what you see in oi-userland or in
solaris-userland makes no sense in the other project.

> regardless of the mediator, selecting and updating the packages for which 
> $(OPENSSL_INCDIR) and $(OPENSSL_LIBDIR) is enough can be done anyaway.
> 
> [1] slightly modified loki reference

I hope this explained at least something regarding out build framework.
Should you have more question, just ask.

Regards.

-- 
+-------------------------------------------+
| Marcel Telka   e-mail:   marcel at telka.sk  |
|                homepage: http://telka.sk/ |
+-------------------------------------------+