[OpenIndiana-discuss] Questions about /var/pkg

[Matthew R. Trower]

On 1/12/24 08:49, Marcel Telka wrote:
> On Fri, Jan 12, 2024 at 08:21:17AM -0600, Matthew R. Trower wrote:
>> 
>> On 1/12/24 04:41, Marcel Telka wrote:
>>> On Fri, Jan 12, 2024 at 04:08:12AM -0600, Matthew R. Trower
>>> wrote:
>>>> /var/pkg/publisher is exactly what I thought it was.  I am
>>>> still not clear as to whether the format is committed(stable)
>>>> across different versions of pkg5, and therefore safe to share
>>>> across BEs.
>>> 
>>> The man page says: Interface Stability: Uncommitted
>> 
>> My understanding of the Interface Stability attribute is that it
>> applies to application programming interfaces (source and binary).
>> I would not expect it to necessarily apply to the data
>> storage/caching layer of a program, as I would not expect that to
>> be considered an application
> 
> Your expectation is wrong.  Almost everything constitutes an
> interface. See also attributes(7).

Interesting, I only knew of attributes(5).  It appears attributes(7) may
have shared a common root, but has long since diverged (or maybe they're
just encoded differently).  Still, they appear identical in regards to 
the Interface Stability attribute.

But alright, well, if you want to bring attributes(*) into this, I would 
like to reference the following excerpts:

===

To make reasonable risk assessments, developers need to know how likely 
an interface is to change in future releases. To aid developers in 
making these assessments, interface stability information is included on 
some manual pages for commands, entry-points, and file formats.

...

The more stable interfaces can safely be used by nearly all applications,

...

The interface stability level classifications described on this manual
page apply to both source and binary interfaces unless otherwise
stated. All stability level classifications are public, with the
exception of the Private classification. The precise stability level of
a public interface (one that is documented in the manual pages) is
unspecified unless explicitly stated. The stability level of an
undocumented interface is implicitly Private.

...

Not-an-Interface

The situation occasionally occurs where there exists an entity that
could be inferred to be an interface, but actually is not.  Common
examples are output from CLIs intended only for human consumption
and the exact layout of a GUI.

This classification is a convenience term to be used to clarify
such situations where such confusion is identified as likely.
Failure to apply this term to an entity is not an indication that
the entity is some form of interface.  It only indicates that the
potential for confusion was not identified.

...

Private

A Private interface is an interface provided by a component (or
product) intended only for the use of that component. A Private
interface might still be visible to or accessible by other
components. Because the use of interfaces private to another
component carries great stability risks, such use is explicitly not
supported. Components not supplied by Sun Microsystems should not
use Private interfaces.
===

Note all the emphasis on "developer" and "application", and how a CLI 
for human-only consumption is NOT an interface, nor is GUI layout.  I'm 
doubling down on "application programming interface".

Furthermore, pkg(1) (and pkg(7), but not pkg(5) .. ??) says this about 
/var/pkg/publisher:

$IMAGE_META/publisher

Contains a directory for each publisher. Each directory stores 
publisher-specific metadata.

While I still don't think /var/pkg/publisher meets a reasonable 
definition of interface, I also don't see its format documented 
anywhere.  Wouldn't that make it private, and therefore independent of 
the IS attribute?

Anyway, this is interesting and all, but hey, maybe we're only 
discussing this because I used the word "committed" in an earlier 
e-mail.  If that implied that I was asking about the IS attribute, then 
I apologize for unfortunate wording.  I really only care about 
likelihoods; an informal statement is fine, such as you made below:

 >> I'm really just asking: how likely, these days, is
 >> /var/pkg/publisher to change in breaking ways?  I know from
 >> observation that it has changed at
 >
 > These days?  Unlikely, because of the activity in the source repo at
 > https://github.com/OpenIndiana/pkg5/commits/oi/ .

I tend to agree.  And it's my working assumption barring further data. 
But Till has mentioned something that makes it moot.


> That really depeneds on your usage scenario.  I run some systems
> almost always up-to-date with non-illumos-gate updates at least once
> a day and full updates (with reboot) every few days and here is an
> example:
> 
> # du -sh /var/pkg/publisher/ 1.17G   /var/pkg/publisher #
> 
> and I do nothing special to keep it that way.  This particular
> machine have almost all available packages installed.

Oh my.

Observe, my workstation:

$ du -hs /var/pkg/publisher /var/pkg/publisher/*
19.1G   /var/pkg/publisher
152M    /var/pkg/publisher/hipster-encumbered
129K    /var/pkg/publisher/localhostoih
18.9G   /var/pkg/publisher/openindiana.org
48K     /var/pkg/publisher/pkg5-nightly
34K     /var/pkg/publisher/sfe
53.5K   /var/pkg/publisher/userland

$ df -h /
Filesystem             Size   Used  Available Capacity  Mounted on
rpool/ROOT/openindiana-2024:01:07
                      224.75G 50.17G     37.49G    58%    /

It's nearly half the size of root at this point!  Which, is unsurprising 
if IPS is caching its packages there:

$ du -hs openindiana.org/file
18.7G   openindiana.org/file

And deleting things here would be fruitless, as the blocks are held by 
snapshots, and the snapshots themselves are held by BE clones.  I'd have 
to nuke pretty much all snapshots and BEs to get my space back at this 
point.  (Exactly how long I need to keep BEs/snapshots around for is out 
of scope for this discussion.)


I wondered about flush-content-cache-on-success:

According to man, the default is true, but:

$ pkg property flush-content-cache-on-success
PROPERTY                       VALUE
flush-content-cache-on-success False


admin at saturn:/usr/src/pkg5/git/src$ git log -1 
c55fcc4b52d2ee8987f5c8bad08359d4f1f03fb1

commit c55fcc4b52d2ee8987f5c8bad08359d4f1f03fb1
Author: Shawn Walker <shawn.walker at oracle.com>
Date:   Mon Aug 22 13:13:03 2011 -0700

     18835 flush-content-cache-on-success should default to True


Ah.  Well.  My system shows its age :-)

Content caching seems intelligent at first blush, but maybe it doesn't 
actually make sense here.  I'll contemplate the matter.


>> If uninstalling a package places files in lost+found, then I'm well
>> and truly confused.
> 
> To confuse you a bit more here is an example:
> 
> # find /var/pkg/lost+found/ -type f # echo test >
> /usr/lib/python3.9/vendor-packages/pytest_randomly-3.15.0.dist-info/TEST
>
> 
# pkg uninstall pytest-randomly-39 pytest-randomly
> ... # find /var/pkg/lost+found/ -type f 
> /var/pkg/lost+found/usr/lib/python3.9/vendor-packages/pytest_randomly-3.15.0.dist-info-20240112T154031Z/TEST
>
> 
# cat 
/var/pkg/lost+found/usr/lib/python3.9/vendor-packages/pytest_randomly-3.15.0.dist-info-20240112T154031Z/TEST
> test #

Ah!  Thank you!  I believe I now fully understand the man page on this 
matter.  Much appreciated.


-- Matthew R. Trower