[oi-dev] pkg -- huge RSS

[Till Wegmüller]

Well yes, that aswell. To be more precise we will at some point need a
way to make IPS more compatible with rolling-release. Everytime we do
spring cleaning people cannot install software until they do an upgrade.
So it's a way to fix one issue but cause another. IPS should be capable
of not needing spring cleaning this often. Rotating consolidations out
after a year or so is ok. preferably 2 years. But not 2 or 3 times every
snapshot.


-Till

On 09.06.20 00:55, Aurélien Larcher wrote:
> Also our pkg repository needs some belated spring cleaning...
> 
> On Tue, Jun 9, 2020 at 12:39 AM Till Wegmüller <toasterson at gmail.com
> <mailto:toasterson at gmail.com>> wrote:
> 
>     Hi Gordon
> 
>     Unfortunately yes. We included a patch on the 7th of March which should
>     make things a little bit better. There is still work needing to be done
>     though. We are approaching this from a few angles but it's going to take
>     time to replace components with more efficient ones. And find the things
>     which are causing this exactly If you want to see some detailed
>     benchmarks how we did it for the Update in march you can find them in
>     the PR here [0]. If I know which specific thing is eating RSS I can work
>     on replacing that with a bit of Rust code. That works well together with
>     python and I know some people which can help make it work.
> 
>     If you are interested in helping, I can add documentation to my quite
>     extensive experiments with the IPS formats. I have a almost working repo
>     server (only search is missing) and pkgrepo in Go [1]. And I am working
>     on getting it ported to Rust [2].
> 
>     Greetings
>     Till
> 
>     [0] https://github.com/OpenIndiana/pkg5/pull/75
>     [1] https://git.wegmueller.it/Illumos/pkg6
>     [2] https://github.com/OpenFlowLabs/libips
> 
>     On 08.06.20 18:20, Gordon Ross wrote:
>     > Is this normal?
>     >
>     >    PID USERNAME  SIZE   RSS STATE  PRI NICE      TIME  CPU
>     PROCESS/NLWP
>     >   1289 root     3574M 3489M cpu3    60    0   0:07:10 2.0% pkg/1
>     >
>     > I bumped up the RAM in this VM to 6GB and it's still struggling to
>     complete...
>     >
>     > root at oi-tank:/export/home/gwr# pkg info package/pkg
>     >                 Name: package/pkg
>     >              Summary: Image Packaging System
>     >          Description: The Image Packaging System (IPS), or pkg(5),
>     is the
>     >                       software delivery system used on
>     OpenIndiana.  This
>     >                       package contains the core command-line
>     components and
>     >                       pkg.depotd server.
>     >             Category: System/Packaging
>     >                State: Installed
>     >            Publisher: openindiana.org <http://openindiana.org>
>     >              Version: 0.5.11
>     >               Branch: 2019.0.0.5528
>     >       Packaging Date: September 25, 2019 at 02:40:19 PM
>     >    Last Install Time: November  6, 2019 at 09:26:38 AM
>     >                 Size: 14.30 MB
>     >                 FMRI:
>     >
>     pkg://openindiana.org/package/pkg@0.5.11-2019.0.0.5528:20190925T144019Z
>     <http://openindiana.org/package/pkg@0.5.11-2019.0.0.5528:20190925T144019Z>
>     > Repository Changeset: a6bfc17b
>     >           Source URL: https://github.com/OpenIndiana/pkg5/
>     >
>     > _______________________________________________
>     > oi-dev mailing list
>     > oi-dev at openindiana.org <mailto:oi-dev at openindiana.org>
>     > https://openindiana.org/mailman/listinfo/oi-dev
>     >
> 
>     _______________________________________________
>     oi-dev mailing list
>     oi-dev at openindiana.org <mailto:oi-dev at openindiana.org>
>     https://openindiana.org/mailman/listinfo/oi-dev
> 
> 
> 
> -- 
> ---
> Praise the Caffeine embeddings
> 
> _______________________________________________
> oi-dev mailing list
> oi-dev at openindiana.org
> https://openindiana.org/mailman/listinfo/oi-dev
>