[oi-dev] Downloading archives to a common area

[Alasdair Lumsden]

On 10 May 2012, at 00:42, Bayard Bell wrote:

> I think this is better than setting ARCHIVE_MIRROR, which may be
> useful for other purposes--it's even more handy for managing our
> source archive if you can populate what's current with a global gmake
> download to one directory and then rsync that to dlc. As you say, with
> multiple workspaces, you can simply symlink this to a single copy of
> the archives, and with this done, you don't have to copy to pop into
> the workspace; you get archive dedup for free with nearly zero effort.

Yes indeedy - it's really helpful especially for bulk builds. You can also restore the old behaviour by setting USERLAND_ARCHIVES=./ (or possibly =$(COMPONENT_DIR)). And you can still make use of an alternative ARCHIVE_MIRROR.

It also looks like Oracle attempted to implement USERLAND_ARCHIVES originally but either didn't finish, or broke it later on when adding support for multiple archives in a component. So this is fixing a currently broken feature.

I'd like to write a continuous-integration rule for pushing valid archives (based on the digest) automagically up to the dlc mirror. I'll look into that soon.

> The only downside is that userland-fetch might blow away your only
> copy because of digest mismatches in the component Makefiles, so you
> probably want to use snapshots a bit more than you might have done.
> People like me doing development on laptop SSDs get a big win for the
> reduced storage overhead.