.github.ci: list the shared derivations in the eval job
The shared job dry-ran every machine to find the derivations needed by
more than one, after a runner start and a prefetch of all drv closures
from hestia. In the benchmark run (PR 68, only neoprism uncached) it
built nothing and still cost the machine job 61 s: 5 s queueing, 6 s
setup, 21 s prefetch (mostly the 47 MiB nixpkgs source), 13 s dry-run,
then 6 s queueing of the machine job.
The eval job has the drvs already, so it dry-runs the machines itself,
in parallel, and only for two or more (one machine shares nothing). It
hands the list to the shared job as an artifact; with an empty list the
shared job is skipped, and the machine jobs wait for the manifest
version of the eval job instead of the shared job's.
.github.ci: list the shared derivations in the eval job
The shared job dry-ran every machine to find the derivations needed by
more than one, after a runner start and a prefetch of all drv closures
from hestia. In the benchmark run (PR 68, only neoprism uncached) it
built nothing and still cost the machine job 61 s: 5 s queueing, 6 s
setup, 21 s prefetch (mostly the 47 MiB nixpkgs source), 13 s dry-run,
then 6 s queueing of the machine job.
The eval job has the drvs already, so it dry-runs the machines itself,
in parallel, and only for two or more (one machine shares nothing). It
hands the list to the shared job as an artifact; with an empty list the
shared job is skipped, and the machine jobs wait for the manifest
version of the eval job instead of the shared job's.
.github.ci: build the machines on a lazy /nix/store
A machine job substituted the whole input closure of the derivations it
builds, e.g. 3.4 GiB for neoprism, of which the builds open a few dozen
MiB. .github/lazy-store.sh now runs before the build: it takes what
`nix build --dry-run` would fetch, collects its closure from the
narinfos of cache.nixos.org and cache.lassul.us (tools.lazy-store),
mounts lazy-store on /nix/store, registers the paths with
`nix-store --load-db` and copies their signatures. The build itself,
ci-build.sh's niks3 push and hestia stay as they were: outputs are
written to the real store dir below the mount, and ci-build.sh's
before/after diff of the store starts after the registration.
If anything fails before the paths are registered, it logs an error,
undoes the mount and exits 0, so the build substitutes eagerly. The FUSE
log and the fetched NARs are printed by an always-run step after the
build.
The eval job builds lazy-store next to niks3 and pushes it, the machine
jobs substitute it. The eval and shared jobs keep the plain store: eval
builds nothing but niks3 and lazy-store, and shared builds package
derivations, which read most of their inputs anyway.