tools.lazy-store: init
FUSE file system for CI's machine builds, ported from the experiment on
ci-lazy-fuse (.github/exp/lazy-store). It mounts on /nix/store the real
store dir (all writes go there) merged with store paths of
cache.nixos.org and cache.lassul.us, whose metadata comes from their
.ls listings and whose NAR is fetched, checked and unpacked on the first
open() of a file in them; reads go through kernel FUSE passthrough. In
the experiment, rebuilding neoprism's system-units, etc, system-path and
toplevel downloaded 62 MiB instead of the 3,455 MiB of an eager
substitution.
Changes from the experiment:
- closure takes the store paths to make valid on stdin (what
`nix build --dry-run` would fetch) instead of walking derivations, and
leaves out paths whose closure is not entirely in the caches, so that
nix never sees a valid path with a missing reference; nix substitutes
those as usual.
- closure also writes the fetched narinfos as a file:// binary cache:
`nix-store --load-db` registers no signatures, and niks3's
--ignore-upstream-key and hestia's upstream-cache-filter go by the
signatures in the local database, so without them both would upload
(and thereby download) every cache.nixos.org path in an output's
closure. `nix store copy-sigs` adds them from there.
- mount fails if a path has no listing instead of leaving it out, as
every path gets registered.
Static, so that once it serves /nix/store it loads nothing from there.
A flake package, so CI builds it once and pushes it to cache.lassul.us,
and the jobs substitute it. passthru.tests.fetch covers fetch, closure
and the registration with signatures against a file:// cache.
tools.lazy-store: init
FUSE file system for CI's machine builds, ported from the experiment on
ci-lazy-fuse (.github/exp/lazy-store). It mounts on /nix/store the real
store dir (all writes go there) merged with store paths of
cache.nixos.org and cache.lassul.us, whose metadata comes from their
.ls listings and whose NAR is fetched, checked and unpacked on the first
open() of a file in them; reads go through kernel FUSE passthrough. In
the experiment, rebuilding neoprism's system-units, etc, system-path and
toplevel downloaded 62 MiB instead of the 3,455 MiB of an eager
substitution.
Changes from the experiment:
- closure takes the store paths to make valid on stdin (what
`nix build --dry-run` would fetch) instead of walking derivations, and
leaves out paths whose closure is not entirely in the caches, so that
nix never sees a valid path with a missing reference; nix substitutes
those as usual.
- closure also writes the fetched narinfos as a file:// binary cache:
`nix-store --load-db` registers no signatures, and niks3's
--ignore-upstream-key and hestia's upstream-cache-filter go by the
signatures in the local database, so without them both would upload
(and thereby download) every cache.nixos.org path in an output's
closure. `nix store copy-sigs` adds them from there.
- mount fails if a path has no listing instead of leaving it out, as
every path gets registered.
Static, so that once it serves /nix/store it loads nothing from there.
A flake package, so CI builds it once and pushes it to cache.lassul.us,
and the jobs substitute it. passthru.tests.fetch covers fetch, closure
and the registration with signatures against a file:// cache.
tools.lazy-store: init
FUSE file system for CI's machine builds, ported from the experiment on
ci-lazy-fuse (.github/exp/lazy-store). It mounts on /nix/store the real
store dir (all writes go there) merged with store paths of
cache.nixos.org and cache.lassul.us, whose metadata comes from their
.ls listings and whose NAR is fetched, checked and unpacked on the first
open() of a file in them; reads go through kernel FUSE passthrough. In
the experiment, rebuilding neoprism's system-units, etc, system-path and
toplevel downloaded 62 MiB instead of the 3,455 MiB of an eager
substitution.
Changes from the experiment:
- closure takes the store paths to make valid on stdin (what
`nix build --dry-run` would fetch) instead of walking derivations, and
leaves out paths whose closure is not entirely in the caches, so that
nix never sees a valid path with a missing reference; nix substitutes
those as usual.
- closure also writes the fetched narinfos as a file:// binary cache:
`nix-store --load-db` registers no signatures, and niks3's
--ignore-upstream-key and hestia's upstream-cache-filter go by the
signatures in the local database, so without them both would upload
(and thereby download) every cache.nixos.org path in an output's
closure. `nix store copy-sigs` adds them from there.
- mount fails if a path has no listing instead of leaving it out, as
every path gets registered.
Static, so that once it serves /nix/store it loads nothing from there.
A flake package, so CI builds it once and pushes it to cache.lassul.us,
and the jobs substitute it. passthru.tests.fetch covers fetch, closure
and the registration with signatures against a file:// cache.
tools.lazy-store: init
FUSE file system for CI's machine builds, ported from the experiment on
ci-lazy-fuse (.github/exp/lazy-store). It mounts on /nix/store the real
store dir (all writes go there) merged with store paths of
cache.nixos.org and cache.lassul.us, whose metadata comes from their
.ls listings and whose NAR is fetched, checked and unpacked on the first
open() of a file in them; reads go through kernel FUSE passthrough. In
the experiment, rebuilding neoprism's system-units, etc, system-path and
toplevel downloaded 62 MiB instead of the 3,455 MiB of an eager
substitution.
Changes from the experiment:
- closure takes the store paths to make valid on stdin (what
`nix build --dry-run` would fetch) instead of walking derivations, and
leaves out paths whose closure is not entirely in the caches, so that
nix never sees a valid path with a missing reference; nix substitutes
those as usual.
- closure also writes the fetched narinfos as a file:// binary cache:
`nix-store --load-db` registers no signatures, and niks3's
--ignore-upstream-key and hestia's upstream-cache-filter go by the
signatures in the local database, so without them both would upload
(and thereby download) every cache.nixos.org path in an output's
closure. `nix store copy-sigs` adds them from there.
- mount fails if a path has no listing instead of leaving it out, as
every path gets registered.
Static, so that once it serves /nix/store it loads nothing from there.
A flake package, so CI builds it once and pushes it to cache.lassul.us,
and the jobs substitute it. passthru.tests.fetch covers fetch, closure
and the registration with signatures against a file:// cache.
tools.lazy-store: init
FUSE file system for CI's machine builds, ported from the experiment on
ci-lazy-fuse (.github/exp/lazy-store). It mounts on /nix/store the real
store dir (all writes go there) merged with store paths of
cache.nixos.org and cache.lassul.us, whose metadata comes from their
.ls listings and whose NAR is fetched, checked and unpacked on the first
open() of a file in them; reads go through kernel FUSE passthrough. In
the experiment, rebuilding neoprism's system-units, etc, system-path and
toplevel downloaded 62 MiB instead of the 3,455 MiB of an eager
substitution.
Changes from the experiment:
- closure takes the store paths to make valid on stdin (what
`nix build --dry-run` would fetch) instead of walking derivations, and
leaves out paths whose closure is not entirely in the caches, so that
nix never sees a valid path with a missing reference; nix substitutes
those as usual.
- closure also writes the fetched narinfos as a file:// binary cache:
`nix-store --load-db` registers no signatures, and niks3's
--ignore-upstream-key and hestia's upstream-cache-filter go by the
signatures in the local database, so without them both would upload
(and thereby download) every cache.nixos.org path in an output's
closure. `nix store copy-sigs` adds them from there.
- mount fails if a path has no listing instead of leaving it out, as
every path gets registered.
Static, so that once it serves /nix/store it loads nothing from there.
A flake package, so CI builds it once and pushes it to cache.lassul.us,
and the jobs substitute it. passthru.tests.fetch covers fetch, closure
and the registration with signatures against a file:// cache.