Clone this repository
For self-hosted knots, clone URLs may differ based on your setup.
A machine job substituted the whole input closure of the derivations it
builds, e.g. 3.4 GiB for neoprism, of which the builds open a few dozen
MiB. .github/lazy-store.sh now runs before the build: it takes what
`nix build --dry-run` would fetch, collects its closure from the
narinfos of cache.nixos.org and cache.lassul.us (tools.lazy-store),
mounts lazy-store on /nix/store, registers the paths with
`nix-store --load-db` and copies their signatures. The build itself,
ci-build.sh's niks3 push and hestia stay as they were: outputs are
written to the real store dir below the mount, and ci-build.sh's
before/after diff of the store starts after the registration.
If anything fails before the paths are registered, it logs an error,
undoes the mount and exits 0, so the build substitutes eagerly. The FUSE
log and the fetched NARs are printed by an always-run step after the
build.
The eval job builds lazy-store next to niks3 and pushes it, the machine
jobs substitute it. The eval and shared jobs keep the plain store: eval
builds nothing but niks3 and lazy-store, and shared builds package
derivations, which read most of their inputs anyway.
FUSE file system for CI's machine builds, ported from the experiment on
ci-lazy-fuse (.github/exp/lazy-store). It mounts on /nix/store the real
store dir (all writes go there) merged with store paths of
cache.nixos.org and cache.lassul.us, whose metadata comes from their
.ls listings and whose NAR is fetched, checked and unpacked on the first
open() of a file in them; reads go through kernel FUSE passthrough. In
the experiment, rebuilding neoprism's system-units, etc, system-path and
toplevel downloaded 62 MiB instead of the 3,455 MiB of an eager
substitution.
Changes from the experiment:
- closure takes the store paths to make valid on stdin (what
`nix build --dry-run` would fetch) instead of walking derivations, and
leaves out paths whose closure is not entirely in the caches, so that
nix never sees a valid path with a missing reference; nix substitutes
those as usual.
- closure also writes the fetched narinfos as a file:// binary cache:
`nix-store --load-db` registers no signatures, and niks3's
--ignore-upstream-key and hestia's upstream-cache-filter go by the
signatures in the local database, so without them both would upload
(and thereby download) every cache.nixos.org path in an output's
closure. `nix store copy-sigs` adds them from there.
- mount fails if a path has no listing instead of leaving it out, as
every path gets registered.
Static, so that once it serves /nix/store it loads nothing from there.
A flake package, so CI builds it once and pushes it to cache.lassul.us,
and the jobs substitute it. passthru.tests.fetch covers fetch, closure
and the registration with signatures against a file:// cache.
The GIBS VIIRS_NOAA20_Thermal_Anomalies_375m_All raster shows each
detection as a coarse blob, far bigger than its 375 m footprint at
wallpaper scale, so the fire layer was loud and distracting.
Fetch FIRMS' keyless global 24 h VIIRS CSVs of NOAA-20, NOAA-21 and
Suomi NPP instead (cached, refreshed hourly like the old layer), drop
low-confidence detections and draw each one as an antialiased dot in
screen space through the same forward mapping as the markers: about
1 px at 1080p, growing slightly with the lens zoom, opacity from FRP on
a log scale between 1 and 500 MW. The colour and max blend under the
clouds stay as before.
Mercator at 16:9 only showed about +-71 degrees, cutting off northern
Greenland and the Arctic. The new variant world is plate carrée, north up,
centred on the equator and the home longitude, showing -90..90 degrees
stretched vertically to fill the screen. It replaces mercator in the CLI
--projection choices and the server's variant parameter, without an alias
for the old name. lens and plain are unchanged.
bluetoothd's hostname plugin overrode General.Name (and Class) with the
hostname and chassis, so disable it. bluetooth-pairing set adapter
properties before AutoEnable powered hci0, failed with Not Powered and
failed the switch; wait for Powered first.
acheron gets BlueZ (adapter "gg23", UB500 dongle) as an A2DP sink only;
WirePlumber's bluez monitor targets received streams at the "hub" sink,
like roc-lan and AirPlay. A bt-agent (NoInputNoOutput) accepts pairing
without PIN. A Home Assistant button (MQTT discovery) opens a 120s
discoverable/pairable window; afterwards bonded devices are trusted so they
reconnect any time, and a binary_sensor shows the window state.
ci.yml builds the machines with hestia now. Drops farm.yml, tools/ci-farm, neoprism's nix-grpc-daemon builder, barnacle's farm client, the nix-farm vars generators and the nix-grpc-store input. cache.lassul.us/niks3 moves to configs/nix-cache.nix.
Daily garbage collection of the hestia cache on the default branch, after
hestia's own gc.yml, so ci.yml's cache stays within the 10 GB GHA
cache quota.
Replaces the farm call. eval evaluates the new ciToplevels output once
with hestia's matrix subaction (the same machines the farm built: all
nixosConfigurations), shared builds the derivations needed by more than
one uncached machine once and hands the hestia manifest version with
their outputs to the machine jobs, one job per uncached machine builds
its toplevel by drv path. The ci job stays the required check; shared
and the machine jobs are skipped when nothing is uncached.
Every job pushes the store paths it added, built or substituted from
hestia, to niks3 with its GitHub OIDC token, as the farm did, so
deploys find the toplevels in cache.lassul.us. The niks3 client is the
new packages.niks3 output.
Measured on the ci-hestia-shared experiment with a nixpkgs bump: 15 min
(eval 3.3, shared 3.3, 1-4 per machine) instead of 49 with the farm. No
more queueing of runs for the farm.