Commits
acheron gets a minimal system-wide pipewire with a virtual "hub" sink.
Any LAN machine sends into it via ROC while `gg23-audio` from
tools/gg23-audio runs (ROC sink loaded into pipewire-pulse, removed again
on Ctrl-C), barnacle via AirPlay 1 (shairport-sync, "gg23", pipewire
backend, mDNS on int0 only).
The hub monitor is linked to ROC sinks for shodan and icarus (room.nix),
chosen with `audio-room shodan|icarus|both|off` or from Home Assistant via
MQTT discovery (select + buttons). Retained MQTT commands restore the choice
after pipewire restarts.
The rooms have no stable LAN address, so the hub sends to their retiolum IPs
and they accept the ROC ports only from acheron on retiolum.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Builders pushed every path of a closure, including everything they had
substituted from cache.nixos.org: 9.9 of 10.3 GiB for ignavia. niks3 now
comes from a fork (lassulus/niks3) with `push --ignore-upstream-key`,
and neoprism's daemon and the CI farm pass cache.nixos.org-1 to it.
cache.lassul.us no longer holds complete closures on its own: clients
need cache.nixos.org next to it, which every machine has.
eternal-terminal 7.0.0 fails to build in the current nixpkgs (cmake_pch.hxx.gch), which failed green's toplevel in the CI farm.
Worker 2 of the first run had looked up paths before other builders pushed them, and its nix-daemon kept answering from the negative cache (1 h by default): 1118 'input not substitutable' retries until the machines failed.
A farm job (scheduler, builder, envoy, client) and a pool of worker jobs
build the requested machines together, each derivation once, and push
the outputs to niks3 with the job's GitHub OIDC token.
Runners can't reach each other, so every hop is a dumbpipe (iroh)
tunnel ending on 127.0.0.1. All daemons and envoy share one TLS
identity (FARM_KEY secret, farm.crt committed, SAN 127.0.0.1) and only
trust it. The iroh endpoint keys are derived from FARM_KEY, so it is the
only secret. Workers probe the farm with fresh dialers before starting
their long-lived one: a dumbpipe dialer started before the farm is
discoverable never reaches it.
configs.nix-farm: niks3 accepts GitHub OIDC tokens of this repository.
Single node: nix-grpc-daemon (builder+scheduler, mTLS, retiolum only)
publishing to niks3 (local postgres, Backblaze B2), cache.lassul.us
proxying the public bucket. barnacle gets the grpc:// plugin and the
shared client cert; all NixOS machines substitute from the cache once
the signing key exists.
The shared nix-farm-ca generator deploys nothing: clan requires a shared
generator to be defined identically on every machine, so per-machine
generators copy out what each needs with their own owner/mode
(nix-farm-server: server cert + ca.crt; nix-farm-client: ca.crt +
client cert/key).
The daemon reads its TLS files via LoadCredential: gRPC's cert file
watcher treats mtime 0 as a stat failure, and clan unpacks /run/secrets
with mtime 0, so pointing it at /run/secrets resets every handshake.
MemoryMax is raised from upstream's 2G to 8G: `niks3 push` runs in the
daemon's cgroup and got OOM-killed on large NARs (kernel modules),
taking the daemon down with it.
acheron gets a minimal system-wide pipewire with a virtual "hub" sink.
Any LAN machine sends into it via ROC while `gg23-audio` from
tools/gg23-audio runs (ROC sink loaded into pipewire-pulse, removed again
on Ctrl-C), barnacle via AirPlay 1 (shairport-sync, "gg23", pipewire
backend, mDNS on int0 only).
The hub monitor is linked to ROC sinks for shodan and icarus (room.nix),
chosen with `audio-room shodan|icarus|both|off` or from Home Assistant via
MQTT discovery (select + buttons). Retained MQTT commands restore the choice
after pipewire restarts.
The rooms have no stable LAN address, so the hub sends to their retiolum IPs
and they accept the ROC ports only from acheron on retiolum.
Builders pushed every path of a closure, including everything they had
substituted from cache.nixos.org: 9.9 of 10.3 GiB for ignavia. niks3 now
comes from a fork (lassulus/niks3) with `push --ignore-upstream-key`,
and neoprism's daemon and the CI farm pass cache.nixos.org-1 to it.
cache.lassul.us no longer holds complete closures on its own: clients
need cache.nixos.org next to it, which every machine has.
A farm job (scheduler, builder, envoy, client) and a pool of worker jobs
build the requested machines together, each derivation once, and push
the outputs to niks3 with the job's GitHub OIDC token.
Runners can't reach each other, so every hop is a dumbpipe (iroh)
tunnel ending on 127.0.0.1. All daemons and envoy share one TLS
identity (FARM_KEY secret, farm.crt committed, SAN 127.0.0.1) and only
trust it. The iroh endpoint keys are derived from FARM_KEY, so it is the
only secret. Workers probe the farm with fresh dialers before starting
their long-lived one: a dumbpipe dialer started before the farm is
discoverable never reaches it.
configs.nix-farm: niks3 accepts GitHub OIDC tokens of this repository.
Single node: nix-grpc-daemon (builder+scheduler, mTLS, retiolum only)
publishing to niks3 (local postgres, Backblaze B2), cache.lassul.us
proxying the public bucket. barnacle gets the grpc:// plugin and the
shared client cert; all NixOS machines substitute from the cache once
the signing key exists.
The shared nix-farm-ca generator deploys nothing: clan requires a shared
generator to be defined identically on every machine, so per-machine
generators copy out what each needs with their own owner/mode
(nix-farm-server: server cert + ca.crt; nix-farm-client: ca.crt +
client cert/key).
The daemon reads its TLS files via LoadCredential: gRPC's cert file
watcher treats mtime 0 as a stat failure, and clan unpacks /run/secrets
with mtime 0, so pointing it at /run/secrets resets every handshake.
MemoryMax is raised from upstream's 2G to 8G: `niks3 push` runs in the
daemon's cgroup and got OOM-killed on large NARs (kernel modules),
taking the daemon down with it.