Commits
autoUpgrade pins the upgrade to the revision the forward-only guard resolved and writes it to /var/lib/superconfig-revision after a successful upgrade, like deploy does. Vector reads that file every 30s and exports nixos_system_info{host,revision,revision_short}; the node dashboard shows host -> revision_short.
The flake's git revision (-dirty with uncommitted changes, unknown for non-git sources) is copied to the target with the source and written by deploy-switch once the switch succeeded. Read by the monitoring agent; it is in no derivation, so a commit doesn't change every machine's closure.
kartei carries both cards (krebs/kartei#92), so the superconfig-only host
set is empty; read hosts, own addresses and tinc host files from kartei
alone.
Query field values for host and unit (unit narrowed by the selected hosts), with multi-select and All = *, and filter the panel with in().
debugLevel 0 still maps to Info, so reconnect, MTU and path-revalidation messages flood journald. RUST_LOG has to be target-scoped: -d 0 sets the global directive and replaces a bare RUST_LOG=warn.
HTTP service rendering wallpapers on demand for a location, variant,
overlay and size. Renders stay fresh for 10 minutes; ones that were
picked up are re-rendered in the background when they go stale.
realwallpaper gains --cache-dir so locations share one data cache, and
its cache cleanup now tracks last use instead of fetch time.
Replaces pkgs.realwallpaper (xplanet, dead NASA NEO sources) with a
Python renderer: oblique cylindrical map centred on home with a zoom
lens (or plain/mercator), multi-resolution WMS imagery (Sentinel-2
cloudless, Blue Marble, Black Marble lights), live EUMETSAT clouds with
synthesised detail, VIIRS snow/fire/chlorophyll, sea ice and the sky.
The krebs.realwallpaper module, its nginx config and the mors
fetchWallpaper config are dropped; the tool runs standalone for now.
pkgs.phonetpm: 0.4.1 -> 0.4.2
Branches, PRs and auto-merge now use the UPDATE_PKGS_TOKEN repo secret
instead of GITHUB_TOKEN, so the normal pull_request ci.yml run (with the
farm build) shows up on each PR and GitHub merges it with a merge commit
once the ruleset is met. The serial verify job (rebase, dispatch ci.yml,
wait, merge) and the final ci dispatch on master are gone: the auto-merge
counts as the token's, so the push to master triggers ci.yml itself.
UPDATE_PKGS_TOKEN: fine-grained PAT (or GitHub App token) for this repo
with Contents: read/write, Pull requests: read/write and Workflows:
read/write (force-pushing an update branch rebased over master commits that
touched .github/workflows is refused without it). The repo needs "Allow
auto-merge" enabled. The workflow fails early if the secret is missing.
Grafana 13 no longer ships the prometheus datasource as a core plugin, so the provisioned VictoriaMetrics datasource was unusable. Switch it and the dashboards to victoriametrics-metrics-datasource.
Every NixOS machine runs Vector (host metrics, internal metrics, journal)
and writes to VictoriaMetrics/VictoriaLogs on both styx and coaxmetal,
each sink with a 512 MiB disk buffer. Both servers run vmalert (metrics and
LogsQL instances), a two-node Alertmanager cluster posting to Matrix via
matrix-alertmanager-receiver, Gatus (results scraped into VictoriaMetrics,
alerted on by vmalert) and Grafana with provisioned datasources and
dashboards. All ports are open on retiolum only.
Replaces telegraf on neoprism, including the borgbackup status file.
The client gives up after connect-timeout on a store it has never reached.
At farm start envoy has no builder yet (none joined, or not yet dialed and
health checked through its tunnel), answers with grpc-status 14, and a few
machine jobs failed on their first QueryMissing. Probe a health check through
the balancer until a builder answers it.
hermes-agent runs with ProtectSystem=strict and cannot write the
group-writable routes dir. crit now asks crit-routes.socket (group crit)
for a route; a root service picks the id, writes the route, regenerates
the nginx map and reloads nginx.
Encrypting to an age1gpg recipient asked gpg-agent (READKEY) for the
public key, which fails on hosts that only have the public key imported
and no secret key or smartcard stub. Export the key by keygrip with gpg
instead in that case and build the same key as the agent path, so
stanzas are unchanged. Decryption still goes through gpg-agent.
ci.yml calls farm.yml (now reusable) instead of the eval-only nixos-*
checks, which are gone from flake.nix. The farm job is only the
coordinator (scheduler + envoy, which now requires the farm client
cert, reachable over iroh). Each machine job joins as a builder,
evaluates its machine, builds the toplevel through the farm and leaves
when done; its outputs are in niks3 by then.
ci runs for master pushes, PRs and dispatches; a newer run for the same
ref cancels the older one, farms of different refs queue FIFO
(queue: max). update-pkgs verifies PRs with ci.yml alone.
The crit wrapper writes a fresh random 10-char id per daemon port into
/var/lib/crit-routes; a path unit turns that into an nginx map on
coaxmetal:7720 (retiolum only). neoprism proxies *.review.lassul.us there
with a wildcard cert obtained via DNS-01 (RFC2136) against a delegated
_acme-challenge.review.lassul.us zone on knot, so ids never hit CT logs.
acheron gets a minimal system-wide pipewire with a virtual "hub" sink.
Any LAN machine sends into it via ROC while `gg23-audio` from
tools/gg23-audio runs (ROC sink loaded into pipewire-pulse, removed again
on Ctrl-C), barnacle via AirPlay 1 (shairport-sync, "gg23", pipewire
backend, mDNS on int0 only).
The hub monitor is linked to ROC sinks for shodan and icarus (room.nix),
chosen with `audio-room shodan|icarus|both|off` or from Home Assistant via
MQTT discovery (select + buttons). Retained MQTT commands restore the choice
after pipewire restarts.
The rooms have no stable LAN address, so the hub sends to their retiolum IPs
and they accept the ROC ports only from acheron on retiolum.
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
autoUpgrade pins the upgrade to the revision the forward-only guard resolved and writes it to /var/lib/superconfig-revision after a successful upgrade, like deploy does. Vector reads that file every 30s and exports nixos_system_info{host,revision,revision_short}; the node dashboard shows host -> revision_short.
HTTP service rendering wallpapers on demand for a location, variant,
overlay and size. Renders stay fresh for 10 minutes; ones that were
picked up are re-rendered in the background when they go stale.
realwallpaper gains --cache-dir so locations share one data cache, and
its cache cleanup now tracks last use instead of fetch time.
Replaces pkgs.realwallpaper (xplanet, dead NASA NEO sources) with a
Python renderer: oblique cylindrical map centred on home with a zoom
lens (or plain/mercator), multi-resolution WMS imagery (Sentinel-2
cloudless, Blue Marble, Black Marble lights), live EUMETSAT clouds with
synthesised detail, VIIRS snow/fire/chlorophyll, sea ice and the sky.
The krebs.realwallpaper module, its nginx config and the mors
fetchWallpaper config are dropped; the tool runs standalone for now.
Branches, PRs and auto-merge now use the UPDATE_PKGS_TOKEN repo secret
instead of GITHUB_TOKEN, so the normal pull_request ci.yml run (with the
farm build) shows up on each PR and GitHub merges it with a merge commit
once the ruleset is met. The serial verify job (rebase, dispatch ci.yml,
wait, merge) and the final ci dispatch on master are gone: the auto-merge
counts as the token's, so the push to master triggers ci.yml itself.
UPDATE_PKGS_TOKEN: fine-grained PAT (or GitHub App token) for this repo
with Contents: read/write, Pull requests: read/write and Workflows:
read/write (force-pushing an update branch rebased over master commits that
touched .github/workflows is refused without it). The repo needs "Allow
auto-merge" enabled. The workflow fails early if the secret is missing.
Every NixOS machine runs Vector (host metrics, internal metrics, journal)
and writes to VictoriaMetrics/VictoriaLogs on both styx and coaxmetal,
each sink with a 512 MiB disk buffer. Both servers run vmalert (metrics and
LogsQL instances), a two-node Alertmanager cluster posting to Matrix via
matrix-alertmanager-receiver, Gatus (results scraped into VictoriaMetrics,
alerted on by vmalert) and Grafana with provisioned datasources and
dashboards. All ports are open on retiolum only.
Replaces telegraf on neoprism, including the borgbackup status file.
The client gives up after connect-timeout on a store it has never reached.
At farm start envoy has no builder yet (none joined, or not yet dialed and
health checked through its tunnel), answers with grpc-status 14, and a few
machine jobs failed on their first QueryMissing. Probe a health check through
the balancer until a builder answers it.
Encrypting to an age1gpg recipient asked gpg-agent (READKEY) for the
public key, which fails on hosts that only have the public key imported
and no secret key or smartcard stub. Export the key by keygrip with gpg
instead in that case and build the same key as the agent path, so
stanzas are unchanged. Decryption still goes through gpg-agent.
ci.yml calls farm.yml (now reusable) instead of the eval-only nixos-*
checks, which are gone from flake.nix. The farm job is only the
coordinator (scheduler + envoy, which now requires the farm client
cert, reachable over iroh). Each machine job joins as a builder,
evaluates its machine, builds the toplevel through the farm and leaves
when done; its outputs are in niks3 by then.
ci runs for master pushes, PRs and dispatches; a newer run for the same
ref cancels the older one, farms of different refs queue FIFO
(queue: max). update-pkgs verifies PRs with ci.yml alone.
The crit wrapper writes a fresh random 10-char id per daemon port into
/var/lib/crit-routes; a path unit turns that into an nginx map on
coaxmetal:7720 (retiolum only). neoprism proxies *.review.lassul.us there
with a wildcard cert obtained via DNS-01 (RFC2136) against a delegated
_acme-challenge.review.lassul.us zone on knot, so ids never hit CT logs.
acheron gets a minimal system-wide pipewire with a virtual "hub" sink.
Any LAN machine sends into it via ROC while `gg23-audio` from
tools/gg23-audio runs (ROC sink loaded into pipewire-pulse, removed again
on Ctrl-C), barnacle via AirPlay 1 (shairport-sync, "gg23", pipewire
backend, mDNS on int0 only).
The hub monitor is linked to ROC sinks for shodan and icarus (room.nix),
chosen with `audio-room shodan|icarus|both|off` or from Home Assistant via
MQTT discovery (select + buttons). Retained MQTT commands restore the choice
after pipewire restarts.
The rooms have no stable LAN address, so the hub sends to their retiolum IPs
and they accept the ROC ports only from acheron on retiolum.