• 0 Posts
  • 41 Comments
Joined 2 months ago
cake
Cake day: September 5th, 2026

help-circle

  • The idea I think is most transferable

    modules/drivers/ is a drift watcher, and I have not seen this pattern used in a Nix repo before.

    endeavouros.txt is a pinned snapshot of another distro’s live-ISO package list. map.toml requires every package in that snapshot to have an entry, and only those, and each entry must either say what provides it or say why it is deliberately absent:

    [device-mapper]
    by = ["package:lvm2", "option:services.lvm.enable"]
    note = "device-mapper ships with lvm2"
    
    [dracut]
    skip = "an initramfs builder: NixOS builds its own initrd"
    

    The by assertions are checked against a real Nix evaluation of the live system, so a claim that an option is set gets verified rather than trusted. A weekly GitHub Action diffs the snapshot against upstream’s current list and opens or updates a labelled issue when something needs a decision.

    The general shape: pick a project whose coverage you want to match, pin its capability list, require every item to be either provided or explicitly skipped with a reason, and let CI tell you when upstream moves. It turns “am I missing hardware support” from a worry into a file where every gap has a dated decision attached. They run the same pattern for their pinned Python build.

    The test suite

    tests/default.nix and tests/fleet.nix are flake checks, x86_64-linux only, and they need KVM. They boot the real ISO in QEMU, install to a virtual disk unattended from a JSON answer file, reboot through LUKS, hibernate and resume through the encrypted disk, and run a multi-node fleet test with a server, an operator machine and a managed host going through enrollment, build, switch and password change.

    Four techniques I took from it:

    • The tested system is the shipped system with NixOS test instrumentation added via extendModules, not a reduced copy built for testing.
    • Boot-time password prompts are answered through systemd’s ask-password protocol (systemd-reply-password against the socket in /run/systemd/ask-password/) instead of typing on the serial console, which otherwise races the login prompt that also reads it.
    • The ISO is used as the backing file of a qcow2 overlay, so the “USB stick” is writable and the image under test is never mutated.
    • Screenshots are build artifacts.

    There is also a note in there that checks pipe into grep >/dev/null rather than grep -q, because with pipefail set, grep -q exits at the first match and the writer can die of a broken pipe and fail the check intermittently. That one has bitten me before and I had not worked out why.

    What I would not take

    • The people, groups, passwords and enrollment machinery, if you are a fleet of one. users.mutableUsers = false stands on its own; the rest is for an organisation.
    • SQLite state and secrets databases and an audit table, at small scale. Git history and journald cover it.
    • Moving a host table out of Nix into TOML with a strict external schema. I tried the equivalent with PureNix and concluded it bought me nothing. Their validation idea is worth having; the format change is a separate question and I would not assume it is an improvement.
    • A pull agent, without weighing what it costs. It is an always-running privileged daemon with network access, which is a real trade against a push model where nothing runs unattended.

  • I went looking for ideas to borrow for my own NixOS fleet config and ended up reading most of MeccanicOS. It is a desktop distro aimed at non-technical users, BSD-3, about 7,700 lines of Nix plus a Python fleet layer. The desktop half is not what interested me. The fleet layer and the test suite are, and there are a handful of correctness details in there that I think are worth lifting into other people’s tooling.

    Two caveats before anyone takes this as a recommendation. The fleet layer is marked experimental in the repo and the README says it has been tested end to end in VMs and not yet in a real organisation. And most of the primitives below are not new. The value is in how they are assembled and in about six small details that hand-rolled deploy scripts usually get wrong. Everything here is from commit 4d6679b; it is a moving target.

    I have no connection to the project.

    The headline design decision

    From docs/fleet/DESIGN.md:

    An earlier version used Kanidm, Forgejo, Harmonia, FleetDM, MySQL, Redis and nginx. Each did its job well, but together they were a lot to run, secure, upgrade and back up for what the fleet needs. This version replaces them.

    What replaced all of it: one git repository, SSH for every connection, Nix to build each host, SQLite for what the hosts report. No web services, no directory server. Managed hosts keep one SSH connection to a server, the server hands down the store path of the system that host should be running, the host fetches the paths it lacks and switches.

    Whether or not you want their architecture, that is a serious argument made by someone who had the full stack working and chose to remove it. Most homelab and small-fleet setups, mine included, accumulate in the opposite direction.

    Details worth stealing

    1. Run activation in a transient unit, not in your deploy session.

    systemd-run --unit mos-fleet-switch-<ts> --wait --collect --pipe --quiet \
      --service-type=exec <system>/bin/switch-to-configuration switch
    

    Their comment: “Outside this service’s cgroup: switching may restart services, and must finish even if the agent itself goes away meanwhile.” If you deploy by SSHing in and running a rebuild, activation lives in a process tree that activation itself can tear down, sshd included.

    2. A deploy succeeded only if the exit code is zero AND the new system is actually current.

    running = current_system() == system
    # Both: switch-to-configuration can fail after the new system became the current one.
    ok = code == 0 and running
    

    The failure mode they handle explicitly is “the new system is running, but switching failed”: some units never started, readlink /run/current-system points at the new closure, and any monitoring that checks only the symlink reports success. They report that state separately and tell the operator the previous system is still in the boot menu.

    3. Per-host branch, declared in the host’s own data.

    Each host directory has a host.toml naming the branch it follows, main or testing. The build server builds each host from its own branch. That gives me real canaries without a separate pipeline: one machine you do not mind breaking follows testing.

    4. Two-tier config validation, with the cheap tier in a git hook.

    scripts/mos_fleet/repo.py validates the repository without evaluating Nix: unknown keys, duplicate uids and gids, duplicate SSH keys, references to groups or profiles that do not exist, no admin holding both an SSH key and an age key. A pre-receive hook runs that and refuses the push, and also refuses deleting or rewriting main. Full Nix evaluation happens on the operator’s machine and again at build time.

    NixOS assertions catch this class of error at eval. A pre-receive hook catches it before it lands on the branch everything else builds from, which is cheaper and keeps the branch always-buildable. post-receive then refreshes derived state and touches a file that a systemd path unit watches, so the build starts on push; a timer also runs it every 10 minutes.

    5. Authorisation by forced command, not by trusting the client.

    sshd uses an AuthorizedKeysCommand for a single unprivileged account. It emits an authorized_keys line with restrict plus a forced command naming the role and the host, for example mos-fleet-serve host ws-001. The client cannot claim to be a different host, and gets no shell, no PTY and no forwarding. Standard SSH, used properly, and a better fit for a small fleet than anything with a web UI.

    6. Fetching a closure where the server decides what you may have.

    The transport is nothing special, it is nix-store --export piped over the existing SSH channel, which is what nix copy does. The part worth copying is the authorisation: the server will serve only the closure of that host’s current desired system and nothing else. The agent enumerates what it is missing first with

    nix-store --check-validity --print-invalid <paths>
    

    and asks for only those, then verifies the whole system path is valid after importing.

    the agent imports as root with no signature checking, on the grounds that the SSH channel already authenticates the server. That is a deliberate trade of signature verification for channel authentication, and it means whoever controls the server controls every managed host. The design doc says so.

    7. Log shipping acknowledged by journald cursor, and jobs acknowledged by id.

    The server writes a batch and its cursor in one transaction, then acks. On reconnect it tells the agent where its copy ends, so a lost ack re-sends rather than duplicating. Jobs are acked on arrival by id, the agent remembers ids it has seen, and an unacked job is re-sent with the same id. At-least-once with dedup on both sides, written down explicitly. Nothing novel if you have built a log shipper, and easy to get wrong if you have not.

    8. Network-bound disk unlock with a fallback that does not hang.

    boot.initrd.clevis = {
      enable = true;
      useTang = true;
      devices.cryptroot.secretFile = "/etc/secrets/meccanicos/cryptroot.jwe";
    };
    boot.initrd.systemd.network.wait-online = {
      anyInterface = true;
      timeout = 30;
    };
    

    boot.initrd.clevis is already in nixpkgs, so the module is not the contribution. The 30 second wait-online timeout is: a laptop that unlocks from a Tang server on the office LAN has to give up quickly and ask for a passphrase when it is somewhere else, or it sits in initrd forever. They also generate a systemd-cryptenroll recovery key on the host, age-encrypt it to the admins’ keys, and store only the ciphertext centrally.

    9. Weekly store deduplication instead of per-build.

    nix.optimise = { automatic = true; dates = [ "weekly" ]; };
    

    With the comment that auto-optimise-store slowed every update down. Small, and I had never bothered to change it.











  • My best friend has has had the Samsung version of this for four different generations now. I’ve always been a hater about it but now in gen 4, there are very few issues with Samsung’s folding phones… but they still suck and are fragile as hell.
    My conclusion: all folding screen phones like this are shite and Apple gets a first generation mulligan on this one. This release is as bad as any in their history and they should have seen it coming a mile away given Samsung’s myriad issues.

    My rule: NEVER BUY AN OVERPRICED, GIMMICKY, FIRST-GENERATION APPLE PRODUCT.




  • I used a frontier model to generate an allium spec from your repo and it found a few bugs. I like to run apps through allium to prepare for formal verification because it tends to find a LOT of flaws in even the most robust apps. The allium spec file is too large to share here but it can be generated by any frontier model.

    Defects, three of them confirmed by running probes against the crate:

    • P1, a late message from the previous chain breaks the session. The decrypt code never keeps the leftover keys of the old receiving chain, and it reads the header’s previous chain length only as associated data. So a delayed message from the old chain looks like a new ratchet key. The session steps the ratchet again and the tag fails. In the probe, the next good message also failed with an AES-GCM error. Signal handles exactly this case, and the crate’s out-of-order test only reorders messages within one chain.
    • P2, one forged message with a made-up ratchet key desyncs the session for good. The ratchet step and key skipping happen before the tag is checked, and they’re never rolled back. In the probe, the next real message failed.
    • P3, a forged message naming a skipped key deletes that key before the tag check. The real message then fails with “Message number mismatch”.
    • The signed prekey’s signature is never checked. x3dh_initiate_internal doesn’t call verify_signed_prekey, and the WASM API doesn’t expose it. Prekey authentication depends on every application remembering to do it.
    • The X3DH associated data isn’t bound to the two identities. It’s the constant “X3DH_Key_Exchange” rather than both identity keys, and the ratchet never uses it anyway.
    • The identity key and the signing key are unrelated. It uses a separate, unbound Ed25519 signing key instead of XEdDSA on the identity key.
    • The first chain gets no DH ratchet. On both sides it’s derived from the X3DH secret alone.
    • Smaller issues:
      • There’s no cap on the total number of skipped keys.
      • Cleanup drops keys in string order, not oldest first.
      • The send counter is a u32 with no overflow check.
      • The DH output isn’t checked for all-zero values from small-order keys.
      • The legacy encrypt_message helper returns the message key to JavaScript. Variants, not defects: the non-standard HKDF salts and info strings. Both sides agree on them, and the spec says so.
    -- VIOLATED: only one decrypt's skip is capped (max_skip); nothing caps
    -- the total unless the application calls cleanup, so a peer that keeps
    -- skipping grows the state without limit.
    invariant SkippedKeysAreBounded {
        for s in Sessions:
            s.skipped.count <= config.max_skipped_total
    }
    
    invariant OneTimePreKeysBelongToOneSession {
        for a in Handshakes:
            for b in Handshakes:
                a != b and a.one_time_prekey != null
                    implies a.one_time_prekey != b.one_time_prekey
    }
    
    invariant ResponderCannotSendBeforeReceiving {
        for s in Sessions:
            s.role = responder and s.receiving_ratchet_public = null
                implies s.sending_chain_key = null
    }
    
    open question "Should x3dh_initiate_internal take the signing key and signature and verify the signed prekey itself, so a caller cannot skip it?"
    
    open question "Should the identity key sign with XEdDSA, so there is one long-term key instead of an unbound X25519 and Ed25519 pair?"
    
    open question "Should decrypt work on a copy of the state and commit only after the tag verifies?"
    
    open question "Should the associated data be IK_A || IK_B and be passed into every seal and open?"