diff --git a/PLAN.md b/PLAN.md index 8fba134..ff2204e 100644 --- a/PLAN.md +++ b/PLAN.md @@ -79,7 +79,7 @@ Learned along the way: - In NixOS tests `fail(...)` also passes on "command not found": install probe tools explicitly and add a positive control per negative check. - Unbound answers `.test` locally (RFC 6761) and validates DNSSEC; tests using fake domains need both `local-zone ... transparent` and `domain-insecure` (both verified necessary; set in the test, not the module). - `modules/router/lib.nix` has IPv4 helpers (parse/format/netmask/cidr/overlap). `ipv4.subnet` is now derived from `address` + `prefixLength` (read-only). Assertions also cover pool bounds, pool containing the router address, and overlapping LANs. IPv6 helpers (prefix arithmetic for PD subnet-IDs) are still to do. -- Checks: `basic`, `pppoe`, `onestick`, `rollback`, `vlans` and `wireguard` (VM tests), `example` (the example system builds), `lib` (helper unit tests), `assertions` (broken configs must fail with exact messages). Not yet covered: NPTv6. +- Checks: `basic`, `pppoe`, `onestick`, `rollback`, `dns`, `vlans` and `wireguard` (VM tests), `example` (the example system builds), `lib` (helper unit tests), `assertions` (broken configs must fail with exact messages). Not yet covered: NPTv6. ## IPv6 design (confirmed) @@ -121,6 +121,19 @@ Learned along the way: - `examples/desktop` + `nixosConfigurations.example` is the deployable starting point (single NIC `eno1`, PPPoE VLAN 1001); it is built as a flake check. Secrets stay on the device (`/var/lib/router/pppoe-password`). - Open for real hardware: Nix is still on the device (`router-confirm` uses `nix-env`); no image/read-only root yet. +## Local DNS and DHCP reservations (confirmed) + +Needed to replace the user's hand-written router (host `antigua`, dnsmasq). Services such as k3s, Docker and Gitea stay on other hosts, so the router has no input rules for them. + +- `router.dns.domain`: local domain, authoritative on the router (`static` zone; unknown names answer NXDOMAIN, never forwarded). +- `router.dns.hosts`: name to IPv4/IPv6 addresses; short names live in the domain, full names override public DNS for that name only (split horizon). Reverse (PTR) records are generated. +- `router.dns.overrides`: like hosts but wildcard (Unbound `redirect` zone). +- `router.dns.forwardZones`: zone to resolvers; these and the local domain are `domain-insecure` (private zones under possibly signed parents). +- `lans..dhcp.reservations. = { mac; address; }`: Kea host reservations; the hostname is published in DNS. Assertions: valid MAC/hostname, address inside the LAN and not the router's, no duplicate MACs/addresses, no clash with `dns.hosts`. +- Tunnels with `dns = true` answer on the LAN addresses too: Unbound's access control admits what the tunnel's peers route (`allowedIPs`), so a site-to-site peer without a transfer address can use the resolver. +- Tests: `dns` (VM), site-to-site DNS in `wireguard`, and assertion cases; mutation-tested. +- Still missing for the `antigua` migration: IPv6 over the PPP session (ipv6cp, DHCPv6-PD on the PPP device), an extra static ULA address with RA on the LAN, tunnels without own addresses/fully trusted tunnels (partly covered), and the PPP device name (`wan` there, `pppoe-wan` here: `ddclient` must follow). + ## WireGuard (confirmed) - **Use cases**: road warrior (listening, dual-stack), site-to-site (no transfer network), and a provider VPN the router dials into (dual-stack transfer net, masquerade). All via `router.wireguard.`. @@ -130,6 +143,29 @@ Learned along the way: - **Addresses** are explicit (`addresses` list, empty for site-to-site); nothing generated. No IPv6 internet for road warriors (the delegated /64 belongs to a LAN; NPTv6 not implemented). - **Keys**: `privateKeyFile` (and `presharedKeyFile`) outside the Nix store, handed to the unprivileged systemd-networkd via `LoadCredential`, so files can stay root-only 0600. Deployment must provide them. +## Replacing `antigua` (the user's hand-written router) + +Source: the user's infra repo (host `antigua`, NixOS, dnsmasq + networkd + pppd). It is also a k3s/Docker/Gitea host today; **those services will stay on antigua / move elsewhere, so the router does not need input rules for them or trusted container interfaces.** The new box is set up next to antigua and tested by swapping the two. + +Done: PPPoE on VLAN 1001 over the shared NIC (`eno1`, LAN untagged), DNS records/domain/forward zones/DHCP reservations/DNS for site-to-site peers, WireGuard (freifunk dial-out with masquerade, family site-to-site with PSK, mobile-clients road warrior), MSS clamp. + +Open (in rough order): +1. **IPv6 over the PPP session**: antigua runs `+ipv6` and DHCPv6-PD on the PPP device itself (not on an untagged link); the module only knows native IPv6 on the uplink. Needs ipv6cp in pppd, networkd DHCPv6/RA on the PPP device, PD to the LAN. +2. **ULA on the LAN** (`fd2c:cdef:cde9::1/64`, announced by RA): needs an extra static address / ULA option per LAN. +3. **Tunnel trust and addressing**: antigua trusts `family`, `freifunk`, `mobile-clients` fully and gives two of them no address; check the schema covers this (`allowManagement`, `dns`, `internet` per tunnel; `addresses` may be empty?). +4. **PPP device name**: `wan` on antigua, `pppoe-wan` here; `ddclient` (Cloudflare, uses the interface address) must follow, or make the name configurable. +5. **Secrets**: the PPPoE credentials are plaintext in antigua's repo/store (rotate the password); here they are read from `/var/lib/router/pppoe-password`. +6. DDNS (ddclient) is not part of the module yet; configure it in the host config for now. + +## Installer and image (in progress) + +- **Delivery order**: installer ISO first (set up a second box next to antigua), `image.repart` disk image + A/B updates later. +- `router-install [--yes] [--secrets DIR] `: GPT with an ESP (label `boot`) and an ext4 root (label `nixos`), `nixos-install --system ` (nothing is built on the box), secrets copied to `/var/lib/router`. The layout contract is `nixosModules.installed-layout` (labels, systemd-boot without EFI variables so it boots via the removable fallback path, generic initrd modules, redistributable firmware); target configs import it. +- `lib.mkInstaller { configuration }` builds the ISO for a given `nixosSystem`; the live system runs sshd with that system's root keys. `packages.x86_64-linux.installer-example` is the example's ISO. +- Test `installer` (VM): the nixpkgs installer-test pattern (an installer VM writes `target.qcow2`, a second VM boots it with UEFI); checks refusal of bad input/mounted disks, the fallback boot file, router services up, secrets owned by root with 0700/0600. Mutation-tested (secret permissions, mounted-disk guard). +- Status: `nix build .#installer-example` produces a 1.6 GB ISO. **Not yet tested**: booting that ISO on real firmware (the VM test exercises `router-install`, not the live ISO), the NIC name in the example, and installing over an existing system. +- Build: `nix build .#installer-example`, write `result/iso/*.iso` to a USB stick, boot the new box (UEFI), `scp -r secrets root@:` then `router-install --secrets /root/secrets /dev/`. + ## Open items - **Web UI** (stats and possibly configuration) — to be discussed later. Considerations: read-only stats first is cheap; config editing must write declarative config (Nix/JSON) that feeds the module, not mutate the live system. diff --git a/examples/desktop/default.nix b/examples/desktop/default.nix index 7a67c49..e70d5d4 100644 --- a/examples/desktop/default.nix +++ b/examples/desktop/default.nix @@ -8,30 +8,23 @@ # untagged uplink for DHCPv6 to run on. # # Before the first deployment: -# 1. Replace the hardware section (`nixos-generate-config --show-hardware-config`). +# 1. Check the NIC name (`eno1`) with `ip link` on the target box. Disk layout and +# drivers come from the installer layout, so no hardware scan is needed. # 2. Put your SSH public key below (otherwise nobody can log in over the network). # 3. Put the PPPoE username below and the password in /var/lib/router/pppoe-password # (mode 0600, root); it is read at runtime and never enters the Nix store. { config, lib, ... }: { - # --- Hardware: replace with the generated configuration -------------------- - boot.loader.systemd-boot.enable = true; - boot.loader.efi.canTouchEfiVariables = true; - fileSystems."/" = { - device = "/dev/disk/by-label/nixos"; - fsType = "ext4"; - }; - fileSystems."/boot" = { - device = "/dev/disk/by-label/boot"; - fsType = "vfat"; - }; - # --------------------------------------------------------------------------- + imports = [ ../../modules/installer/layout.nix ]; + + # --- Hardware: generic UEFI PC. Disk layout and drivers come from the + # installer's layout module; adjust only if you do not use `router-install`. networking.hostName = "router"; system.stateVersion = "25.11"; users.users.root.openssh.authorizedKeys.keys = [ - "ssh-ed25519 AAAA-REPLACE-WITH-YOUR-PUBLIC-KEY you@workstation" + "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHro2zo4L13P2ukShg3JFga5/LYOkKeoOrUiFRJN3j82 nstinnesbeck nils.stinnesbeck@t-systems.com" ]; warnings = lib.optional (lib.any (lib.hasInfix "REPLACE") config.users.users.root.openssh.authorizedKeys.keys) '' examples/desktop: the SSH key is still the placeholder; remote deployment will not work. diff --git a/flake.nix b/flake.nix index 2b405bc..fe5aaea 100644 --- a/flake.nix +++ b/flake.nix @@ -16,8 +16,34 @@ nixosModules = { router = ./modules/router; default = self.nixosModules.router; + # Disk layout contract shared by router-install and target configs. + installed-layout = ./modules/installer/layout.nix; + # The installer pieces; `lib.mkInstaller` wires them up. + installer = ./modules/installer/install-script.nix; + installer-iso = ./modules/installer/iso.nix; }; + # Installer ISO for a given nixosSystem: + # nix build .#installer-example (see packages) + lib.mkInstaller = + { configuration }: + nixpkgs.lib.nixosSystem { + inherit (configuration.pkgs.stdenv.hostPlatform) system; + modules = [ + self.nixosModules.installer-iso + { + router.installer.system = configuration.config.system.build.toplevel; + # Reach the live system with the keys that will reach the router. + users.users.root.openssh.authorizedKeys.keys = + configuration.config.users.users.root.openssh.authorizedKeys.keys; + } + ]; + }; + + packages.x86_64-linux.installer-example = + (self.lib.mkInstaller { configuration = self.nixosConfigurations.example; }) + .config.system.build.isoImage; + # A deployable starting point; copy examples/desktop and adapt it. nixosConfigurations.example = nixpkgs.lib.nixosSystem { system = "x86_64-linux"; diff --git a/modules/installer/install-script.nix b/modules/installer/install-script.nix new file mode 100644 index 0000000..e07bf9c --- /dev/null +++ b/modules/installer/install-script.nix @@ -0,0 +1,118 @@ +# `router-install`: write a prebuilt router system to a disk. Nothing is built +# on the box; the system closure travels with the installer. +{ + config, + lib, + pkgs, + ... +}: +let + cfg = config.router.installer; + + installScript = pkgs.writeShellApplication { + name = "router-install"; + runtimeInputs = [ + config.system.build.nixos-install + pkgs.coreutils + pkgs.dosfstools + pkgs.e2fsprogs + pkgs.gptfdisk + pkgs.util-linux + ]; + text = '' + usage() { + echo "usage: router-install [--yes] [--secrets DIR] " >&2 + echo " Wipes (e.g. /dev/nvme0n1) and installs ${cfg.system}." >&2 + echo " --secrets DIR copied to /var/lib/router (PPPoE password, WireGuard keys, ...)" >&2 + exit 2 + } + + yes=0 + secrets= + disk= + while [ $# -gt 0 ]; do + case $1 in + --yes) yes=1 ;; + --secrets) shift; [ $# -gt 0 ] || usage; secrets=$1 ;; + -*) usage ;; + *) [ -z "$disk" ] || usage; disk=$1 ;; + esac + shift + done + [ -n "$disk" ] || usage + + [ "$(id -u)" -eq 0 ] || { echo "router-install: must run as root" >&2; exit 1; } + [ -b "$disk" ] || { echo "router-install: $disk is not a block device" >&2; exit 1; } + if [ -n "$secrets" ] && [ ! -d "$secrets" ]; then + echo "router-install: $secrets is not a directory" >&2 + exit 1 + fi + if lsblk -nr -o MOUNTPOINTS "$disk" | grep -q .; then + echo "router-install: $disk or one of its partitions is mounted; refusing" >&2 + exit 1 + fi + [ -d /sys/firmware/efi ] || echo "warning: not booted in UEFI mode; the box itself must boot UEFI" >&2 + + echo "About to ERASE $disk:" + lsblk -o NAME,SIZE,MODEL,FSTYPE,LABEL "$disk" + if [ "$yes" -ne 1 ]; then + printf 'Type the device name (%s) to continue: ' "$disk" + read -r answer + [ "$answer" = "$disk" ] || { echo "aborted" >&2; exit 1; } + fi + + part() { + case $disk in + *[0-9]) echo "''${disk}p$1" ;; + *) echo "$disk$1" ;; + esac + } + + echo ">> partitioning" + sgdisk --zap-all "$disk" >/dev/null + sgdisk -n 1:0:+512M -t 1:ef00 -c 1:boot -n 2:0:0 -t 2:8300 -c 2:nixos "$disk" >/dev/null + partprobe "$disk" || true + udevadm settle + + echo ">> formatting" + mkfs.fat -F 32 -n boot "$(part 1)" >/dev/null + mkfs.ext4 -q -F -L nixos "$(part 2)" + + echo ">> mounting" + mkdir -p /mnt + mount "$(part 2)" /mnt + mkdir -p /mnt/boot + mount -o umask=0077 "$(part 1)" /mnt/boot + + if [ -n "$secrets" ]; then + echo ">> copying secrets" + install -d -m 0700 /mnt/var/lib/router + cp -rT "$secrets" /mnt/var/lib/router + chown -R root:root /mnt/var/lib/router + chmod -R go-rwx /mnt/var/lib/router + else + echo "note: no --secrets given; put the PPPoE password etc. in /var/lib/router later" >&2 + fi + + echo ">> installing ${cfg.system}" + nixos-install --system ${cfg.system} --root /mnt --no-root-passwd --no-channel-copy + + echo ">> unmounting" + umount -R /mnt + sync + echo "Done. Remove the installer and reboot." + ''; + }; +in +{ + options.router.installer.system = lib.mkOption { + type = lib.types.package; + description = "The system (a `system.build.toplevel`) `router-install` writes to disk."; + }; + + config = { + environment.systemPackages = [ installScript ]; + # The closure must be on the installer itself. + system.extraDependencies = [ cfg.system ]; + }; +} diff --git a/modules/installer/iso.nix b/modules/installer/iso.nix new file mode 100644 index 0000000..7e93890 --- /dev/null +++ b/modules/installer/iso.nix @@ -0,0 +1,31 @@ +# Bootable installer ISO: the live system, `router-install` and the target +# system's closure. SSH is open to the target's root keys so secrets can be +# copied over (`scp -r secrets root@:`) before installing. +{ + config, + lib, + modulesPath, + ... +}: +{ + imports = [ + (modulesPath + "/installer/cd-dvd/installation-cd-minimal.nix") + ./install-script.nix + ]; + + isoImage.volumeID = "ROUTER_INSTALL"; + isoImage.squashfsCompression = "zstd -Xcompression-level 6"; + + services.openssh = { + enable = true; + settings.PermitRootLogin = "prohibit-password"; + }; + + users.motd = '' + + nixos-router installer + 1. (optional) scp -r secrets root@:secrets + 2. router-install --secrets /root/secrets /dev/ + The disk is erased after confirmation. + ''; +} diff --git a/modules/installer/layout.nix b/modules/installer/layout.nix new file mode 100644 index 0000000..702413a --- /dev/null +++ b/modules/installer/layout.nix @@ -0,0 +1,34 @@ +# The disk layout `router-install` creates, as a NixOS module: an ESP labelled +# `boot` and an ext4 root labelled `nixos`. Target configurations import this so +# the installed system finds its disks on any hardware. +{ lib, ... }: +{ + fileSystems."/" = { + device = "/dev/disk/by-label/nixos"; + fsType = "ext4"; + }; + fileSystems."/boot" = { + device = "/dev/disk/by-label/boot"; + fsType = "vfat"; + options = [ "umask=0077" ]; + }; + + boot.loader.systemd-boot.enable = true; + # The installer may run anywhere (even a BIOS-booted stick) and the firmware + # may be reset: boot through the removable fallback path, not NVRAM entries. + boot.loader.efi.canTouchEfiVariables = lib.mkDefault false; + + # Storage and USB controllers of common hardware, so the root is found on + # an arbitrary box (and in VMs). + boot.initrd.availableKernelModules = [ + "ahci" + "nvme" + "sd_mod" + "sdhci_pci" + "usb_storage" + "xhci_pci" + "virtio_blk" + "virtio_pci" + ]; + hardware.enableRedistributableFirmware = lib.mkDefault true; +} diff --git a/modules/router/assertions.nix b/modules/router/assertions.nix index 9ae4859..92151f5 100644 --- a/modules/router/assertions.nix +++ b/modules/router/assertions.nix @@ -7,6 +7,12 @@ let lans = lib.attrValues lansByName; validIp = s: ipv4.parse s != null; + hostnameOk = s: builtins.match "[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?" s != null; + # A DNS name: labels of letters, digits, dashes and underscores. + dnsNameOk = + s: + builtins.match "[A-Za-z0-9_]([A-Za-z0-9_-]*[A-Za-z0-9_])?(\\.[A-Za-z0-9_]([A-Za-z0-9_-]*[A-Za-z0-9_])?)*" s + != null; # Assertions about a single LAN; skipped (reported once) when its addresses # do not even parse, so we never feed null into the integer helpers. @@ -58,6 +64,60 @@ let message = "router: ${where}.dhcp.pool contains the router's own address ${lan.ipv4.address}."; } ] + ) + ++ lib.concatLists ( + lib.mapAttrsToList ( + host: r: + let + rwhere = "${where}.dhcp.reservations.${host}"; + rok = validIp r.address; + in + [ + { + assertion = hostnameOk host; + message = "router: ${rwhere}: `${host}` is not a valid hostname (letters, digits and dashes)."; + } + { + assertion = builtins.match "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}" r.mac != null; + message = "router: ${rwhere}.mac `${r.mac}` is not a MAC address like aa:bb:cc:dd:ee:ff."; + } + { + assertion = rok; + message = "router: ${rwhere}.address `${r.address}` is not a valid IPv4 address."; + } + { + assertion = lan.dhcp.enable; + message = "router: ${rwhere}: reservations need dhcp.enable on the LAN."; + } + ] + ++ lib.optionals (rok && addrOk) [ + { + assertion = ipv4.inSubnet net (ipv4.parse r.address); + message = "router: ${rwhere}.address ${r.address} is not inside ${lan.ipv4.subnet}."; + } + { + assertion = r.address != lan.ipv4.address; + message = "router: ${rwhere}.address is the router's own address."; + } + ] + ) lan.dhcp.reservations + ) + ++ ( + let + rs = lib.attrValues lan.dhcp.reservations; + macs = map (r: lib.toLower r.mac) rs; + addrs = map (r: r.address) rs; + in + [ + { + assertion = lib.length macs == lib.length (lib.unique macs); + message = "router: ${where}.dhcp.reservations: two reservations share a MAC address."; + } + { + assertion = lib.length addrs == lib.length (lib.unique addrs); + message = "router: ${where}.dhcp.reservations: two reservations share an address."; + } + ] ); wgs = lib.attrValues cfg.wireguard; @@ -234,6 +294,50 @@ let } ]; + dnsCfg = cfg.dns; + anyIp = s: validIp s || v6Cidr "${s}/128"; + dnsAssertions = + let + names = + lib.attrNames dnsCfg.hosts ++ lib.attrNames dnsCfg.overrides ++ lib.attrNames dnsCfg.forwardZones; + recordSets = dnsCfg.hosts // dnsCfg.overrides; + shortNames = lib.filter (n: !(lib.hasInfix "." n)) (lib.attrNames dnsCfg.hosts); + reservationNames = lib.concatMap (l: lib.attrNames l.dhcp.reservations) lans; + in + [ + { + assertion = lib.all dnsNameOk names && (dnsCfg.domain == null || dnsNameOk dnsCfg.domain); + message = "router.dns: a name in domain/hosts/overrides/forwardZones is not a valid DNS name."; + } + { + assertion = lib.all (as: as != [ ] && lib.all anyIp as) (lib.attrValues recordSets); + message = "router.dns: every host/override needs at least one valid IPv4 or IPv6 address."; + } + { + assertion = lib.all (as: as != [ ] && lib.all anyIp as) (lib.attrValues dnsCfg.forwardZones); + message = "router.dns.forwardZones: every zone needs at least one valid resolver address."; + } + { + assertion = dnsCfg.domain != null || shortNames == [ ]; + message = "router.dns.hosts: `${lib.concatStringsSep ", " shortNames}` has no dot, so router.dns.domain must be set."; + } + { + assertion = + lib.length (lib.attrNames dnsCfg.hosts) == lib.length ( + lib.unique ( + map (n: if lib.hasInfix "." n || dnsCfg.domain == null then n else "${n}.${dnsCfg.domain}") ( + lib.attrNames dnsCfg.hosts + ) + ) + ); + message = "router.dns.hosts: two names are the same once placed in the domain."; + } + { + assertion = lib.all (n: !(lib.elem n (lib.attrNames dnsCfg.hosts))) reservationNames; + message = "router.dns.hosts: a name is also a DHCP reservation, which already publishes it."; + } + ]; + overlapsLan = n: lib.any (l: ipv4.overlaps n l) lanNets; wgAssertions = @@ -347,6 +451,7 @@ in } ] ++ lib.concatLists (lib.mapAttrsToList perLan lansByName) + ++ dnsAssertions ++ lib.concatMap ruleAssertions cfg.firewall.rules ++ [ { diff --git a/modules/router/dhcp.nix b/modules/router/dhcp.nix index 29ea5ad..aa4cce3 100644 --- a/modules/router/dhcp.nix +++ b/modules/router/dhcp.nix @@ -27,6 +27,11 @@ in inherit (l.ipv4) subnet; interface = l.device; pools = [ { pool = "${l.dhcp.pool.start} - ${l.dhcp.pool.end}"; } ]; + reservations = lib.mapAttrsToList (hostname: r: { + hw-address = lib.toLower r.mac; + ip-address = r.address; + inherit hostname; + }) l.dhcp.reservations; option-data = [ { name = "routers"; diff --git a/modules/router/dns.nix b/modules/router/dns.nix index 43d4b9f..a0cd434 100644 --- a/modules/router/dns.nix +++ b/modules/router/dns.nix @@ -17,6 +17,37 @@ let a else ipv4.cidr (builtins.head parts) (lib.toInt (builtins.elemAt parts 1)); + + dns = cfg.dns; + inherit (dns) domain; + fqdn = name: if lib.hasInfix "." name || domain == null then name else "${name}.${domain}"; + isV6 = lib.hasInfix ":"; + rrOf = a: if isV6 a then "AAAA" else "A"; + + # name -> addresses: explicit hosts plus DHCP reservations (A records only). + reservations = lib.concatMap ( + l: lib.mapAttrsToList (n: r: lib.nameValuePair n [ r.address ]) l.dhcp.reservations + ) lans; + records = + lib.mapAttrsToList (n: a: lib.nameValuePair (fqdn n) a) dns.hosts + ++ map (r: lib.nameValuePair (fqdn r.name) r.value) reservations + ++ lib.mapAttrsToList (n: a: lib.nameValuePair n a) dns.overrides; + + # Unbound reads these as one quoted argument. + localData = lib.concatMap (r: map (a: ''"${r.name}. ${rrOf a} ${a}"'') r.value) records; + # Reverse records for hosts and reservations (not for wildcard overrides). + ptrs = lib.concatMap (r: map (a: ''"${a} ${r.name}."'') r.value) ( + lib.filter (r: !(lib.hasAttr r.name dns.overrides)) records + ); + privateZones = lib.attrNames dns.forwardZones ++ lib.optional (domain != null) domain; + + # Peers of tunnels that use the resolver may be remote networks with no + # address of their own on the tunnel (site-to-site); allow what they route. + dnsPeerNets = lib.filter (n: !lib.hasSuffix "/0" n) ( + lib.concatMap (w: lib.concatMap (p: p.allowedIPs) (lib.attrValues w.peers)) ( + lib.filter (w: w.dns) (lib.attrValues cfg.wireguard) + ) + ); in { config = lib.mkIf cfg.enable { @@ -31,16 +62,26 @@ in "127.0.0.0/8 allow" ] ++ map (l: "${l.ipv4.subnet} allow") lans - ++ map (a: "${netOf a} allow") dnsAddrs; + ++ map (a: "${netOf a} allow") dnsAddrs + ++ map (n: "${n} allow") dnsPeerNets; + local-zone = + lib.optional (domain != null) "${domain}. static" + ++ map (n: "${n}. redirect") (lib.attrNames dns.overrides); + local-data = localData; + local-data-ptr = ptrs; + domain-insecure = privateZones ++ lib.attrNames dns.overrides; }; } - // lib.optionalAttrs (cfg.dns.forwarders != [ ]) { - forward-zone = [ - { + // lib.optionalAttrs (cfg.dns.forwarders != [ ] || dns.forwardZones != { }) { + forward-zone = + lib.mapAttrsToList (name: addrs: { + inherit name; + forward-addr = addrs; + }) dns.forwardZones + ++ lib.optional (cfg.dns.forwarders != [ ]) { name = "."; forward-addr = cfg.dns.forwarders; - } - ]; + }; }; }; }; diff --git a/modules/router/options.nix b/modules/router/options.nix index 3541f8e..b669594 100644 --- a/modules/router/options.nix +++ b/modules/router/options.nix @@ -156,6 +156,29 @@ in example = "192.168.10.199"; }; }; + reservations = mkOption { + type = types.attrsOf ( + types.submodule { + options = { + mac = mkOption { + type = types.str; + example = "a8:a1:59:0b:4b:97"; + description = "Hardware address of the client."; + }; + address = mkOption { + type = types.str; + example = "192.168.10.20"; + description = "Address handed to the client; must lie inside the LAN."; + }; + }; + } + ); + default = { }; + description = '' + Fixed leases, keyed by hostname. The name is also sent to the + client and published in DNS (under `router.dns.domain` if set). + ''; + }; }; ipv6.pd = mkOption { @@ -415,6 +438,55 @@ in ); }; + dns.domain = mkOption { + type = types.nullOr types.str; + default = null; + example = "home.example.org"; + description = '' + Local domain. It is authoritative on the router: names below it that + have no record answer NXDOMAIN and are never forwarded. Short names in + `dns.hosts` and DHCP reservation names live in it. + ''; + }; + + dns.hosts = mkOption { + type = types.attrsOf (types.listOf types.str); + default = { }; + example = { + nas = [ "192.168.10.20" ]; + "git.example.org" = [ + "10.43.0.50" + "fd00::50" + ]; + }; + description = '' + Local records: name to IPv4/IPv6 addresses. A name without a dot is + placed in `dns.domain`; a full name overrides public DNS for that + name only (split horizon). Reverse records are created as well. + ''; + }; + + dns.overrides = mkOption { + type = types.attrsOf (types.listOf types.str); + default = { }; + example = { + "k8s.example.org" = [ "10.43.0.50" ]; + }; + description = "Like `dns.hosts`, but the name and everything below it answer with these addresses (wildcard)."; + }; + + dns.forwardZones = mkOption { + type = types.attrsOf (types.listOf types.str); + default = { }; + example = { + "branch.example.org" = [ "10.1.0.254" ]; + }; + description = '' + Zones answered by other resolvers (e.g. across a tunnel). DNSSEC + validation is skipped for them: they are private zones. + ''; + }; + dns.forwarders = mkOption { type = types.listOf types.str; default = [ ]; diff --git a/tests/assertions.nix b/tests/assertions.nix index d790037..3319eba 100644 --- a/tests/assertions.nix +++ b/tests/assertions.nix @@ -783,6 +783,163 @@ let }; expect = [ "used by both the PPPoE uplink and a LAN" ]; }; + dnsValid = { + router = { + lans.main = lan { + dhcp.reservations = { + nas = { + mac = "a8:a1:59:0b:4b:97"; + address = "192.168.10.20"; + }; + }; + }; + dns = { + domain = "home.example.org"; + hosts = { + printer = [ "192.168.10.30" ]; + "git.example.org" = [ + "10.43.0.50" + "fd00::50" + ]; + }; + overrides."k8s.example.org" = [ "10.43.0.50" ]; + forwardZones."branch.example.org" = [ "10.1.0.254" ]; + }; + }; + expect = [ ]; + }; + dnsShortNoDomain = { + router = { + lans.main = lan { dhcp.reservations = { }; }; + dns = { + hosts.printer = [ "192.168.10.30" ]; + }; + }; + expect = [ "router.dns.domain must be set" ]; + }; + dnsBadName = { + router = { + lans.main = lan { dhcp.reservations = { }; }; + dns = { + domain = "home.example.org"; + hosts."bad name" = [ "192.168.10.30" ]; + }; + }; + expect = [ "not a valid DNS name" ]; + }; + dnsBadAddress = { + router = { + lans.main = lan { dhcp.reservations = { }; }; + dns = { + domain = "home.example.org"; + hosts.x = [ "192.168.10.300" ]; + }; + }; + expect = [ "valid IPv4 or IPv6 address" ]; + }; + dnsBadZone = { + router = { + lans.main = lan { dhcp.reservations = { }; }; + dns = { + forwardZones."a.example.org" = [ ]; + }; + }; + expect = [ "valid resolver address" ]; + }; + dnsHostVsReservation = { + router = { + lans.main = lan { + dhcp.reservations = { + nas = { + mac = "a8:a1:59:0b:4b:97"; + address = "192.168.10.20"; + }; + }; + }; + dns = { + domain = "home.example.org"; + hosts.nas = [ "192.168.10.99" ]; + }; + }; + expect = [ "also a DHCP reservation" ]; + }; + resBadMac = { + router = { + lans.main = lan { + dhcp.reservations = { + nas = { + mac = "nonsense"; + address = "192.168.10.20"; + }; + }; + }; + dns = { }; + }; + expect = [ "is not a MAC address" ]; + }; + resOutside = { + router = { + lans.main = lan { + dhcp.reservations = { + nas = { + mac = "a8:a1:59:0b:4b:97"; + address = "192.168.99.20"; + }; + }; + }; + dns = { }; + }; + expect = [ "is not inside" ]; + }; + resRouterAddress = { + router = { + lans.main = lan { + dhcp.reservations = { + nas = { + mac = "a8:a1:59:0b:4b:97"; + address = "192.168.10.1"; + }; + }; + }; + dns = { }; + }; + expect = [ "router's own address" ]; + }; + resDuplicates = { + router = { + lans.main = lan { + dhcp.reservations = { + a = { + mac = "a8:a1:59:0b:4b:97"; + address = "192.168.10.20"; + }; + b = { + mac = "A8:A1:59:0B:4B:97"; + address = "192.168.10.20"; + }; + }; + }; + dns = { }; + }; + expect = [ + "share a MAC address" + "share an address" + ]; + }; + resBadHostname = { + router = { + lans.main = lan { + dhcp.reservations = { + "bad name" = { + mac = "a8:a1:59:0b:4b:97"; + address = "192.168.10.20"; + }; + }; + }; + dns = { }; + }; + expect = [ "not a valid hostname" ]; + }; nonOverlappingLans = { router.lans = { a = lan { }; @@ -831,6 +988,7 @@ let "wgValid" "wanValid" "sharedLinkValid" + "dnsValid" ]; subnet = diff --git a/tests/default.nix b/tests/default.nix index a1dd738..bdce0e1 100644 --- a/tests/default.nix +++ b/tests/default.nix @@ -8,6 +8,8 @@ vlans = import ./vlans.nix { inherit self pkgs; }; onestick = import ./onestick.nix { inherit self pkgs; }; rollback = import ./rollback.nix { inherit self pkgs; }; + dns = import ./dns.nix { inherit self pkgs; }; + installer = import ./installer.nix { inherit self pkgs; }; wireguard = import ./wireguard.nix { inherit self pkgs; }; lib = import ./lib.nix { inherit pkgs; }; assertions = import ./assertions.nix { inherit self pkgs; }; diff --git a/tests/dns.nix b/tests/dns.nix new file mode 100644 index 0000000..3446f82 --- /dev/null +++ b/tests/dns.nix @@ -0,0 +1,163 @@ +# Local DNS and DHCP reservations. +# +# upstream (ISP: DHCP, public DNS, a "branch" resolver) -- router -- client +# +# The router is authoritative for lan.test, answers a few names locally even +# though public DNS knows them (split horizon), forwards branch.test to its own +# resolver, and gives the client a fixed lease by MAC address. +{ self, pkgs }: +let + common = import ./common.nix; + clientMac = "02:00:00:00:00:51"; +in +pkgs.testers.runNixOSTest { + name = "router-dns"; + + defaults.imports = [ common.tools ]; + interactive.defaults = common.interactiveDefaults; + + nodes = { + upstream = + { lib, ... }: + { + virtualisation.vlans = [ 1 ]; + networking.useNetworkd = true; + networking.useDHCP = false; + networking.firewall.enable = false; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ]; + systemd.network.networks."10-wan" = { + matchConfig.Name = "eth1"; + address = [ "203.0.113.1/24" ]; + routes = [ + { + Destination = "192.168.10.0/24"; + Gateway = "203.0.113.100"; + } + ]; + }; + services.dnsmasq = { + enable = true; + settings = { + interface = "eth1"; + bind-interfaces = true; + dhcp-range = "203.0.113.100,203.0.113.100,255.255.255.0,1h"; + dhcp-option = [ "option:router,203.0.113.1" ]; + address = [ + # Public answers; the router overrides the first one locally. + "/photos.example.test/203.0.113.7" + "/other.example.test/203.0.113.8" + "/host.branch.test/10.9.9.9" + ]; + }; + }; + }; + + router = + { lib, ... }: + { + imports = [ self.nixosModules.router ]; + virtualisation.vlans = [ + 1 + 2 + ]; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ]; + networking.interfaces.eth2.ipv4.addresses = lib.mkForce [ ]; + + router = { + enable = true; + wan.interface = "eth1"; + lans.main = { + interface = "eth2"; + internet = true; + ipv4.address = "192.168.10.1"; + dhcp = { + pool = { + start = "192.168.10.100"; + end = "192.168.10.199"; + }; + reservations.printer = { + mac = clientMac; + address = "192.168.10.51"; + }; + }; + }; + dns = { + forwarders = [ "203.0.113.1" ]; + domain = "lan.test"; + hosts = { + nas = [ + "192.168.10.20" + "fd00::20" + ]; + "photos.example.test" = [ "192.168.10.50" ]; + }; + overrides."k8s.example.test" = [ "192.168.10.60" ]; + forwardZones."branch.test" = [ "203.0.113.1" ]; + }; + }; + + # `.test` is special-cased by Unbound (RFC 6761) and the fake ISP cannot + # do DNSSEC; relax both for the made-up public names used here. + services.unbound.settings.server = { + local-zone = [ "test. transparent" ]; + domain-insecure = [ "example.test" ]; + }; + }; + + client = + { lib, ... }: + { + virtualisation.vlans = [ 2 ]; + networking.useNetworkd = true; + networking.useDHCP = false; + networking.firewall.enable = false; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ]; + systemd.network.networks."10-lan" = { + matchConfig.Name = "eth1"; + linkConfig.MACAddress = "${clientMac}"; + networkConfig.DHCP = "ipv4"; + }; + }; + }; + + testScript = '' + start_all() + upstream.wait_for_unit("dnsmasq.service") + router.wait_for_unit("network-online.target") + router.wait_until_succeeds("ip -4 addr show dev eth1 | grep -q 203.0.113.100") + router.wait_for_unit("kea-dhcp4-server.service") + router.wait_for_unit("unbound.service") + + def q(name, rr="A", server="192.168.10.1"): + return client.succeed(f"dig +short +time=2 +tries=1 @{server} {name} {rr}").strip() + + with subtest("the reservation gives the client its fixed address"): + client.wait_until_succeeds("ip -4 addr show dev eth1 | grep -q 192.168.10.51") + assert "${clientMac}" in client.succeed("ip link show eth1") + # Not from the pool. + client.fail("ip -4 addr show dev eth1 | grep -q 192.168.10.1[0-9][0-9]") + + with subtest("hosts and reservations are published in the local domain"): + assert q("nas.lan.test") == "192.168.10.20", q("nas.lan.test") + assert q("nas.lan.test", "AAAA") == "fd00::20" + assert q("printer.lan.test") == "192.168.10.51" + assert q("20.10.168.192.in-addr.arpa", "PTR") == "nas.lan.test." + + with subtest("the local domain is authoritative: unknown names are not forwarded"): + # Positive control: resolving through the router works at all. + assert q("other.example.test") == "203.0.113.8" + out = client.succeed("dig +time=2 +tries=1 @192.168.10.1 nothere.lan.test") + assert "status: NXDOMAIN" in out, out + + with subtest("split horizon: only the named host is answered locally"): + assert q("photos.example.test") == "192.168.10.50", q("photos.example.test") + assert q("other.example.test") == "203.0.113.8" + + with subtest("overrides cover the name and everything below it"): + assert q("k8s.example.test") == "192.168.10.60" + assert q("deep.sub.k8s.example.test") == "192.168.10.60" + + with subtest("forward zones go to their own resolver"): + assert q("host.branch.test") == "10.9.9.9", q("host.branch.test") + ''; +} diff --git a/tests/installer.nix b/tests/installer.nix new file mode 100644 index 0000000..b25f1ba --- /dev/null +++ b/tests/installer.nix @@ -0,0 +1,132 @@ +# router-install: an installer VM writes a router system to an empty disk +# (/dev/vda), then a second VM boots that disk with UEFI and the installed +# router must come up, with its secrets in place. +# +# Follows nixos/tests/installer.nix: both machines share one disk image; the +# installer runs from a small second disk. +{ self, pkgs }: +let + inherit (pkgs) lib; + common = import ./common.nix; + + # The system to install: a minimal router on the layout the installer creates. + target = import "${pkgs.path}/nixos/lib/eval-config.nix" { + system = "x86_64-linux"; + modules = [ + self.nixosModules.router + self.nixosModules.installed-layout + # Only so the test driver can talk to the installed system. + "${pkgs.path}/nixos/modules/testing/test-instrumentation.nix" + { + system.stateVersion = "25.05"; + users.users.root.openssh.authorizedKeys.keys = [ + "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFakeKeyForTestsOnlyXXXXXXXXXXXXXXXXXXXXXXXXXX test" + ]; + router = { + enable = true; + wan.interface = "eth1"; + lans.main = { + interface = "eth2"; + internet = true; + ipv4.address = "192.168.10.1"; + dhcp.pool = { + start = "192.168.10.100"; + end = "192.168.10.199"; + }; + }; + }; + } + ]; + }; + toplevel = target.config.system.build.toplevel; + + # Both machines must see the same disk. + sharedDisk = { + virtualisation.diskImage = "./target.qcow2"; + virtualisation.diskSize = 6144; + virtualisation.memorySize = 2048; + }; +in +pkgs.testers.runNixOSTest { + name = "router-installer"; + + nodes = { + installer = + { lib, ... }: + { + imports = [ + sharedDisk + self.nixosModules.installer + common.tools + ]; + router.installer.system = toplevel; + + # Run from a small second disk so /dev/vda is the target. + virtualisation.fileSystems."/".autoFormat = true; + virtualisation.emptyDiskImages = [ 1024 ]; + virtualisation.rootDevice = "/dev/vdb"; + # No network in the sandbox; everything needed is in the closure. + nix.settings.substituters = lib.mkForce [ ]; + }; + + target = { + imports = [ sharedDisk ]; + virtualisation.vlans = [ + 1 + 2 + ]; + virtualisation.useBootLoader = true; + virtualisation.useEFIBoot = true; + virtualisation.useDefaultFilesystems = false; + virtualisation.efi.keepVariables = false; + virtualisation.fileSystems."/" = { + device = "/dev/disk/by-label/this-is-not-real-and-will-never-be-used"; + fsType = "ext4"; + }; + }; + }; + + testScript = '' + installer.start() + installer.wait_for_unit("multi-user.target") + + with subtest("router-install refuses bad input"): + installer.fail("router-install") + installer.fail("router-install --yes /etc/hostname") + # The installer's own root disk is mounted. + out = installer.fail("router-install --yes /dev/vdb 2>&1") + assert "mounted" in out, out + installer.fail("router-install --yes --secrets /nonexistent /dev/vda") + # Nothing was touched. + installer.fail("blkid /dev/vda1") + + with subtest("installation onto the empty disk"): + installer.succeed("mkdir -p /root/secrets && echo -n hunter2 > /root/secrets/pppoe-password") + installer.succeed("router-install --yes --secrets /root/secrets /dev/vda >&2") + # Layout contract with the installed-layout module. + assert "boot" in installer.succeed("blkid -s LABEL -o value /dev/vda1") + assert "nixos" in installer.succeed("blkid -s LABEL -o value /dev/vda2") + installer.shutdown() + + # The same machine, booting from the installed disk this time. + target.state_dir = installer.state_dir + target.start() + + with subtest("the installed system boots and runs the router"): + target.wait_for_unit("multi-user.target") + assert target.succeed("readlink -f /run/current-system").strip() == "${toplevel}" + target.succeed("findmnt /boot | grep -q vfat") + target.succeed("test -e /boot/EFI/BOOT/BOOTX64.EFI") + target.wait_for_unit("nftables.service") + target.wait_for_unit("unbound.service") + target.wait_for_unit("kea-dhcp4-server.service") + target.wait_until_succeeds("ip -4 addr show dev eth2 | grep -q 192.168.10.1") + target.succeed("command -v router-arm router-confirm") + + with subtest("secrets arrive with strict permissions"): + assert target.succeed("cat /var/lib/router/pppoe-password") == "hunter2" + assert target.succeed("stat -c %a /var/lib/router").strip() == "700" + assert target.succeed("stat -c %a /var/lib/router/pppoe-password").strip() == "600" + assert target.succeed("stat -c %U /var/lib/router/pppoe-password").strip() == "root" + ''; +} diff --git a/tests/wireguard.nix b/tests/wireguard.nix index 80352f5..2194f2c 100644 --- a/tests/wireguard.nix +++ b/tests/wireguard.nix @@ -223,6 +223,9 @@ pkgs.testers.runNixOSTest { privateKeyFile = "/etc/wg/s2s.key"; listenPort = 51821; zone = "remote"; + # No address on the tunnel: the site queries the LAN address and + # is allowed because it routes 192.168.50.0/24 through the tunnel. + dns = true; peers.siteb = { publicKey = key.siteb.pub; endpoint = "203.0.113.60:51821"; @@ -383,6 +386,9 @@ pkgs.testers.runNixOSTest { siteb.succeed("ping -c 1 -W 2 -I 192.168.50.1 192.168.10.1") siteb.fail("ping -c 1 -W 2 -I 192.168.50.1 192.168.10.100") + with subtest("site-to-site: the remote site can use the router's resolver"): + siteb.succeed("dig +short +time=2 +tries=1 -b 192.168.50.1 @192.168.10.1 example.test | grep -qx 203.0.113.1") + with subtest("provider tunnel: only the listed destinations, masqueraded"): out = main.succeed("curl -sf --max-time 5 http://198.18.0.1/") assert out.strip() == "10.200.0.2", f"expected masquerade to 10.200.0.2, got {out!r}"