Add local DNS records, DHCP reservations and an installer ISO

DNS and DHCP (needed to replace a dnsmasq-based router):
- router.dns.domain/hosts/overrides/forwardZones: authoritative local
  domain, split-horizon host records with reverse entries, wildcard
  overrides and per-zone forwarders (private zones skip DNSSEC).
- lans.<n>.dhcp.reservations: Kea host reservations, published in DNS.
- Tunnels with dns = true admit what their peers route, so a
  site-to-site peer without a transfer address can use the resolver.

Installer:
- router-install wipes a disk, creates an ESP and an ext4 root and
  installs a prebuilt system with its secrets; nothing is built on the
  box. installed-layout is the shared disk layout contract.
- lib.mkInstaller builds an ISO for a given nixosSystem;
  packages.installer-example is the example's ISO.

Covered by the dns and installer VM tests (the latter installs onto an
empty disk and boots it under UEFI), wireguard DNS and assertion cases.
PLAN.md records the remaining gaps for replacing the hand-written router.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-10-28 12:31:44 +01:00
co-authored by Claude Sonnet 5.5
parent 0653b5cf2f
commit b5429ad876
15 changed files with 943 additions and 21 deletions
+37 -1
View File
@@ -79,7 +79,7 @@ Learned along the way:
- In NixOS tests `fail(...)` also passes on "command not found": install probe tools explicitly and add a positive control per negative check. - In NixOS tests `fail(...)` also passes on "command not found": install probe tools explicitly and add a positive control per negative check.
- Unbound answers `.test` locally (RFC 6761) and validates DNSSEC; tests using fake domains need both `local-zone ... transparent` and `domain-insecure` (both verified necessary; set in the test, not the module). - Unbound answers `.test` locally (RFC 6761) and validates DNSSEC; tests using fake domains need both `local-zone ... transparent` and `domain-insecure` (both verified necessary; set in the test, not the module).
- `modules/router/lib.nix` has IPv4 helpers (parse/format/netmask/cidr/overlap). `ipv4.subnet` is now derived from `address` + `prefixLength` (read-only). Assertions also cover pool bounds, pool containing the router address, and overlapping LANs. IPv6 helpers (prefix arithmetic for PD subnet-IDs) are still to do. - `modules/router/lib.nix` has IPv4 helpers (parse/format/netmask/cidr/overlap). `ipv4.subnet` is now derived from `address` + `prefixLength` (read-only). Assertions also cover pool bounds, pool containing the router address, and overlapping LANs. IPv6 helpers (prefix arithmetic for PD subnet-IDs) are still to do.
- Checks: `basic`, `pppoe`, `onestick`, `rollback`, `vlans` and `wireguard` (VM tests), `example` (the example system builds), `lib` (helper unit tests), `assertions` (broken configs must fail with exact messages). Not yet covered: NPTv6. - Checks: `basic`, `pppoe`, `onestick`, `rollback`, `dns`, `vlans` and `wireguard` (VM tests), `example` (the example system builds), `lib` (helper unit tests), `assertions` (broken configs must fail with exact messages). Not yet covered: NPTv6.
## IPv6 design (confirmed) ## IPv6 design (confirmed)
@@ -121,6 +121,19 @@ Learned along the way:
- `examples/desktop` + `nixosConfigurations.example` is the deployable starting point (single NIC `eno1`, PPPoE VLAN 1001); it is built as a flake check. Secrets stay on the device (`/var/lib/router/pppoe-password`). - `examples/desktop` + `nixosConfigurations.example` is the deployable starting point (single NIC `eno1`, PPPoE VLAN 1001); it is built as a flake check. Secrets stay on the device (`/var/lib/router/pppoe-password`).
- Open for real hardware: Nix is still on the device (`router-confirm` uses `nix-env`); no image/read-only root yet. - Open for real hardware: Nix is still on the device (`router-confirm` uses `nix-env`); no image/read-only root yet.
## Local DNS and DHCP reservations (confirmed)
Needed to replace the user's hand-written router (host `antigua`, dnsmasq). Services such as k3s, Docker and Gitea stay on other hosts, so the router has no input rules for them.
- `router.dns.domain`: local domain, authoritative on the router (`static` zone; unknown names answer NXDOMAIN, never forwarded).
- `router.dns.hosts`: name to IPv4/IPv6 addresses; short names live in the domain, full names override public DNS for that name only (split horizon). Reverse (PTR) records are generated.
- `router.dns.overrides`: like hosts but wildcard (Unbound `redirect` zone).
- `router.dns.forwardZones`: zone to resolvers; these and the local domain are `domain-insecure` (private zones under possibly signed parents).
- `lans.<n>.dhcp.reservations.<hostname> = { mac; address; }`: Kea host reservations; the hostname is published in DNS. Assertions: valid MAC/hostname, address inside the LAN and not the router's, no duplicate MACs/addresses, no clash with `dns.hosts`.
- Tunnels with `dns = true` answer on the LAN addresses too: Unbound's access control admits what the tunnel's peers route (`allowedIPs`), so a site-to-site peer without a transfer address can use the resolver.
- Tests: `dns` (VM), site-to-site DNS in `wireguard`, and assertion cases; mutation-tested.
- Still missing for the `antigua` migration: IPv6 over the PPP session (ipv6cp, DHCPv6-PD on the PPP device), an extra static ULA address with RA on the LAN, tunnels without own addresses/fully trusted tunnels (partly covered), and the PPP device name (`wan` there, `pppoe-wan` here: `ddclient` must follow).
## WireGuard (confirmed) ## WireGuard (confirmed)
- **Use cases**: road warrior (listening, dual-stack), site-to-site (no transfer network), and a provider VPN the router dials into (dual-stack transfer net, masquerade). All via `router.wireguard.<name>`. - **Use cases**: road warrior (listening, dual-stack), site-to-site (no transfer network), and a provider VPN the router dials into (dual-stack transfer net, masquerade). All via `router.wireguard.<name>`.
@@ -130,6 +143,29 @@ Learned along the way:
- **Addresses** are explicit (`addresses` list, empty for site-to-site); nothing generated. No IPv6 internet for road warriors (the delegated /64 belongs to a LAN; NPTv6 not implemented). - **Addresses** are explicit (`addresses` list, empty for site-to-site); nothing generated. No IPv6 internet for road warriors (the delegated /64 belongs to a LAN; NPTv6 not implemented).
- **Keys**: `privateKeyFile` (and `presharedKeyFile`) outside the Nix store, handed to the unprivileged systemd-networkd via `LoadCredential`, so files can stay root-only 0600. Deployment must provide them. - **Keys**: `privateKeyFile` (and `presharedKeyFile`) outside the Nix store, handed to the unprivileged systemd-networkd via `LoadCredential`, so files can stay root-only 0600. Deployment must provide them.
## Replacing `antigua` (the user's hand-written router)
Source: the user's infra repo (host `antigua`, NixOS, dnsmasq + networkd + pppd). It is also a k3s/Docker/Gitea host today; **those services will stay on antigua / move elsewhere, so the router does not need input rules for them or trusted container interfaces.** The new box is set up next to antigua and tested by swapping the two.
Done: PPPoE on VLAN 1001 over the shared NIC (`eno1`, LAN untagged), DNS records/domain/forward zones/DHCP reservations/DNS for site-to-site peers, WireGuard (freifunk dial-out with masquerade, family site-to-site with PSK, mobile-clients road warrior), MSS clamp.
Open (in rough order):
1. **IPv6 over the PPP session**: antigua runs `+ipv6` and DHCPv6-PD on the PPP device itself (not on an untagged link); the module only knows native IPv6 on the uplink. Needs ipv6cp in pppd, networkd DHCPv6/RA on the PPP device, PD to the LAN.
2. **ULA on the LAN** (`fd2c:cdef:cde9::1/64`, announced by RA): needs an extra static address / ULA option per LAN.
3. **Tunnel trust and addressing**: antigua trusts `family`, `freifunk`, `mobile-clients` fully and gives two of them no address; check the schema covers this (`allowManagement`, `dns`, `internet` per tunnel; `addresses` may be empty?).
4. **PPP device name**: `wan` on antigua, `pppoe-wan` here; `ddclient` (Cloudflare, uses the interface address) must follow, or make the name configurable.
5. **Secrets**: the PPPoE credentials are plaintext in antigua's repo/store (rotate the password); here they are read from `/var/lib/router/pppoe-password`.
6. DDNS (ddclient) is not part of the module yet; configure it in the host config for now.
## Installer and image (in progress)
- **Delivery order**: installer ISO first (set up a second box next to antigua), `image.repart` disk image + A/B updates later.
- `router-install [--yes] [--secrets DIR] <disk>`: GPT with an ESP (label `boot`) and an ext4 root (label `nixos`), `nixos-install --system <prebuilt toplevel>` (nothing is built on the box), secrets copied to `/var/lib/router`. The layout contract is `nixosModules.installed-layout` (labels, systemd-boot without EFI variables so it boots via the removable fallback path, generic initrd modules, redistributable firmware); target configs import it.
- `lib.mkInstaller { configuration }` builds the ISO for a given `nixosSystem`; the live system runs sshd with that system's root keys. `packages.x86_64-linux.installer-example` is the example's ISO.
- Test `installer` (VM): the nixpkgs installer-test pattern (an installer VM writes `target.qcow2`, a second VM boots it with UEFI); checks refusal of bad input/mounted disks, the fallback boot file, router services up, secrets owned by root with 0700/0600. Mutation-tested (secret permissions, mounted-disk guard).
- Status: `nix build .#installer-example` produces a 1.6 GB ISO. **Not yet tested**: booting that ISO on real firmware (the VM test exercises `router-install`, not the live ISO), the NIC name in the example, and installing over an existing system.
- Build: `nix build .#installer-example`, write `result/iso/*.iso` to a USB stick, boot the new box (UEFI), `scp -r secrets root@<box>:` then `router-install --secrets /root/secrets /dev/<disk>`.
## Open items ## Open items
- **Web UI** (stats and possibly configuration) — to be discussed later. Considerations: read-only stats first is cheap; config editing must write declarative config (Nix/JSON) that feeds the module, not mutate the live system. - **Web UI** (stats and possibly configuration) — to be discussed later. Considerations: read-only stats first is cheap; config editing must write declarative config (Nix/JSON) that feeds the module, not mutate the live system.
+7 -14
View File
@@ -8,30 +8,23 @@
# untagged uplink for DHCPv6 to run on. # untagged uplink for DHCPv6 to run on.
# #
# Before the first deployment: # Before the first deployment:
# 1. Replace the hardware section (`nixos-generate-config --show-hardware-config`). # 1. Check the NIC name (`eno1`) with `ip link` on the target box. Disk layout and
# drivers come from the installer layout, so no hardware scan is needed.
# 2. Put your SSH public key below (otherwise nobody can log in over the network). # 2. Put your SSH public key below (otherwise nobody can log in over the network).
# 3. Put the PPPoE username below and the password in /var/lib/router/pppoe-password # 3. Put the PPPoE username below and the password in /var/lib/router/pppoe-password
# (mode 0600, root); it is read at runtime and never enters the Nix store. # (mode 0600, root); it is read at runtime and never enters the Nix store.
{ config, lib, ... }: { config, lib, ... }:
{ {
# --- Hardware: replace with the generated configuration -------------------- imports = [ ../../modules/installer/layout.nix ];
boot.loader.systemd-boot.enable = true;
boot.loader.efi.canTouchEfiVariables = true; # --- Hardware: generic UEFI PC. Disk layout and drivers come from the
fileSystems."/" = { # installer's layout module; adjust only if you do not use `router-install`.
device = "/dev/disk/by-label/nixos";
fsType = "ext4";
};
fileSystems."/boot" = {
device = "/dev/disk/by-label/boot";
fsType = "vfat";
};
# ---------------------------------------------------------------------------
networking.hostName = "router"; networking.hostName = "router";
system.stateVersion = "25.11"; system.stateVersion = "25.11";
users.users.root.openssh.authorizedKeys.keys = [ users.users.root.openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAA-REPLACE-WITH-YOUR-PUBLIC-KEY you@workstation" "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHro2zo4L13P2ukShg3JFga5/LYOkKeoOrUiFRJN3j82 nstinnesbeck nils.stinnesbeck@t-systems.com"
]; ];
warnings = lib.optional (lib.any (lib.hasInfix "REPLACE") config.users.users.root.openssh.authorizedKeys.keys) '' warnings = lib.optional (lib.any (lib.hasInfix "REPLACE") config.users.users.root.openssh.authorizedKeys.keys) ''
examples/desktop: the SSH key is still the placeholder; remote deployment will not work. examples/desktop: the SSH key is still the placeholder; remote deployment will not work.
+26
View File
@@ -16,8 +16,34 @@
nixosModules = { nixosModules = {
router = ./modules/router; router = ./modules/router;
default = self.nixosModules.router; default = self.nixosModules.router;
# Disk layout contract shared by router-install and target configs.
installed-layout = ./modules/installer/layout.nix;
# The installer pieces; `lib.mkInstaller` wires them up.
installer = ./modules/installer/install-script.nix;
installer-iso = ./modules/installer/iso.nix;
}; };
# Installer ISO for a given nixosSystem:
# nix build .#installer-example (see packages)
lib.mkInstaller =
{ configuration }:
nixpkgs.lib.nixosSystem {
inherit (configuration.pkgs.stdenv.hostPlatform) system;
modules = [
self.nixosModules.installer-iso
{
router.installer.system = configuration.config.system.build.toplevel;
# Reach the live system with the keys that will reach the router.
users.users.root.openssh.authorizedKeys.keys =
configuration.config.users.users.root.openssh.authorizedKeys.keys;
}
];
};
packages.x86_64-linux.installer-example =
(self.lib.mkInstaller { configuration = self.nixosConfigurations.example; })
.config.system.build.isoImage;
# A deployable starting point; copy examples/desktop and adapt it. # A deployable starting point; copy examples/desktop and adapt it.
nixosConfigurations.example = nixpkgs.lib.nixosSystem { nixosConfigurations.example = nixpkgs.lib.nixosSystem {
system = "x86_64-linux"; system = "x86_64-linux";
+118
View File
@@ -0,0 +1,118 @@
# `router-install`: write a prebuilt router system to a disk. Nothing is built
# on the box; the system closure travels with the installer.
{
config,
lib,
pkgs,
...
}:
let
cfg = config.router.installer;
installScript = pkgs.writeShellApplication {
name = "router-install";
runtimeInputs = [
config.system.build.nixos-install
pkgs.coreutils
pkgs.dosfstools
pkgs.e2fsprogs
pkgs.gptfdisk
pkgs.util-linux
];
text = ''
usage() {
echo "usage: router-install [--yes] [--secrets DIR] <disk>" >&2
echo " Wipes <disk> (e.g. /dev/nvme0n1) and installs ${cfg.system}." >&2
echo " --secrets DIR copied to /var/lib/router (PPPoE password, WireGuard keys, ...)" >&2
exit 2
}
yes=0
secrets=
disk=
while [ $# -gt 0 ]; do
case $1 in
--yes) yes=1 ;;
--secrets) shift; [ $# -gt 0 ] || usage; secrets=$1 ;;
-*) usage ;;
*) [ -z "$disk" ] || usage; disk=$1 ;;
esac
shift
done
[ -n "$disk" ] || usage
[ "$(id -u)" -eq 0 ] || { echo "router-install: must run as root" >&2; exit 1; }
[ -b "$disk" ] || { echo "router-install: $disk is not a block device" >&2; exit 1; }
if [ -n "$secrets" ] && [ ! -d "$secrets" ]; then
echo "router-install: $secrets is not a directory" >&2
exit 1
fi
if lsblk -nr -o MOUNTPOINTS "$disk" | grep -q .; then
echo "router-install: $disk or one of its partitions is mounted; refusing" >&2
exit 1
fi
[ -d /sys/firmware/efi ] || echo "warning: not booted in UEFI mode; the box itself must boot UEFI" >&2
echo "About to ERASE $disk:"
lsblk -o NAME,SIZE,MODEL,FSTYPE,LABEL "$disk"
if [ "$yes" -ne 1 ]; then
printf 'Type the device name (%s) to continue: ' "$disk"
read -r answer
[ "$answer" = "$disk" ] || { echo "aborted" >&2; exit 1; }
fi
part() {
case $disk in
*[0-9]) echo "''${disk}p$1" ;;
*) echo "$disk$1" ;;
esac
}
echo ">> partitioning"
sgdisk --zap-all "$disk" >/dev/null
sgdisk -n 1:0:+512M -t 1:ef00 -c 1:boot -n 2:0:0 -t 2:8300 -c 2:nixos "$disk" >/dev/null
partprobe "$disk" || true
udevadm settle
echo ">> formatting"
mkfs.fat -F 32 -n boot "$(part 1)" >/dev/null
mkfs.ext4 -q -F -L nixos "$(part 2)"
echo ">> mounting"
mkdir -p /mnt
mount "$(part 2)" /mnt
mkdir -p /mnt/boot
mount -o umask=0077 "$(part 1)" /mnt/boot
if [ -n "$secrets" ]; then
echo ">> copying secrets"
install -d -m 0700 /mnt/var/lib/router
cp -rT "$secrets" /mnt/var/lib/router
chown -R root:root /mnt/var/lib/router
chmod -R go-rwx /mnt/var/lib/router
else
echo "note: no --secrets given; put the PPPoE password etc. in /var/lib/router later" >&2
fi
echo ">> installing ${cfg.system}"
nixos-install --system ${cfg.system} --root /mnt --no-root-passwd --no-channel-copy
echo ">> unmounting"
umount -R /mnt
sync
echo "Done. Remove the installer and reboot."
'';
};
in
{
options.router.installer.system = lib.mkOption {
type = lib.types.package;
description = "The system (a `system.build.toplevel`) `router-install` writes to disk.";
};
config = {
environment.systemPackages = [ installScript ];
# The closure must be on the installer itself.
system.extraDependencies = [ cfg.system ];
};
}
+31
View File
@@ -0,0 +1,31 @@
# Bootable installer ISO: the live system, `router-install` and the target
# system's closure. SSH is open to the target's root keys so secrets can be
# copied over (`scp -r secrets root@<installer>:`) before installing.
{
config,
lib,
modulesPath,
...
}:
{
imports = [
(modulesPath + "/installer/cd-dvd/installation-cd-minimal.nix")
./install-script.nix
];
isoImage.volumeID = "ROUTER_INSTALL";
isoImage.squashfsCompression = "zstd -Xcompression-level 6";
services.openssh = {
enable = true;
settings.PermitRootLogin = "prohibit-password";
};
users.motd = ''
nixos-router installer
1. (optional) scp -r secrets root@<this-host>:secrets
2. router-install --secrets /root/secrets /dev/<disk>
The disk is erased after confirmation.
'';
}
+34
View File
@@ -0,0 +1,34 @@
# The disk layout `router-install` creates, as a NixOS module: an ESP labelled
# `boot` and an ext4 root labelled `nixos`. Target configurations import this so
# the installed system finds its disks on any hardware.
{ lib, ... }:
{
fileSystems."/" = {
device = "/dev/disk/by-label/nixos";
fsType = "ext4";
};
fileSystems."/boot" = {
device = "/dev/disk/by-label/boot";
fsType = "vfat";
options = [ "umask=0077" ];
};
boot.loader.systemd-boot.enable = true;
# The installer may run anywhere (even a BIOS-booted stick) and the firmware
# may be reset: boot through the removable fallback path, not NVRAM entries.
boot.loader.efi.canTouchEfiVariables = lib.mkDefault false;
# Storage and USB controllers of common hardware, so the root is found on
# an arbitrary box (and in VMs).
boot.initrd.availableKernelModules = [
"ahci"
"nvme"
"sd_mod"
"sdhci_pci"
"usb_storage"
"xhci_pci"
"virtio_blk"
"virtio_pci"
];
hardware.enableRedistributableFirmware = lib.mkDefault true;
}
+105
View File
@@ -7,6 +7,12 @@ let
lans = lib.attrValues lansByName; lans = lib.attrValues lansByName;
validIp = s: ipv4.parse s != null; validIp = s: ipv4.parse s != null;
hostnameOk = s: builtins.match "[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?" s != null;
# A DNS name: labels of letters, digits, dashes and underscores.
dnsNameOk =
s:
builtins.match "[A-Za-z0-9_]([A-Za-z0-9_-]*[A-Za-z0-9_])?(\\.[A-Za-z0-9_]([A-Za-z0-9_-]*[A-Za-z0-9_])?)*" s
!= null;
# Assertions about a single LAN; skipped (reported once) when its addresses # Assertions about a single LAN; skipped (reported once) when its addresses
# do not even parse, so we never feed null into the integer helpers. # do not even parse, so we never feed null into the integer helpers.
@@ -58,6 +64,60 @@ let
message = "router: ${where}.dhcp.pool contains the router's own address ${lan.ipv4.address}."; message = "router: ${where}.dhcp.pool contains the router's own address ${lan.ipv4.address}.";
} }
] ]
)
++ lib.concatLists (
lib.mapAttrsToList (
host: r:
let
rwhere = "${where}.dhcp.reservations.${host}";
rok = validIp r.address;
in
[
{
assertion = hostnameOk host;
message = "router: ${rwhere}: `${host}` is not a valid hostname (letters, digits and dashes).";
}
{
assertion = builtins.match "([0-9a-fA-F]{2}:){5}[0-9a-fA-F]{2}" r.mac != null;
message = "router: ${rwhere}.mac `${r.mac}` is not a MAC address like aa:bb:cc:dd:ee:ff.";
}
{
assertion = rok;
message = "router: ${rwhere}.address `${r.address}` is not a valid IPv4 address.";
}
{
assertion = lan.dhcp.enable;
message = "router: ${rwhere}: reservations need dhcp.enable on the LAN.";
}
]
++ lib.optionals (rok && addrOk) [
{
assertion = ipv4.inSubnet net (ipv4.parse r.address);
message = "router: ${rwhere}.address ${r.address} is not inside ${lan.ipv4.subnet}.";
}
{
assertion = r.address != lan.ipv4.address;
message = "router: ${rwhere}.address is the router's own address.";
}
]
) lan.dhcp.reservations
)
++ (
let
rs = lib.attrValues lan.dhcp.reservations;
macs = map (r: lib.toLower r.mac) rs;
addrs = map (r: r.address) rs;
in
[
{
assertion = lib.length macs == lib.length (lib.unique macs);
message = "router: ${where}.dhcp.reservations: two reservations share a MAC address.";
}
{
assertion = lib.length addrs == lib.length (lib.unique addrs);
message = "router: ${where}.dhcp.reservations: two reservations share an address.";
}
]
); );
wgs = lib.attrValues cfg.wireguard; wgs = lib.attrValues cfg.wireguard;
@@ -234,6 +294,50 @@ let
} }
]; ];
dnsCfg = cfg.dns;
anyIp = s: validIp s || v6Cidr "${s}/128";
dnsAssertions =
let
names =
lib.attrNames dnsCfg.hosts ++ lib.attrNames dnsCfg.overrides ++ lib.attrNames dnsCfg.forwardZones;
recordSets = dnsCfg.hosts // dnsCfg.overrides;
shortNames = lib.filter (n: !(lib.hasInfix "." n)) (lib.attrNames dnsCfg.hosts);
reservationNames = lib.concatMap (l: lib.attrNames l.dhcp.reservations) lans;
in
[
{
assertion = lib.all dnsNameOk names && (dnsCfg.domain == null || dnsNameOk dnsCfg.domain);
message = "router.dns: a name in domain/hosts/overrides/forwardZones is not a valid DNS name.";
}
{
assertion = lib.all (as: as != [ ] && lib.all anyIp as) (lib.attrValues recordSets);
message = "router.dns: every host/override needs at least one valid IPv4 or IPv6 address.";
}
{
assertion = lib.all (as: as != [ ] && lib.all anyIp as) (lib.attrValues dnsCfg.forwardZones);
message = "router.dns.forwardZones: every zone needs at least one valid resolver address.";
}
{
assertion = dnsCfg.domain != null || shortNames == [ ];
message = "router.dns.hosts: `${lib.concatStringsSep ", " shortNames}` has no dot, so router.dns.domain must be set.";
}
{
assertion =
lib.length (lib.attrNames dnsCfg.hosts) == lib.length (
lib.unique (
map (n: if lib.hasInfix "." n || dnsCfg.domain == null then n else "${n}.${dnsCfg.domain}") (
lib.attrNames dnsCfg.hosts
)
)
);
message = "router.dns.hosts: two names are the same once placed in the domain.";
}
{
assertion = lib.all (n: !(lib.elem n (lib.attrNames dnsCfg.hosts))) reservationNames;
message = "router.dns.hosts: a name is also a DHCP reservation, which already publishes it.";
}
];
overlapsLan = n: lib.any (l: ipv4.overlaps n l) lanNets; overlapsLan = n: lib.any (l: ipv4.overlaps n l) lanNets;
wgAssertions = wgAssertions =
@@ -347,6 +451,7 @@ in
} }
] ]
++ lib.concatLists (lib.mapAttrsToList perLan lansByName) ++ lib.concatLists (lib.mapAttrsToList perLan lansByName)
++ dnsAssertions
++ lib.concatMap ruleAssertions cfg.firewall.rules ++ lib.concatMap ruleAssertions cfg.firewall.rules
++ [ ++ [
{ {
+5
View File
@@ -27,6 +27,11 @@ in
inherit (l.ipv4) subnet; inherit (l.ipv4) subnet;
interface = l.device; interface = l.device;
pools = [ { pool = "${l.dhcp.pool.start} - ${l.dhcp.pool.end}"; } ]; pools = [ { pool = "${l.dhcp.pool.start} - ${l.dhcp.pool.end}"; } ];
reservations = lib.mapAttrsToList (hostname: r: {
hw-address = lib.toLower r.mac;
ip-address = r.address;
inherit hostname;
}) l.dhcp.reservations;
option-data = [ option-data = [
{ {
name = "routers"; name = "routers";
+47 -6
View File
@@ -17,6 +17,37 @@ let
a a
else else
ipv4.cidr (builtins.head parts) (lib.toInt (builtins.elemAt parts 1)); ipv4.cidr (builtins.head parts) (lib.toInt (builtins.elemAt parts 1));
dns = cfg.dns;
inherit (dns) domain;
fqdn = name: if lib.hasInfix "." name || domain == null then name else "${name}.${domain}";
isV6 = lib.hasInfix ":";
rrOf = a: if isV6 a then "AAAA" else "A";
# name -> addresses: explicit hosts plus DHCP reservations (A records only).
reservations = lib.concatMap (
l: lib.mapAttrsToList (n: r: lib.nameValuePair n [ r.address ]) l.dhcp.reservations
) lans;
records =
lib.mapAttrsToList (n: a: lib.nameValuePair (fqdn n) a) dns.hosts
++ map (r: lib.nameValuePair (fqdn r.name) r.value) reservations
++ lib.mapAttrsToList (n: a: lib.nameValuePair n a) dns.overrides;
# Unbound reads these as one quoted argument.
localData = lib.concatMap (r: map (a: ''"${r.name}. ${rrOf a} ${a}"'') r.value) records;
# Reverse records for hosts and reservations (not for wildcard overrides).
ptrs = lib.concatMap (r: map (a: ''"${a} ${r.name}."'') r.value) (
lib.filter (r: !(lib.hasAttr r.name dns.overrides)) records
);
privateZones = lib.attrNames dns.forwardZones ++ lib.optional (domain != null) domain;
# Peers of tunnels that use the resolver may be remote networks with no
# address of their own on the tunnel (site-to-site); allow what they route.
dnsPeerNets = lib.filter (n: !lib.hasSuffix "/0" n) (
lib.concatMap (w: lib.concatMap (p: p.allowedIPs) (lib.attrValues w.peers)) (
lib.filter (w: w.dns) (lib.attrValues cfg.wireguard)
)
);
in in
{ {
config = lib.mkIf cfg.enable { config = lib.mkIf cfg.enable {
@@ -31,16 +62,26 @@ in
"127.0.0.0/8 allow" "127.0.0.0/8 allow"
] ]
++ map (l: "${l.ipv4.subnet} allow") lans ++ map (l: "${l.ipv4.subnet} allow") lans
++ map (a: "${netOf a} allow") dnsAddrs; ++ map (a: "${netOf a} allow") dnsAddrs
++ map (n: "${n} allow") dnsPeerNets;
local-zone =
lib.optional (domain != null) "${domain}. static"
++ map (n: "${n}. redirect") (lib.attrNames dns.overrides);
local-data = localData;
local-data-ptr = ptrs;
domain-insecure = privateZones ++ lib.attrNames dns.overrides;
}; };
} }
// lib.optionalAttrs (cfg.dns.forwarders != [ ]) { // lib.optionalAttrs (cfg.dns.forwarders != [ ] || dns.forwardZones != { }) {
forward-zone = [ forward-zone =
{ lib.mapAttrsToList (name: addrs: {
inherit name;
forward-addr = addrs;
}) dns.forwardZones
++ lib.optional (cfg.dns.forwarders != [ ]) {
name = "."; name = ".";
forward-addr = cfg.dns.forwarders; forward-addr = cfg.dns.forwarders;
} };
];
}; };
}; };
}; };
+72
View File
@@ -156,6 +156,29 @@ in
example = "192.168.10.199"; example = "192.168.10.199";
}; };
}; };
reservations = mkOption {
type = types.attrsOf (
types.submodule {
options = {
mac = mkOption {
type = types.str;
example = "a8:a1:59:0b:4b:97";
description = "Hardware address of the client.";
};
address = mkOption {
type = types.str;
example = "192.168.10.20";
description = "Address handed to the client; must lie inside the LAN.";
};
};
}
);
default = { };
description = ''
Fixed leases, keyed by hostname. The name is also sent to the
client and published in DNS (under `router.dns.domain` if set).
'';
};
}; };
ipv6.pd = mkOption { ipv6.pd = mkOption {
@@ -415,6 +438,55 @@ in
); );
}; };
dns.domain = mkOption {
type = types.nullOr types.str;
default = null;
example = "home.example.org";
description = ''
Local domain. It is authoritative on the router: names below it that
have no record answer NXDOMAIN and are never forwarded. Short names in
`dns.hosts` and DHCP reservation names live in it.
'';
};
dns.hosts = mkOption {
type = types.attrsOf (types.listOf types.str);
default = { };
example = {
nas = [ "192.168.10.20" ];
"git.example.org" = [
"10.43.0.50"
"fd00::50"
];
};
description = ''
Local records: name to IPv4/IPv6 addresses. A name without a dot is
placed in `dns.domain`; a full name overrides public DNS for that
name only (split horizon). Reverse records are created as well.
'';
};
dns.overrides = mkOption {
type = types.attrsOf (types.listOf types.str);
default = { };
example = {
"k8s.example.org" = [ "10.43.0.50" ];
};
description = "Like `dns.hosts`, but the name and everything below it answer with these addresses (wildcard).";
};
dns.forwardZones = mkOption {
type = types.attrsOf (types.listOf types.str);
default = { };
example = {
"branch.example.org" = [ "10.1.0.254" ];
};
description = ''
Zones answered by other resolvers (e.g. across a tunnel). DNSSEC
validation is skipped for them: they are private zones.
'';
};
dns.forwarders = mkOption { dns.forwarders = mkOption {
type = types.listOf types.str; type = types.listOf types.str;
default = [ ]; default = [ ];
+158
View File
@@ -783,6 +783,163 @@ let
}; };
expect = [ "used by both the PPPoE uplink and a LAN" ]; expect = [ "used by both the PPPoE uplink and a LAN" ];
}; };
dnsValid = {
router = {
lans.main = lan {
dhcp.reservations = {
nas = {
mac = "a8:a1:59:0b:4b:97";
address = "192.168.10.20";
};
};
};
dns = {
domain = "home.example.org";
hosts = {
printer = [ "192.168.10.30" ];
"git.example.org" = [
"10.43.0.50"
"fd00::50"
];
};
overrides."k8s.example.org" = [ "10.43.0.50" ];
forwardZones."branch.example.org" = [ "10.1.0.254" ];
};
};
expect = [ ];
};
dnsShortNoDomain = {
router = {
lans.main = lan { dhcp.reservations = { }; };
dns = {
hosts.printer = [ "192.168.10.30" ];
};
};
expect = [ "router.dns.domain must be set" ];
};
dnsBadName = {
router = {
lans.main = lan { dhcp.reservations = { }; };
dns = {
domain = "home.example.org";
hosts."bad name" = [ "192.168.10.30" ];
};
};
expect = [ "not a valid DNS name" ];
};
dnsBadAddress = {
router = {
lans.main = lan { dhcp.reservations = { }; };
dns = {
domain = "home.example.org";
hosts.x = [ "192.168.10.300" ];
};
};
expect = [ "valid IPv4 or IPv6 address" ];
};
dnsBadZone = {
router = {
lans.main = lan { dhcp.reservations = { }; };
dns = {
forwardZones."a.example.org" = [ ];
};
};
expect = [ "valid resolver address" ];
};
dnsHostVsReservation = {
router = {
lans.main = lan {
dhcp.reservations = {
nas = {
mac = "a8:a1:59:0b:4b:97";
address = "192.168.10.20";
};
};
};
dns = {
domain = "home.example.org";
hosts.nas = [ "192.168.10.99" ];
};
};
expect = [ "also a DHCP reservation" ];
};
resBadMac = {
router = {
lans.main = lan {
dhcp.reservations = {
nas = {
mac = "nonsense";
address = "192.168.10.20";
};
};
};
dns = { };
};
expect = [ "is not a MAC address" ];
};
resOutside = {
router = {
lans.main = lan {
dhcp.reservations = {
nas = {
mac = "a8:a1:59:0b:4b:97";
address = "192.168.99.20";
};
};
};
dns = { };
};
expect = [ "is not inside" ];
};
resRouterAddress = {
router = {
lans.main = lan {
dhcp.reservations = {
nas = {
mac = "a8:a1:59:0b:4b:97";
address = "192.168.10.1";
};
};
};
dns = { };
};
expect = [ "router's own address" ];
};
resDuplicates = {
router = {
lans.main = lan {
dhcp.reservations = {
a = {
mac = "a8:a1:59:0b:4b:97";
address = "192.168.10.20";
};
b = {
mac = "A8:A1:59:0B:4B:97";
address = "192.168.10.20";
};
};
};
dns = { };
};
expect = [
"share a MAC address"
"share an address"
];
};
resBadHostname = {
router = {
lans.main = lan {
dhcp.reservations = {
"bad name" = {
mac = "a8:a1:59:0b:4b:97";
address = "192.168.10.20";
};
};
};
dns = { };
};
expect = [ "not a valid hostname" ];
};
nonOverlappingLans = { nonOverlappingLans = {
router.lans = { router.lans = {
a = lan { }; a = lan { };
@@ -831,6 +988,7 @@ let
"wgValid" "wgValid"
"wanValid" "wanValid"
"sharedLinkValid" "sharedLinkValid"
"dnsValid"
]; ];
subnet = subnet =
+2
View File
@@ -8,6 +8,8 @@
vlans = import ./vlans.nix { inherit self pkgs; }; vlans = import ./vlans.nix { inherit self pkgs; };
onestick = import ./onestick.nix { inherit self pkgs; }; onestick = import ./onestick.nix { inherit self pkgs; };
rollback = import ./rollback.nix { inherit self pkgs; }; rollback = import ./rollback.nix { inherit self pkgs; };
dns = import ./dns.nix { inherit self pkgs; };
installer = import ./installer.nix { inherit self pkgs; };
wireguard = import ./wireguard.nix { inherit self pkgs; }; wireguard = import ./wireguard.nix { inherit self pkgs; };
lib = import ./lib.nix { inherit pkgs; }; lib = import ./lib.nix { inherit pkgs; };
assertions = import ./assertions.nix { inherit self pkgs; }; assertions = import ./assertions.nix { inherit self pkgs; };
+163
View File
@@ -0,0 +1,163 @@
# Local DNS and DHCP reservations.
#
# upstream (ISP: DHCP, public DNS, a "branch" resolver) -- router -- client
#
# The router is authoritative for lan.test, answers a few names locally even
# though public DNS knows them (split horizon), forwards branch.test to its own
# resolver, and gives the client a fixed lease by MAC address.
{ self, pkgs }:
let
common = import ./common.nix;
clientMac = "02:00:00:00:00:51";
in
pkgs.testers.runNixOSTest {
name = "router-dns";
defaults.imports = [ common.tools ];
interactive.defaults = common.interactiveDefaults;
nodes = {
upstream =
{ lib, ... }:
{
virtualisation.vlans = [ 1 ];
networking.useNetworkd = true;
networking.useDHCP = false;
networking.firewall.enable = false;
networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ];
systemd.network.networks."10-wan" = {
matchConfig.Name = "eth1";
address = [ "203.0.113.1/24" ];
routes = [
{
Destination = "192.168.10.0/24";
Gateway = "203.0.113.100";
}
];
};
services.dnsmasq = {
enable = true;
settings = {
interface = "eth1";
bind-interfaces = true;
dhcp-range = "203.0.113.100,203.0.113.100,255.255.255.0,1h";
dhcp-option = [ "option:router,203.0.113.1" ];
address = [
# Public answers; the router overrides the first one locally.
"/photos.example.test/203.0.113.7"
"/other.example.test/203.0.113.8"
"/host.branch.test/10.9.9.9"
];
};
};
};
router =
{ lib, ... }:
{
imports = [ self.nixosModules.router ];
virtualisation.vlans = [
1
2
];
networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ];
networking.interfaces.eth2.ipv4.addresses = lib.mkForce [ ];
router = {
enable = true;
wan.interface = "eth1";
lans.main = {
interface = "eth2";
internet = true;
ipv4.address = "192.168.10.1";
dhcp = {
pool = {
start = "192.168.10.100";
end = "192.168.10.199";
};
reservations.printer = {
mac = clientMac;
address = "192.168.10.51";
};
};
};
dns = {
forwarders = [ "203.0.113.1" ];
domain = "lan.test";
hosts = {
nas = [
"192.168.10.20"
"fd00::20"
];
"photos.example.test" = [ "192.168.10.50" ];
};
overrides."k8s.example.test" = [ "192.168.10.60" ];
forwardZones."branch.test" = [ "203.0.113.1" ];
};
};
# `.test` is special-cased by Unbound (RFC 6761) and the fake ISP cannot
# do DNSSEC; relax both for the made-up public names used here.
services.unbound.settings.server = {
local-zone = [ "test. transparent" ];
domain-insecure = [ "example.test" ];
};
};
client =
{ lib, ... }:
{
virtualisation.vlans = [ 2 ];
networking.useNetworkd = true;
networking.useDHCP = false;
networking.firewall.enable = false;
networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ];
systemd.network.networks."10-lan" = {
matchConfig.Name = "eth1";
linkConfig.MACAddress = "${clientMac}";
networkConfig.DHCP = "ipv4";
};
};
};
testScript = ''
start_all()
upstream.wait_for_unit("dnsmasq.service")
router.wait_for_unit("network-online.target")
router.wait_until_succeeds("ip -4 addr show dev eth1 | grep -q 203.0.113.100")
router.wait_for_unit("kea-dhcp4-server.service")
router.wait_for_unit("unbound.service")
def q(name, rr="A", server="192.168.10.1"):
return client.succeed(f"dig +short +time=2 +tries=1 @{server} {name} {rr}").strip()
with subtest("the reservation gives the client its fixed address"):
client.wait_until_succeeds("ip -4 addr show dev eth1 | grep -q 192.168.10.51")
assert "${clientMac}" in client.succeed("ip link show eth1")
# Not from the pool.
client.fail("ip -4 addr show dev eth1 | grep -q 192.168.10.1[0-9][0-9]")
with subtest("hosts and reservations are published in the local domain"):
assert q("nas.lan.test") == "192.168.10.20", q("nas.lan.test")
assert q("nas.lan.test", "AAAA") == "fd00::20"
assert q("printer.lan.test") == "192.168.10.51"
assert q("20.10.168.192.in-addr.arpa", "PTR") == "nas.lan.test."
with subtest("the local domain is authoritative: unknown names are not forwarded"):
# Positive control: resolving through the router works at all.
assert q("other.example.test") == "203.0.113.8"
out = client.succeed("dig +time=2 +tries=1 @192.168.10.1 nothere.lan.test")
assert "status: NXDOMAIN" in out, out
with subtest("split horizon: only the named host is answered locally"):
assert q("photos.example.test") == "192.168.10.50", q("photos.example.test")
assert q("other.example.test") == "203.0.113.8"
with subtest("overrides cover the name and everything below it"):
assert q("k8s.example.test") == "192.168.10.60"
assert q("deep.sub.k8s.example.test") == "192.168.10.60"
with subtest("forward zones go to their own resolver"):
assert q("host.branch.test") == "10.9.9.9", q("host.branch.test")
'';
}
+132
View File
@@ -0,0 +1,132 @@
# router-install: an installer VM writes a router system to an empty disk
# (/dev/vda), then a second VM boots that disk with UEFI and the installed
# router must come up, with its secrets in place.
#
# Follows nixos/tests/installer.nix: both machines share one disk image; the
# installer runs from a small second disk.
{ self, pkgs }:
let
inherit (pkgs) lib;
common = import ./common.nix;
# The system to install: a minimal router on the layout the installer creates.
target = import "${pkgs.path}/nixos/lib/eval-config.nix" {
system = "x86_64-linux";
modules = [
self.nixosModules.router
self.nixosModules.installed-layout
# Only so the test driver can talk to the installed system.
"${pkgs.path}/nixos/modules/testing/test-instrumentation.nix"
{
system.stateVersion = "25.05";
users.users.root.openssh.authorizedKeys.keys = [
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFakeKeyForTestsOnlyXXXXXXXXXXXXXXXXXXXXXXXXXX test"
];
router = {
enable = true;
wan.interface = "eth1";
lans.main = {
interface = "eth2";
internet = true;
ipv4.address = "192.168.10.1";
dhcp.pool = {
start = "192.168.10.100";
end = "192.168.10.199";
};
};
};
}
];
};
toplevel = target.config.system.build.toplevel;
# Both machines must see the same disk.
sharedDisk = {
virtualisation.diskImage = "./target.qcow2";
virtualisation.diskSize = 6144;
virtualisation.memorySize = 2048;
};
in
pkgs.testers.runNixOSTest {
name = "router-installer";
nodes = {
installer =
{ lib, ... }:
{
imports = [
sharedDisk
self.nixosModules.installer
common.tools
];
router.installer.system = toplevel;
# Run from a small second disk so /dev/vda is the target.
virtualisation.fileSystems."/".autoFormat = true;
virtualisation.emptyDiskImages = [ 1024 ];
virtualisation.rootDevice = "/dev/vdb";
# No network in the sandbox; everything needed is in the closure.
nix.settings.substituters = lib.mkForce [ ];
};
target = {
imports = [ sharedDisk ];
virtualisation.vlans = [
1
2
];
virtualisation.useBootLoader = true;
virtualisation.useEFIBoot = true;
virtualisation.useDefaultFilesystems = false;
virtualisation.efi.keepVariables = false;
virtualisation.fileSystems."/" = {
device = "/dev/disk/by-label/this-is-not-real-and-will-never-be-used";
fsType = "ext4";
};
};
};
testScript = ''
installer.start()
installer.wait_for_unit("multi-user.target")
with subtest("router-install refuses bad input"):
installer.fail("router-install")
installer.fail("router-install --yes /etc/hostname")
# The installer's own root disk is mounted.
out = installer.fail("router-install --yes /dev/vdb 2>&1")
assert "mounted" in out, out
installer.fail("router-install --yes --secrets /nonexistent /dev/vda")
# Nothing was touched.
installer.fail("blkid /dev/vda1")
with subtest("installation onto the empty disk"):
installer.succeed("mkdir -p /root/secrets && echo -n hunter2 > /root/secrets/pppoe-password")
installer.succeed("router-install --yes --secrets /root/secrets /dev/vda >&2")
# Layout contract with the installed-layout module.
assert "boot" in installer.succeed("blkid -s LABEL -o value /dev/vda1")
assert "nixos" in installer.succeed("blkid -s LABEL -o value /dev/vda2")
installer.shutdown()
# The same machine, booting from the installed disk this time.
target.state_dir = installer.state_dir
target.start()
with subtest("the installed system boots and runs the router"):
target.wait_for_unit("multi-user.target")
assert target.succeed("readlink -f /run/current-system").strip() == "${toplevel}"
target.succeed("findmnt /boot | grep -q vfat")
target.succeed("test -e /boot/EFI/BOOT/BOOTX64.EFI")
target.wait_for_unit("nftables.service")
target.wait_for_unit("unbound.service")
target.wait_for_unit("kea-dhcp4-server.service")
target.wait_until_succeeds("ip -4 addr show dev eth2 | grep -q 192.168.10.1")
target.succeed("command -v router-arm router-confirm")
with subtest("secrets arrive with strict permissions"):
assert target.succeed("cat /var/lib/router/pppoe-password") == "hunter2"
assert target.succeed("stat -c %a /var/lib/router").strip() == "700"
assert target.succeed("stat -c %a /var/lib/router/pppoe-password").strip() == "600"
assert target.succeed("stat -c %U /var/lib/router/pppoe-password").strip() == "root"
'';
}
+6
View File
@@ -223,6 +223,9 @@ pkgs.testers.runNixOSTest {
privateKeyFile = "/etc/wg/s2s.key"; privateKeyFile = "/etc/wg/s2s.key";
listenPort = 51821; listenPort = 51821;
zone = "remote"; zone = "remote";
# No address on the tunnel: the site queries the LAN address and
# is allowed because it routes 192.168.50.0/24 through the tunnel.
dns = true;
peers.siteb = { peers.siteb = {
publicKey = key.siteb.pub; publicKey = key.siteb.pub;
endpoint = "203.0.113.60:51821"; endpoint = "203.0.113.60:51821";
@@ -383,6 +386,9 @@ pkgs.testers.runNixOSTest {
siteb.succeed("ping -c 1 -W 2 -I 192.168.50.1 192.168.10.1") siteb.succeed("ping -c 1 -W 2 -I 192.168.50.1 192.168.10.1")
siteb.fail("ping -c 1 -W 2 -I 192.168.50.1 192.168.10.100") siteb.fail("ping -c 1 -W 2 -I 192.168.50.1 192.168.10.100")
with subtest("site-to-site: the remote site can use the router's resolver"):
siteb.succeed("dig +short +time=2 +tries=1 -b 192.168.50.1 @192.168.10.1 example.test | grep -qx 203.0.113.1")
with subtest("provider tunnel: only the listed destinations, masqueraded"): with subtest("provider tunnel: only the listed destinations, masqueraded"):
out = main.succeed("curl -sf --max-time 5 http://198.18.0.1/") out = main.succeed("curl -sf --max-time 5 http://198.18.0.1/")
assert out.strip() == "10.200.0.2", f"expected masquerade to 10.200.0.2, got {out!r}" assert out.strip() == "10.200.0.2", f"expected masquerade to 10.200.0.2, got {out!r}"