From 603836725270ea26383ac8da4a141d4a731d5f1d Mon Sep 17 00:00:00 2001 From: Nils Stinnesbeck Date: Tue, 27 Oct 2026 16:29:52 +0100 Subject: [PATCH] Add IPv6 prefix delegation and PPPoE uplink IPv6: router.wan.ipv6.enable mirrors the existing uplink config (DHCPv6 with WithoutRA=solicit, delegated prefix). The ISP delegates a single /64, so exactly one LAN may claim it via router.lans..ipv6.pd (asserted); other LANs stay IPv4-only. SLAAC only, no NAT66. PPPoE: router.wan.ipv4.type = "pppoe" runs pppd on an optional VLAN of the uplink, with the password read from a file at service start (never the Nix store). NAT, forwarding and MSS clamping (MTU 1492) use the derived PPP device; native IPv6 stays on the untagged link. Tests: basic.nix is parameterised by uplink type; the pppoe variant uses a fake ISP with pppoe-server (VLAN 7, CHAP). Both variants cover IPv6 PD, SLAAC, routing without NAT and WAN-deny. Assertion tests gain PD/PPPoE cases and a full-system evaluation step. Co-Authored-By: Claude Sonnet 5.5 --- PLAN.md | 22 ++- modules/router/assertions.nix | 16 ++ modules/router/default.nix | 1 + modules/router/dhcp.nix | 10 +- modules/router/dns.nix | 6 +- modules/router/firewall.nix | 24 ++- modules/router/lib.nix | 4 +- modules/router/networking.nix | 82 ++++++++- modules/router/options.nix | 76 +++++++- modules/router/pppoe.nix | 86 +++++++++ tests/assertions.nix | 75 +++++++- tests/basic.nix | 337 +++++++++++++++++++++++++++++----- tests/default.nix | 4 + 13 files changed, 671 insertions(+), 72 deletions(-) create mode 100644 modules/router/pppoe.nix diff --git a/PLAN.md b/PLAN.md index 3a6c95e..2a59da7 100644 --- a/PLAN.md +++ b/PLAN.md @@ -62,6 +62,10 @@ examples/ # sample configs ## Status +**PPPoE done**: `pppoe` VM test = the same topology with a fake ISP running `pppoe-server` (VLAN 7, CHAP, password containing a quote) — covers session, NAT, MTU, MSS clamp (mutation-tested) and WAN-deny. `basic` remains the DHCPv4 variant. + +**IPv6 + prefix delegation done (single /64)**: `router.wan.ipv6.enable` and `router.lans..ipv6.pd` (at most one LAN, asserted). The VM test now covers the router's DHCPv6 address and delegated /64, SLAAC on the client, routing without NAT66, and WAN-deny over IPv6 (mutation-tested). Still open: RDNSS, other LANs are IPv4-only, DHCPv6 reservations. + **Phase 1 (MVP) skeleton done**: flake, `router.*` schema (wan, lans, dns.forwarders), networkd, nftables (zone-less for now: WAN/LAN), Kea DHCPv4, Unbound, assertions. `nix build .#checks.x86_64-linux.basic` runs a 3-node VM test (upstream/router/client) covering WAN DHCP, LAN DHCP+DNS, NAT, WAN-deny and LAN SSH. Interactive lab: `nix run .#lab` (x86_64) opens tmux with one console pane per VM plus the test-driver REPL (`router.shell_interact()`, `client.succeed(...)`, `test_script()`). The REPL pane also shows all VM output interleaved. @@ -71,7 +75,23 @@ Learned along the way: - In NixOS tests `fail(...)` also passes on "command not found": install probe tools explicitly and add a positive control per negative check. - Unbound answers `.test` locally (RFC 6761) and validates DNSSEC; tests using fake domains need both `local-zone ... transparent` and `domain-insecure` (both verified necessary; set in the test, not the module). - `modules/router/lib.nix` has IPv4 helpers (parse/format/netmask/cidr/overlap). `ipv4.subnet` is now derived from `address` + `prefixLength` (read-only). Assertions also cover pool bounds, pool containing the router address, and overlapping LANs. IPv6 helpers (prefix arithmetic for PD subnet-IDs) are still to do. -- Checks: `basic` (VM test), `lib` (helper unit tests), `assertions` (broken configs must fail with exact messages). Not yet covered: IPv6, VLANs, WireGuard, formal zones. +- Checks: `basic` and `pppoe` (VM tests), `lib` (helper unit tests), `assertions` (broken configs must fail with exact messages). Not yet covered: VLANs, WireGuard, formal zones. + +## IPv6 design (confirmed) + +- **WAN**: mirrors the existing router's uplink config — networkd `DHCP` (v6), `IPv6AcceptRA = true`, `KeepConfiguration = yes`, `dhcpV6Config = { WithoutRA = "solicit"; UseDelegatedPrefix = true; }`. The ISP sends RAs (default route) without DHCPv6 flags, so the router solicits on its own. +- **Delegation**: the ISP delegates a single **/64**, which cannot be split. Exactly one LAN may claim it via an explicit flag (`router.lans..ipv6.pd = true`); an assertion rejects more than one. +- **LANs**: SLAAC only (RA, no DHCPv6). All other LANs/VLANs are **IPv4-only** for now (ULA or NPTv6 can come later). +- **Firewall**: inet table covers v6; WAN→LAN forwarding stays denied, no NAT66. +- **Test**: fake ISP on `upstream` = Kea DHCPv6 (IA_NA + IA_PD /64 from a /48 pool) + networkd RA sender (default route only) + a static route for the delegated prefix back to the router. +- **Known gap**: no RDNSS in RAs yet (the router's global LAN address is not known at eval time); clients still get DNS via DHCPv4. + +## PPPoE uplink (confirmed) + +- **IPv4 via PPPoE** (`router.wan.ipv4.type = "pppoe"`), carried on an optional tagged **VLAN** on the WAN link (`pppoe.vlan`); native DHCPv6/PD stays on the untagged link. +- `wan.interface` is the physical link; the derived `wan.ipv4.interface` (`pppoe-wan` or the link) is where NAT, MSS clamping and the IPv4 forwarding rule apply. IPv6 forwarding keeps using the physical link. +- pppd (`pppoe.so`) runs as `router-pppoe.service`, ordered after the VLAN device. Credentials: `username` inline, password from `pppoe.passwordFile` (type forbids Nix-store paths); the options file is generated into a root-only runtime dir. **Deployment must provide that file** on persistent storage (or via agenix/sops). +- **MTU 1492 + TCP MSS clamp** (1452) in both directions on the PPP device; RFC 4638 not implemented. ## Open items diff --git a/modules/router/assertions.nix b/modules/router/assertions.nix index 0223d39..cd8d15d 100644 --- a/modules/router/assertions.nix +++ b/modules/router/assertions.nix @@ -79,6 +79,22 @@ in assertion = !(lib.elem cfg.wan.interface (map (l: l.interface) lans)); message = "router: the WAN interface `${cfg.wan.interface}` is also used by a LAN."; } + { + assertion = cfg.wan.ipv4.type != "pppoe" || cfg.wan.ipv4.pppoe.username != ""; + message = "router: router.wan.ipv4.pppoe.username must be set for a PPPoE uplink."; + } + { + assertion = cfg.wan.ipv4.type != "pppoe" || cfg.wan.ipv4.pppoe.passwordFile != null; + message = "router: router.wan.ipv4.pppoe.passwordFile must be set for a PPPoE uplink."; + } + { + assertion = lib.length (lib.filter (l: l.ipv6.pd) lans) <= 1; + message = "router: more than one LAN sets ipv6.pd, but the delegated /64 cannot be split."; + } + { + assertion = !(lib.any (l: l.ipv6.pd) lans) || cfg.wan.ipv6.enable; + message = "router: a LAN sets ipv6.pd but router.wan.ipv6.enable is false."; + } { assertion = let diff --git a/modules/router/default.nix b/modules/router/default.nix index ad1d190..083886e 100644 --- a/modules/router/default.nix +++ b/modules/router/default.nix @@ -3,6 +3,7 @@ ./options.nix ./assertions.nix ./networking.nix + ./pppoe.nix ./firewall.nix ./dhcp.nix ./dns.nix diff --git a/modules/router/dhcp.nix b/modules/router/dhcp.nix index 27075ab..93771c5 100644 --- a/modules/router/dhcp.nix +++ b/modules/router/dhcp.nix @@ -25,8 +25,14 @@ in interface = l.interface; pools = [ { pool = "${l.dhcp.pool.start} - ${l.dhcp.pool.end}"; } ]; option-data = [ - { name = "routers"; data = l.ipv4.address; } - { name = "domain-name-servers"; data = l.ipv4.address; } + { + name = "routers"; + data = l.ipv4.address; + } + { + name = "domain-name-servers"; + data = l.ipv4.address; + } ]; }) dhcpLans; }; diff --git a/modules/router/dns.nix b/modules/router/dns.nix index 9c8b5be..2f68607 100644 --- a/modules/router/dns.nix +++ b/modules/router/dns.nix @@ -14,9 +14,11 @@ in ip-freebind = true; access-control = [ "127.0.0.0/8 allow" - ] ++ map (l: "${l.ipv4.subnet} allow") lans; + ] + ++ map (l: "${l.ipv4.subnet} allow") lans; }; - } // lib.optionalAttrs (cfg.dns.forwarders != [ ]) { + } + // lib.optionalAttrs (cfg.dns.forwarders != [ ]) { forward-zone = [ { name = "."; diff --git a/modules/router/firewall.nix b/modules/router/firewall.nix index c8fa601..7019c35 100644 --- a/modules/router/firewall.nix +++ b/modules/router/firewall.nix @@ -6,6 +6,14 @@ let set = ifaces: "{ ${lib.concatMapStringsSep ", " (i: ''"${i}"'') ifaces} }"; lanIfaces = map (l: l.interface) lans; mgmtIfaces = map (l: l.interface) (lib.filter (l: l.allowManagement) lans); + uplink4 = cfg.wan.ipv4.interface; + pppoe = cfg.wan.ipv4.type == "pppoe"; + # IPv4 leaves via the PPP device, IPv6 via the physical uplink. + uplinks = lib.unique [ + cfg.wan.interface + uplink4 + ]; + mss = cfg.wan.ipv4.pppoe.mtu - 40; dhcpIfaces = map (l: l.interface) (lib.filter (l: l.dhcp.enable) lans); in { @@ -34,12 +42,13 @@ in iifname ${set lanIfaces} icmp type echo-request accept iifname ${set lanIfaces} icmpv6 type echo-request accept - iifname "${cfg.wan.interface}" udp dport 68 accept + ${lib.optionalString (!pppoe) ''iifname "${cfg.wan.interface}" udp dport 68 accept''} + ${lib.optionalString cfg.wan.ipv6.enable ''iifname "${cfg.wan.interface}" udp dport 546 accept''} iifname ${set lanIfaces} udp dport 53 accept iifname ${set lanIfaces} tcp dport 53 accept - ${lib.optionalString (dhcpIfaces != [ ]) ''iifname ${set dhcpIfaces} udp dport 67 accept''} - ${lib.optionalString (mgmtIfaces != [ ]) ''iifname ${set mgmtIfaces} tcp dport 22 accept''} + ${lib.optionalString (dhcpIfaces != [ ]) "iifname ${set dhcpIfaces} udp dport 67 accept"} + ${lib.optionalString (mgmtIfaces != [ ]) "iifname ${set mgmtIfaces} tcp dport 22 accept"} } chain forward { @@ -50,7 +59,12 @@ in icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem } accept - iifname ${set lanIfaces} oifname "${cfg.wan.interface}" accept + ${lib.optionalString pppoe '' + # Clamp MSS to the PPP MTU in both directions (never raised). + iifname "${uplink4}" tcp flags syn tcp option maxseg size > ${toString mss} tcp option maxseg size set ${toString mss} + oifname "${uplink4}" tcp flags syn tcp option maxseg size > ${toString mss} tcp option maxseg size set ${toString mss} + ''} + iifname ${set lanIfaces} oifname ${set uplinks} accept } ''; }; @@ -60,7 +74,7 @@ in content = '' chain postrouting { type nat hook postrouting priority srcnat; policy accept; - oifname "${cfg.wan.interface}" masquerade + oifname "${uplink4}" masquerade } ''; }; diff --git a/modules/router/lib.nix b/modules/router/lib.nix index 20bdbf5..22e20ad 100644 --- a/modules/router/lib.nix +++ b/modules/router/lib.nix @@ -33,7 +33,9 @@ rec { networkOf = address: prefixLength: builtins.bitAnd address (netMask prefixLength); # "192.168.10.1" /24 -> "192.168.10.0/24" - cidr = address: prefixLength: "${format (networkOf (parse address) prefixLength)}/${toString prefixLength}"; + cidr = + address: prefixLength: + "${format (networkOf (parse address) prefixLength)}/${toString prefixLength}"; inSubnet = { address, prefixLength }: ip: networkOf ip prefixLength == networkOf address prefixLength; diff --git a/modules/router/networking.nix b/modules/router/networking.nix index fc7d9d9..9ea2220 100644 --- a/modules/router/networking.nix +++ b/modules/router/networking.nix @@ -1,6 +1,11 @@ { config, lib, ... }: let cfg = config.router; + v6 = cfg.wan.ipv6.enable; + pppoe = cfg.wan.ipv4.type == "pppoe"; + vlan = cfg.wan.ipv4.pppoe.vlan; + vlanDevice = "${cfg.wan.interface}.${toString vlan}"; + dhcp = if pppoe then (if v6 then "ipv6" else "no") else (if v6 then "yes" else "ipv4"); in { config = lib.mkIf cfg.enable { @@ -9,22 +14,83 @@ in boot.kernel.sysctl = { "net.ipv4.conf.all.forwarding" = 1; + "net.ipv6.conf.all.forwarding" = 1; + }; + + systemd.network.netdevs = lib.optionalAttrs (pppoe && vlan != null) { + "05-wan-pppoe" = { + netdevConfig = { + Kind = "vlan"; + Name = vlanDevice; + }; + vlanConfig.Id = vlan; + }; }; systemd.network.networks = { "10-wan" = { matchConfig.Name = cfg.wan.interface; - networkConfig.DHCP = "ipv4"; + networkConfig = { + DHCP = dhcp; + } + // lib.optionalAttrs v6 { + IPv6AcceptRA = true; + KeepConfiguration = "yes"; + } + // lib.optionalAttrs (pppoe && vlan != null) { + VLAN = [ vlanDevice ]; + }; + } + // lib.optionalAttrs (!pppoe) { # The router runs its own resolver; never adopt the ISP's DNS for the host. dhcpV4Config.UseDNS = false; - }; - } // lib.mapAttrs' ( - name: lan: - lib.nameValuePair "20-lan-${name}" { - matchConfig.Name = lan.interface; - address = [ "${lan.ipv4.address}/${toString lan.ipv4.prefixLength}" ]; - networkConfig.ConfigureWithoutCarrier = true; } + // lib.optionalAttrs v6 { + # The ISP sends RAs (default route) without DHCPv6 flags, so solicit anyway. + dhcpV6Config = { + WithoutRA = "solicit"; + UseDelegatedPrefix = true; + UseDNS = false; + }; + }; + } + // lib.optionalAttrs (pppoe && vlan != null) { + # pppd configures the session; networkd only has to bring the device up. + "11-wan-pppoe" = { + matchConfig.Name = vlanDevice; + networkConfig = { + LinkLocalAddressing = "no"; + IPv6AcceptRA = false; + }; + }; + } + // lib.mapAttrs' ( + name: lan: + lib.nameValuePair "20-lan-${name}" ( + { + matchConfig.Name = lan.interface; + address = [ "${lan.ipv4.address}/${toString lan.ipv4.prefixLength}" ]; + networkConfig = { + ConfigureWithoutCarrier = true; + # Never accept RAs from the inside. + IPv6AcceptRA = false; + } + // lib.optionalAttrs lan.ipv6.pd { + DHCPPrefixDelegation = true; + IPv6SendRA = true; + }; + } + // lib.optionalAttrs lan.ipv6.pd { + dhcpPrefixDelegationConfig = { + UplinkInterface = cfg.wan.interface; + SubnetId = 0; + Announce = true; + Assign = true; + }; + # No RDNSS yet: the global address is unknown at evaluation time. + ipv6SendRAConfig.EmitDNS = false; + } + ) ) cfg.lans; }; } diff --git a/modules/router/options.nix b/modules/router/options.nix index a5b799d..a6adc14 100644 --- a/modules/router/options.nix +++ b/modules/router/options.nix @@ -1,4 +1,4 @@ -{ lib, ... }: +{ config, lib, ... }: let inherit (lib) mkOption mkEnableOption types; rlib = import ./lib.nix { inherit lib; }; @@ -11,8 +11,66 @@ in interface = mkOption { type = types.str; example = "eth0"; - description = "Name of the uplink interface."; + description = '' + Physical uplink interface. Native IPv6 (DHCPv6) runs on it untagged; + with PPPoE it also carries the (optionally tagged) PPPoE session. + ''; }; + + ipv4 = { + type = mkOption { + type = types.enum [ + "dhcp" + "pppoe" + ]; + default = "dhcp"; + description = "How the router gets its IPv4 uplink."; + }; + + pppoe = { + vlan = mkOption { + type = types.nullOr (types.ints.between 1 4094); + default = null; + description = "VLAN ID the ISP requires for PPPoE, or null for untagged."; + }; + username = mkOption { + type = types.str; + default = ""; + description = "PPP authentication username."; + }; + passwordFile = mkOption { + type = types.nullOr ( + types.pathWith { + absolute = true; + inStore = false; + } + ); + default = null; + example = "/var/lib/secrets/pppoe-password"; + description = '' + File containing the PPP password, read at service start. Must not + be in the Nix store (it would be world-readable). + ''; + }; + mtu = mkOption { + type = types.ints.between 576 1500; + default = 1492; + description = "PPP MTU/MRU; TCP MSS is clamped to match on forwarded traffic."; + }; + }; + + interface = mkOption { + type = types.str; + readOnly = true; + internal = true; + description = "Device IPv4 traffic leaves through (derived)."; + }; + }; + + ipv6.enable = mkEnableOption '' + IPv6 on the uplink: DHCPv6 (soliciting even without RA flags) plus + prefix delegation. The ISP is expected to delegate a single /64 + ''; }; lans = mkOption { @@ -63,6 +121,17 @@ in }; }; + ipv6.pd = mkOption { + type = types.bool; + default = false; + description = '' + Whether this LAN receives the /64 delegated on the uplink + (assigned to the interface and announced via SLAAC). A /64 + cannot be split, so at most one LAN may set this; the others + stay IPv4-only. + ''; + }; + allowManagement = mkOption { type = types.bool; default = true; @@ -87,4 +156,7 @@ in description = "Upstream resolvers. When empty, Unbound resolves recursively."; }; }; + + config.router.wan.ipv4.interface = + if config.router.wan.ipv4.type == "pppoe" then "pppoe-wan" else config.router.wan.interface; } diff --git a/modules/router/pppoe.nix b/modules/router/pppoe.nix new file mode 100644 index 0000000..c572792 --- /dev/null +++ b/modules/router/pppoe.nix @@ -0,0 +1,86 @@ +# PPPoE uplink for IPv4. pppd owns the session (address, default route); the +# VLAN device it runs on is created in networking.nix. +{ + config, + lib, + pkgs, + ... +}: +let + cfg = config.router; + wan = cfg.wan; + pppoe = wan.ipv4.pppoe; + + # Device carrying the PPPoE session: the VLAN on the uplink, or the uplink itself. + device = if pppoe.vlan == null then wan.interface else "${wan.interface}.${toString pppoe.vlan}"; + + # Unit that appears once udev has seen the device (VLAN devices are created + # by networkd after boot has begun, so pppd must not start before it). + deviceUnit = "sys-subsystem-net-devices-${lib.replaceStrings [ "-" ] [ "\\x2d" ] device}.device"; + + staticOptions = pkgs.writeText "router-pppoe-options" '' + plugin pppoe.so + nic-${device} + ifname ${wan.ipv4.interface} + user "${pppoe.username}" + noauth + hide-password + defaultroute + replacedefaultroute + noipdefault + nodefaultroute6 + noccp + mtu ${toString pppoe.mtu} + mru ${toString pppoe.mtu} + lcp-echo-interval 10 + lcp-echo-failure 3 + nodetach + persist + maxfail 0 + holdoff 5 + ''; + + prepare = pkgs.writeShellScript "router-pppoe-prepare" '' + set -eu + umask 077 + password=$(cat ${lib.escapeShellArg (toString pppoe.passwordFile)}) + # pppd option files use C-style escapes inside double quotes. + password=''${password//\\/\\\\} + password=''${password//\"/\\\"} + { + cat ${staticOptions} + printf 'password "%s"\n' "$password" + } > /run/router-pppoe/options + ''; +in +{ + config = lib.mkIf (cfg.enable && wan.ipv4.type == "pppoe") { + boot.kernelModules = [ + "pppoe" + "pppox" + "ppp_generic" + ]; + + # pppd refuses to start without a (possibly empty) system options file. + environment.etc."ppp/options".text = ""; + + systemd.services.router-pppoe = { + description = "PPPoE uplink (${device})"; + wantedBy = [ "multi-user.target" ]; + wants = [ deviceUnit ]; + after = [ + "network.target" + "systemd-networkd.service" + deviceUnit + ]; + serviceConfig = { + RuntimeDirectory = "router-pppoe"; + RuntimeDirectoryMode = "0700"; + ExecStartPre = prepare; + ExecStart = "${pkgs.ppp}/bin/pppd file /run/router-pppoe/options"; + Restart = "always"; + RestartSec = 5; + }; + }; + }; +} diff --git a/tests/assertions.nix b/tests/assertions.nix index 6a24f2e..772a530 100644 --- a/tests/assertions.nix +++ b/tests/assertions.nix @@ -17,10 +17,10 @@ let fsType = "ext4"; }; system.stateVersion = "25.05"; - router = { + router = lib.recursiveUpdate { enable = true; wan.interface = "eth0"; - } // router; + } router; } ]; }).config; @@ -73,6 +73,59 @@ let }; expect = [ "`a` (192.168.10.0/24) and `b` (192.168.10.0/24) overlap" ]; }; + pdOnOneLan = { + router = { + wan.ipv6.enable = true; + lans.main = lan { ipv6.pd = true; }; + }; + expect = [ ]; + }; + pdOnTwoLans = { + router = { + wan.ipv6.enable = true; + lans = { + a = lan { ipv6.pd = true; }; + b = lan { + interface = "eth2"; + ipv4.address = "192.168.20.1"; + dhcp.pool = { start = "192.168.20.100"; end = "192.168.20.199"; }; + ipv6.pd = true; + }; + }; + }; + expect = [ "delegated /64 cannot be split" ]; + }; + pdWithoutWanIpv6 = { + router.lans.main = lan { ipv6.pd = true; }; + expect = [ "router.wan.ipv6.enable is false" ]; + }; + pppoeMissingCredentials = { + router = { + wan.ipv4.type = "pppoe"; + lans.main = lan { }; + }; + expect = [ + "pppoe.username must be set" + "pppoe.passwordFile must be set" + ]; + }; + pppoeValid = { + router = { + wan = { + ipv4 = { + type = "pppoe"; + pppoe = { + vlan = 7; + username = "user@isp"; + passwordFile = "/var/lib/secrets/pppoe"; + }; + }; + ipv6.enable = true; + }; + lans.main = lan { ipv6.pd = true; }; + }; + expect = [ ]; + }; nonOverlappingLans = { router.lans = { a = lan { }; @@ -94,15 +147,29 @@ let ) cases ); + # Assertions alone do not force every option (e.g. networkd unit checks), so + # fully evaluate the system for the configs that are meant to be valid. + fullEval = + name: + let + r = builtins.tryEval (builtins.seq (evalRouter cases.${name}.router).system.build.toplevel.drvPath true); + in + lib.optional (!r.success) "${name}: full system evaluation failed"; + fullEvalProblems = lib.concatMap fullEval [ + "valid" + "pdOnOneLan" + "pppoeValid" + ]; + subnet = (evalRouter { lans.main = lan { ipv4.prefixLength = 20; ipv4.address = "172.16.37.1"; dhcp.enable = false; }; }).router.lans.main.ipv4.subnet; subnetProblem = lib.optional (subnet != "172.16.32.0/20") "derived subnet: expected 172.16.32.0/20, got ${subnet}"; in pkgs.runCommand "router-assertion-tests" { } ( - if problems ++ subnetProblem == [ ] then + if problems ++ subnetProblem ++ fullEvalProblems == [ ] then "touch $out" else '' - printf '%s\n' ${lib.escapeShellArgs (problems ++ subnetProblem)} >&2 + printf '%s\n' ${lib.escapeShellArgs (problems ++ subnetProblem ++ fullEvalProblems)} >&2 exit 1 '' ) diff --git a/tests/basic.nix b/tests/basic.nix index d2bdedf..9e9b248 100644 --- a/tests/basic.nix +++ b/tests/basic.nix @@ -1,57 +1,255 @@ -# Single WAN (DHCP) + single LAN: DHCP, DNS, NAT and the default firewall policy. +# Single WAN + single LAN: DHCP, DNS, NAT, SLAAC and the default firewall +# policy for both address families. `wan4` selects how the IPv4 uplink works: # -# upstream (ISP, 203.0.113.1) -- vlan 1 -- router -- vlan 2 -- client -{ self, pkgs }: +# dhcp - DHCPv4 on the uplink (ISP 203.0.113.1) +# pppoe - PPPoE over VLAN 7 on the uplink (ISP 198.51.100.1, PPP MTU 1492) +# +# IPv6 is native and untagged on the uplink in both variants. +# +# upstream (ISP) -- vlan 1 -- router -- vlan 2 -- client +# +# The fake ISP mimics the real uplink: RAs carry only a default route (no +# prefix to autoconfigure, no DHCPv6 flags), and Kea hands out one address +# plus a single delegated /64 out of 2001:db8:2::/48. +{ + self, + pkgs, + wan4 ? "dhcp", +}: let + pppoe = wan4 == "pppoe"; + ispV4 = if pppoe then "198.51.100.1" else "203.0.113.1"; + routerV4 = if pppoe then "198.51.100.100" else "203.0.113.100"; + pppoeVlan = 7; + + inherit (pkgs) lib; + # Indent every line but the first (the interpolation site supplies that one). + indent = + n: text: + lib.concatStringsSep "\n${lib.fixedWidthString n " " ""}" ( + lib.splitString "\n" (lib.removeSuffix "\n" text) + ); + + uplinkWait = + if pppoe then + '' + upstream.wait_for_unit("pppoe-server.service") + router.wait_for_unit("router-pppoe.service") + router.wait_until_succeeds("ip -4 addr show dev pppoe-wan | grep -q ${routerV4}") + router.wait_until_succeeds("ip -4 route show default | grep -q 'dev pppoe-wan'") + # The ISP's route to the LAN needs the session to exist first. + upstream.wait_until_succeeds("ip -4 addr show | grep -q ${ispV4}") + upstream.succeed("ip route replace 192.168.10.0/24 via ${routerV4}") + '' + else + '' + router.wait_until_succeeds("ip -4 addr show dev eth1 | grep -q ${routerV4}") + ''; + + mssSubtest = '' + with subtest("PPP link has MTU 1492 and forwarded TCP MSS is clamped"): + assert "mtu 1492" in router.succeed("ip link show pppoe-wan"), "unexpected PPP MTU" + # Capture the SYN as it leaves the router; the client itself offers MSS 1460. + router.succeed("tcpdump -nn -i pppoe-wan -c 1 -w /tmp/syn.pcap 'tcp[tcpflags] & (tcp-syn|tcp-ack) = tcp-syn' >/dev/null 2>&1 &") + router.wait_until_succeeds("pgrep tcpdump") + router.sleep(1) + client.succeed("curl -sf --max-time 5 --resolve example.test:80:${ispV4} http://example.test/") + mss = router.wait_until_succeeds("tcpdump -nn -v -r /tmp/syn.pcap 2>&1 | grep -o 'mss [0-9]*'") + assert mss.strip() == "mss 1452", f"expected the SYN clamped to mss 1452, got {mss!r}" + ''; + # `fail` in the test script also passes on "command not found", so make sure # the tools the script relies on are really there. tools = { pkgs, ... }: { - environment.systemPackages = with pkgs; [ dnsutils netcat-openbsd curl tcpdump ]; + environment.systemPackages = with pkgs; [ + dnsutils + netcat-openbsd + curl + tcpdump + ]; }; in pkgs.testers.runNixOSTest { - name = "router-basic"; + name = "router-${if pppoe then "pppoe" else "basic"}"; defaults.imports = [ tools ]; # `driverInteractive` uses the full QEMU, which would open a window per VM; # `nix run .#lab` wants the consoles in the terminal instead. - interactive.defaults.virtualisation.qemu.options = [ "-display" "none" ]; + interactive.defaults.virtualisation.qemu.options = [ + "-display" + "none" + ]; nodes = { - # Plays the ISP: hands out one WAN lease, answers DNS, and serves a page - # that echoes the client address it sees (to prove NAT happens). - upstream = { lib, ... }: { - virtualisation.vlans = [ 1 ]; - networking.useDHCP = false; - networking.firewall.enable = false; - networking.interfaces.eth1.ipv4 = { - addresses = lib.mkForce [ { address = "203.0.113.1"; prefixLength = 24; } ]; - # Route back to the LAN via the router so that "unreachable" below - # is due to the firewall and not a missing route. - routes = [ { address = "192.168.10.0"; prefixLength = 24; via = "203.0.113.100"; } ]; - }; + # Plays the ISP: hands out one WAN lease (v4 and v6) plus a delegated /64, + # answers DNS, and serves a page that echoes the client address it sees + # (to prove NAT happens for v4 and does not for v6). + upstream = + { lib, pkgs, ... }: + lib.mkMerge [ + { + virtualisation.vlans = [ 1 ]; + networking.useNetworkd = true; + networking.useDHCP = false; + networking.firewall.enable = false; + networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ]; - services.dnsmasq = { - enable = true; - settings = { - interface = "eth1"; - bind-interfaces = true; - dhcp-range = "203.0.113.100,203.0.113.100,255.255.255.0,1h"; - dhcp-option = [ "option:router,203.0.113.1" ]; - address = "/example.test/203.0.113.1"; - }; - }; + systemd.network.networks."10-wan" = { + matchConfig.Name = "eth1"; + address = [ "2001:db8:1::1/64" ] ++ lib.optional (!pppoe) "203.0.113.1/24"; + # Routes back to the LAN via the router so that "unreachable" below is + # due to the firewall and not a missing route. (With PPPoE the IPv4 + # route can only exist once the session is up; the test adds it.) + routes = [ + { + Destination = "2001:db8:2::/48"; + Gateway = "2001:db8:1::100"; + } + ] + ++ lib.optional (!pppoe) { + Destination = "192.168.10.0/24"; + Gateway = "203.0.113.100"; + }; + networkConfig = { + IPv6SendRA = true; + } + // lib.optionalAttrs pppoe { VLAN = [ "eth1.${toString pppoeVlan}" ]; }; + ipv6SendRAConfig = { + Managed = false; + OtherInformation = false; + RouterLifetimeSec = 1800; + EmitDNS = false; + }; + # On-link only: the router must not autoconfigure a second address. + ipv6Prefixes = [ + { + Prefix = "2001:db8:1::/64"; + OnLink = true; + AddressAutoconfiguration = false; + } + ]; + }; - services.nginx = { - enable = true; - virtualHosts."example.test".locations."/".return = ''200 "$remote_addr"''; - }; - }; + services.dnsmasq = { + enable = true; + settings = { + address = "/example.test/${ispV4}"; + } + // ( + if pppoe then + # The PPP interface only exists once a session is up. + { bind-dynamic = true; } + else + { + interface = "eth1"; + bind-interfaces = true; + dhcp-range = "203.0.113.100,203.0.113.100,255.255.255.0,1h"; + dhcp-option = [ "option:router,203.0.113.1" ]; + } + ); + }; + + services.kea.dhcp6 = { + enable = true; + settings = { + interfaces-config = { + interfaces = [ "eth1" ]; + re-detect = true; + }; + lease-database = { + type = "memfile"; + persist = false; + }; + preferred-lifetime = 3000; + valid-lifetime = 4000; + subnet6 = [ + { + id = 1; + subnet = "2001:db8:1::/64"; + interface = "eth1"; + pools = [ { pool = "2001:db8:1::100 - 2001:db8:1::100"; } ]; + pd-pools = [ + { + prefix = "2001:db8:2::"; + prefix-len = 48; + delegated-len = 64; + } + ]; + } + ]; + }; + }; + + services.nginx = { + enable = true; + virtualHosts."example.test".locations."/".return = ''200 "$remote_addr"''; + }; + } + (lib.mkIf pppoe { + boot.kernelModules = [ + "pppoe" + "pppox" + "ppp_generic" + ]; + systemd.network.netdevs."05-pppoe" = { + netdevConfig = { + Kind = "vlan"; + Name = "eth1.${toString pppoeVlan}"; + }; + vlanConfig.Id = pppoeVlan; + }; + systemd.network.networks."11-pppoe" = { + matchConfig.Name = "eth1.${toString pppoeVlan}"; + networkConfig.LinkLocalAddressing = "no"; + }; + + environment.etc = { + "ppp/options".text = ""; + "ppp/chap-secrets".text = '' + "testuser" * "p4ss\"w0rd" * + ''; + "ppp/pppoe-server-options".text = '' + require-chap + noccp + mtu 1492 + mru 1492 + lcp-echo-interval 10 + lcp-echo-failure 3 + ''; + }; + + # The ISP's access concentrator: one session, 198.51.100.1 <-> .100. + systemd.services.pppoe-server = { + wantedBy = [ "multi-user.target" ]; + wants = [ "sys-subsystem-net-devices-eth1.${toString pppoeVlan}.device" ]; + after = [ + "systemd-networkd.service" + "sys-subsystem-net-devices-eth1.${toString pppoeVlan}.device" + ]; + serviceConfig = { + ExecStart = lib.concatStringsSep " " [ + "${pkgs.rp-pppoe}/bin/pppoe-server -F -I eth1.${toString pppoeVlan}" + "-L 198.51.100.1 -R 198.51.100.100 -N 1" + "-O /etc/ppp/pppoe-server-options -q ${pkgs.ppp}/bin/pppd" + ]; + Restart = "always"; + }; + }; + }) + ]; router = { lib, ... }: { imports = [ self.nixosModules.router ]; - virtualisation.vlans = [ 1 2 ]; + virtualisation.vlans = [ + 1 + 2 + ]; + + environment.etc."pppoe-test-password" = { + text = "p4ss\"w0rd"; + mode = "0600"; + }; # The test driver preconfigures static addresses on every vlan interface. networking.interfaces.eth1.ipv4.addresses = lib.mkForce [ ]; @@ -59,13 +257,32 @@ pkgs.testers.runNixOSTest { router = { enable = true; - wan.interface = "eth1"; + wan = { + interface = "eth1"; + ipv6.enable = true; + ipv4 = { + type = wan4; + } + // lib.optionalAttrs pppoe { + pppoe = { + vlan = pppoeVlan; + username = "testuser"; + passwordFile = "/etc/pppoe-test-password"; + }; + }; + }; lans.main = { interface = "eth2"; - ipv4 = { address = "192.168.10.1"; }; - dhcp.pool = { start = "192.168.10.100"; end = "192.168.10.199"; }; + ipv4 = { + address = "192.168.10.1"; + }; + ipv6.pd = true; + dhcp.pool = { + start = "192.168.10.100"; + end = "192.168.10.199"; + }; }; - dns.forwarders = [ "203.0.113.1" ]; + dns.forwarders = [ ispV4 ]; }; # Unbound answers `.test` locally (RFC 6761) and would validate DNSSEC @@ -96,9 +313,9 @@ pkgs.testers.runNixOSTest { upstream.wait_for_unit("dnsmasq.service") upstream.wait_for_unit("nginx.service") - with subtest("router obtains its WAN lease and brings up services"): + with subtest("router obtains its IPv4 uplink and brings up services"): router.wait_for_unit("network-online.target") - router.wait_until_succeeds("ip -4 addr show dev eth1 | grep -q 203.0.113.100") + ${indent 4 uplinkWait} router.wait_for_unit("nftables.service") router.wait_for_unit("kea-dhcp4-server.service") router.wait_for_unit("unbound.service") @@ -109,22 +326,48 @@ pkgs.testers.runNixOSTest { client.succeed("resolvectl status eth1 | grep -q 192.168.10.1") with subtest("client resolves names through the router"): - client.wait_until_succeeds("dig +short +time=2 +tries=1 @192.168.10.1 example.test | grep -qx 203.0.113.1") + client.wait_until_succeeds("dig +short +time=2 +tries=1 @192.168.10.1 example.test | grep -qx ${ispV4}") with subtest("client reaches the 'internet' and is NATed"): - out = client.wait_until_succeeds("curl -sf --max-time 5 --resolve example.test:80:203.0.113.1 http://example.test/") - assert out.strip() == "203.0.113.100", f"expected NATed source 203.0.113.100, got {out!r}" + out = client.wait_until_succeeds("curl -sf --max-time 5 --resolve example.test:80:${ispV4} http://example.test/") + assert out.strip() == "${routerV4}", f"expected NATed source ${routerV4}, got {out!r}" + ${lib.optionalString pppoe (indent 0 mssSubtest)} with subtest("WAN cannot initiate to the router or the LAN"): # Positive control: the probe tool works against a port that is open. - upstream.succeed("nc -z -w 2 203.0.113.1 80") - upstream.fail("nc -z -w 2 203.0.113.100 22") - upstream.fail("nc -z -w 2 203.0.113.100 53") + upstream.succeed("nc -z -w 2 ${ispV4} 80") + upstream.fail("nc -z -w 2 ${routerV4} 22") + upstream.fail("nc -z -w 2 ${routerV4} 53") upstream.fail("ping -c 1 -W 2 192.168.10.1") - upstream.fail("ping -c 1 -W 2 203.0.113.100") + upstream.fail("ping -c 1 -W 2 ${routerV4}") client_ip = client.succeed("ip -4 -o addr show dev eth1 | awk '{print $4}' | cut -d/ -f1").strip() upstream.fail(f"ping -c 1 -W 2 {client_ip}") + with subtest("router gets an IPv6 address, the delegated /64 and a default route"): + router.wait_until_succeeds("ip -6 addr show dev eth1 scope global | grep -q 2001:db8:1::100") + router.wait_until_succeeds("ip -6 addr show dev eth2 scope global | grep -q 2001:db8:2:") + router.wait_until_succeeds("ip -6 route show default | grep -q 'via fe80:.* dev eth1'") + # The WAN prefix is not autoconfigured, so DHCPv6 is the only global address there. + assert router.succeed("ip -6 -o addr show dev eth1 scope global | wc -l").strip() == "1" + + with subtest("client autoconfigures a global address from the delegated prefix"): + client.wait_until_succeeds("ip -6 addr show dev eth1 scope global | grep -q 2001:db8:2:") + client.wait_until_succeeds("ip -6 route show default | grep -q 'dev eth1'") + client_v6 = client.succeed("ip -6 -o addr show dev eth1 scope global | awk '{print $4}' | cut -d/ -f1 | head -1").strip() + + with subtest("IPv6 is routed without NAT"): + client.wait_until_succeeds("ping -6 -c 1 -W 2 2001:db8:1::1") + out = client.succeed("curl -sf -g --max-time 5 -H 'Host: example.test' 'http://[2001:db8:1::1]/'") + assert out.strip() == client_v6, f"expected the client's own address {client_v6}, got {out!r}" + + with subtest("WAN cannot initiate IPv6 to the router or the LAN"): + # Positive control: the ISP really has a route to the client, so a + # failure below is down to the firewall. + upstream.succeed(f"ip -6 route get {client_v6} | grep -q 'via 2001:db8:1::100'") + upstream.fail(f"ping -6 -c 1 -W 2 {client_v6}") + upstream.fail("ping -6 -c 1 -W 2 2001:db8:1::100") + upstream.fail("nc -6 -z -w 2 2001:db8:1::100 22") + with subtest("LAN can reach SSH and ping on the router"): client.succeed("nc -z -w 2 192.168.10.1 22") client.succeed("ping -c 1 -W 2 192.168.10.1") diff --git a/tests/default.nix b/tests/default.nix index 722a724..77b7367 100644 --- a/tests/default.nix +++ b/tests/default.nix @@ -1,6 +1,10 @@ { self, pkgs }: { basic = import ./basic.nix { inherit self pkgs; }; + pppoe = import ./basic.nix { + inherit self pkgs; + wan4 = "pppoe"; + }; lib = import ./lib.nix { inherit pkgs; }; assertions = import ./assertions.nix { inherit self pkgs; }; }