Add MVP router flake with VM test and tmux lab
router.* schema (wan, lans, dns.forwarders) compiled to systemd-networkd, a generated nftables ruleset with NAT, Kea DHCPv4 and Unbound, plus build-time assertions against lock-out and interface conflicts. tests/basic.nix is a three-node VM test (upstream/router/client) covering WAN DHCP, LAN DHCP and DNS, NAT, WAN-deny and LAN SSH/ping. `nix run .#lab` boots the same topology in tmux with one console pane per VM and the driver REPL. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,86 @@
|
||||
# nixos-router — Plan
|
||||
|
||||
Working name: `nixos-router`. A NixOS-based distribution for routers/firewalls, delivered as a flake exporting NixOS modules and bootable images (not a nixpkgs fork).
|
||||
|
||||
## Decisions
|
||||
|
||||
| Area | Decision |
|
||||
|---|---|
|
||||
| Audience | Own homelab first; keep structure clean enough to publish later. |
|
||||
| Config model | Opinionated `router.*` option schema compiled to systemd-networkd + nftables, with build-time assertions. |
|
||||
| Hardware | x86_64 UEFI first. aarch64 is a planned target; avoid x86-only assumptions. |
|
||||
| v1 features | IPv6 + prefix delegation, VLANs / multiple LANs, WireGuard. |
|
||||
| Deferred | Multi-WAN failover, QoS (CAKE/SQM), web UI. |
|
||||
| Updates (v1) | Push from workstation (`nixos-rebuild --target-host` / colmena) + commit-confirm rollback timer. |
|
||||
| Updates (later) | **A/B image updates** with automatic rollback. |
|
||||
| Footprint | Minimal closure, read-only root, no Nix daemon/compiler on device, small persistent `/var`. |
|
||||
|
||||
## Architecture
|
||||
|
||||
1. **`router.*` schema** — users describe intent (wan, lan, vlans, zones, port forwards, wireguard); modules compile it to:
|
||||
- systemd-networkd (DHCPv4/v6, DHCPv6-PD, VLANs, bridges, PPPoE later)
|
||||
- nftables zone-based ruleset (generated, not `networking.firewall`)
|
||||
- DHCP / DNS / RA services
|
||||
2. **Build-time validation** — Nix assertions reject lock-out or inconsistent configs (no management access rule, overlapping subnets, zone referencing missing interface); generated ruleset checked with `nft -c`.
|
||||
3. **Images** — x86_64 first (installer ISO + raw disk), aarch64 later. Read-only root (erofs/squashfs store), small persistent `/var`.
|
||||
4. **Deployment** — deploy a prebuilt closure (`nix copy` + `switch-to-configuration`); never evaluate or build on the router. This keeps the device light and leads naturally to A/B updates.
|
||||
|
||||
## Implications
|
||||
|
||||
- No on-device evaluation: all designs must work from a prebuilt closure.
|
||||
- Flake exposes `x86_64-linux` and `aarch64-linux` outputs from the start; hardware specifics (bootloader, kernel, NIC naming) live in separate `boards/` modules.
|
||||
- IPv6 PD fan-out to multiple VLANs needs stable per-LAN subnet-ID allocation; this is the hardest part of the schema.
|
||||
|
||||
## Default zone policy (confirmed)
|
||||
|
||||
- WAN: deny inbound (except established/related, required ICMPv6, explicit port forwards).
|
||||
- LAN: allow outbound to WAN.
|
||||
- Other zones (e.g. IoT): isolated from each other and from LAN unless a rule opens them.
|
||||
|
||||
## Management access (confirmed)
|
||||
|
||||
- Key-only SSH, reachable from LAN zones (trusted, non-WAN, non-isolated zones). No GUI in v1.
|
||||
- Build-time assertion fails if no management path exists.
|
||||
- Isolated zones (e.g. IoT) get no router access beyond DHCP/DNS/NDP.
|
||||
|
||||
## Services (confirmed)
|
||||
|
||||
- DHCP: Kea (DHCPv4 + DHCPv6 incl. PD).
|
||||
- DNS: Unbound.
|
||||
- RA: systemd-networkd (or radvd if networkd falls short).
|
||||
|
||||
## Repo layout (confirmed)
|
||||
|
||||
```
|
||||
flake.nix
|
||||
modules/ # router.* schema and service modules
|
||||
boards/ # hardware-specific modules (x86_64 first, aarch64 later)
|
||||
tests/ # nixosTest VM tests
|
||||
images/ # image/ISO builds
|
||||
examples/ # sample configs
|
||||
```
|
||||
|
||||
## Status
|
||||
|
||||
**Phase 1 (MVP) skeleton done**: flake, `router.*` schema (wan, lans, dns.forwarders), networkd, nftables (zone-less for now: WAN/LAN), Kea DHCPv4, Unbound, assertions. `nix build .#checks.x86_64-linux.basic` runs a 3-node VM test (upstream/router/client) covering WAN DHCP, LAN DHCP+DNS, NAT, WAN-deny and LAN SSH.
|
||||
|
||||
Interactive lab: `nix run .#lab` (x86_64) opens tmux with one console pane per VM plus the test-driver REPL (`router.shell_interact()`, `client.succeed(...)`, `test_script()`). The REPL pane also shows all VM output interleaved.
|
||||
|
||||
Learned along the way:
|
||||
- The default policy now explicitly answers ICMP echo on LANs only; the WAN gets only ICMP error types and ND.
|
||||
- In NixOS tests `fail(...)` also passes on "command not found": install probe tools explicitly and add a positive control per negative check.
|
||||
- Unbound answers `.test` locally (RFC 6761) and validates DNSSEC; tests using fake domains need both `local-zone ... transparent` and `domain-insecure` (both verified necessary; set in the test, not the module).
|
||||
- Subnets are given explicitly (`subnet` + `address`) for now; a small CIDR helper lib is needed before IPv6 PD / VLAN subnet-ID allocation.
|
||||
- Not yet covered: IPv6, VLANs, WireGuard, formal zones, a test for the assertions themselves (verified manually via `nix eval`).
|
||||
|
||||
## Open items
|
||||
|
||||
- **Web UI** (stats and possibly configuration) — to be discussed later. Considerations: read-only stats first is cheap; config editing must write declarative config (Nix/JSON) that feeds the module, not mutate the live system.
|
||||
|
||||
## Phases
|
||||
|
||||
1. **MVP** — flake with `router.*` schema: single WAN (DHCP), single LAN, NAT, basic firewall, DHCP + DNS. NixOS VM tests (`nixosTest`) in CI.
|
||||
2. **Real-world features** — IPv6 PD, VLANs, zone policy, port forwards, WireGuard.
|
||||
3. **Images and hardware** — installer ISO, read-only root, aarch64.
|
||||
4. **Operations** — commit-confirm rollback, monitoring (node exporter, nftables counters), A/B image updates; multi-WAN failover.
|
||||
5. **Optional** — web UI (stats, then configuration).
|
||||
Reference in New Issue
Block a user