nils/nixos-router
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add IPv6 inside the PPP session and use enp2s0 in the example

Browse Source 
- wan.ipv6.transport = "ppp": IPv6CP in pppd, router advertisements and
  DHCPv6 prefix delegation on the PPP device, DHCPv6 accepted on the PPP
  device, IPv6 MSS clamped to MTU-60. Allowed on a shared single-NIC
  link. A build-time check requires a PPPoE uplink.
- IPv4 MSS clamping now only matches IPv4 packets.
- The example uses the new box's NIC name (enp2s0).

Covered by the onestick VM test (radvd and a minimal DHCPv6-PD
responder as the ISP) and assertion cases.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
nils
2026-10-28 15:03:57 +01:00
co-authored by Claude Sonnet 5.5
parent b5429ad876
commit 70dddad206
9 changed files with 227 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files
PLAN.md
+2 -2
View File
@@ -118,7 +118,7 @@ Learned along the way:
- **Router on a stick**: `wan.interface` may also carry LANs, but only for PPPoE on a VLAN with IPv6 disabled (the untagged link is then free for a LAN; native IPv6 needs an untagged uplink). In that mode only the PPP device is an uplink in the firewall, so LAN traffic can never match WAN rules; the WAN VLAN is created from the LAN's/trunk's networkd unit. A LAN VLAN may not reuse the PPPoE VLAN id.
- **Commit-confirm** (`router.rollback`, on by default): `router-arm [time]` starts a transient timer; `router-rollback` re-activates the confirmed generation (`/nix/var/nix/profiles/system`, else `/run/booted-system`) with `switch-to-configuration test`; `router-confirm` stops the timer, sets the profile and the boot default. Deploy with `nix run .#deploy -- <flake-attr> <user@host> [--yes]`: build, arm, `nixos-rebuild test --target-host`, check SSH, confirm. A lost connection means the timer rolls back by itself; a reboot also returns to the confirmed generation.
- `systemd.network.wait-online` waits only for the LAN devices (not the uplink) by default: an unplugged uplink must not block boot or activation, while VLAN devices still exist before Kea starts (anyInterface broke this; caught by `vlans`).
- `examples/desktop` + `nixosConfigurations.example` is the deployable starting point (single NIC `eno1`, PPPoE VLAN 1001); it is built as a flake check. Secrets stay on the device (`/var/lib/router/pppoe-password`).
- `examples/desktop` + `nixosConfigurations.example` is the deployable starting point (single NIC `enp2s0` on the new box (antigua calls its NIC `eno1`), PPPoE VLAN 1001); it is built as a flake check. Secrets stay on the device (`/var/lib/router/pppoe-password`).
- Open for real hardware: Nix is still on the device (`router-confirm` uses `nix-env`); no image/read-only root yet.
## Local DNS and DHCP reservations (confirmed)
@@ -150,7 +150,7 @@ Source: the user's infra repo (host `antigua`, NixOS, dnsmasq + networkd + pppd)
Done: PPPoE on VLAN 1001 over the shared NIC (`eno1`, LAN untagged), DNS records/domain/forward zones/DHCP reservations/DNS for site-to-site peers, WireGuard (freifunk dial-out with masquerade, family site-to-site with PSK, mobile-clients road warrior), MSS clamp.
Open (in rough order):
1. **IPv6 over the PPP session**: antigua runs `+ipv6` and DHCPv6-PD on the PPP device itself (not on an untagged link); the module only knows native IPv6 on the uplink. Needs ipv6cp in pppd, networkd DHCPv6/RA on the PPP device, PD to the LAN.
1. ~~**IPv6 over the PPP session**~~ **done**: `wan.ipv6.transport = "ppp"` (default `native`) enables IPv6CP in pppd, RA + DHCPv6-PD via networkd on the PPP device, PD to the LAN, DHCPv6 replies accepted on the PPP device and the IPv6 MSS clamped to MTU-60 (1432). Allowed on a shared link (single NIC). Tested in `onestick` (ISP with radvd and a minimal DHCPv6-PD responder, since Kea cannot serve point-to-point links); mutation-tested (MSS clamp, DHCPv6 input rule). `ipv6cp-use-ipaddr` is set as on antigua.
2. **ULA on the LAN** (`fd2c:cdef:cde9::1/64`, announced by RA): needs an extra static address / ULA option per LAN.
3. **Tunnel trust and addressing**: antigua trusts `family`, `freifunk`, `mobile-clients` fully and gives two of them no address; check the schema covers this (`allowManagement`, `dns`, `internet` per tunnel; `addresses` may be empty?).
4. **PPP device name**: `wan` on antigua, `pppoe-wan` here; `ddclient` (Cloudflare, uses the interface address) must follow, or make the name configurable.
examples/desktop/default.nix
+5 -5
View File
@@ -1,14 +1,14 @@
# A single-NIC desktop PC as the router ("router on a stick").
#
# ISP -- ONT/modem -- switch/port --+-- eno1 (untagged) LAN 192.168.10.0/24
# +-- eno1.1001 PPPoE uplink (IPv4)
# ISP -- ONT/modem -- switch/port --+-- enp2s0 (untagged) LAN 192.168.10.0/24
# +-- enp2s0.1001 PPPoE uplink (IPv4)
#
# The switch port the PC is plugged into must carry VLAN 1001 tagged (towards
# the ISP) and the LAN untagged. IPv6 is off: without a second link there is no
# untagged uplink for DHCPv6 to run on.
#
# Before the first deployment:
# 1. Check the NIC name (`eno1`) with `ip link` on the target box. Disk layout and
# 1. Confirm the NIC name (`enp2s0`) with `ip link` on the target box. Disk layout and
# drivers come from the installer layout, so no hardware scan is needed.
# 2. Put your SSH public key below (otherwise nobody can log in over the network).
# 3. Put the PPPoE username below and the password in /var/lib/router/pppoe-password
@@ -34,7 +34,7 @@
enable = true;
wan = {
interface = "eno1";
interface = "enp2s0";
ipv4 = {
type = "pppoe";
pppoe = {
@@ -46,7 +46,7 @@
};
lans.main = {
interface = "eno1";
interface = "enp2s0";
internet = true;
ipv4.address = "192.168.10.1";
dhcp.pool = {
modules/router/assertions.nix
+10 -2
View File
@@ -416,8 +416,16 @@ in
{
assertion =
!(lib.elem cfg.wan.interface (map (l: l.interface) lans))
|| (cfg.wan.ipv4.type == "pppoe" && cfg.wan.ipv4.pppoe.vlan != null && !cfg.wan.ipv6.enable);
message = "router: the WAN interface `${cfg.wan.interface}` is also used by a LAN (only allowed for PPPoE on a VLAN with IPv6 disabled).";
|| (
cfg.wan.ipv4.type == "pppoe"
&& cfg.wan.ipv4.pppoe.vlan != null
&& (!cfg.wan.ipv6.enable || cfg.wan.ipv6.transport == "ppp")
);
message = "router: the WAN interface `${cfg.wan.interface}` is also used by a LAN (only allowed for PPPoE on a VLAN, with IPv6 disabled or inside the PPP session).";
}
{
assertion = cfg.wan.ipv6.transport == "native" || cfg.wan.ipv4.type == "pppoe";
message = "router: wan.ipv6.transport = \"ppp\" needs a PPPoE uplink (wan.ipv4.type = \"pppoe\").";
}
{
assertion =
modules/router/firewall.nix
+11 -3
View File
@@ -183,6 +183,10 @@ let
sharedLink = lib.elem cfg.wan.interface (map (l: l.interface) lans);
uplinks = lib.unique ([ uplink4 ] ++ lib.optional (!sharedLink) cfg.wan.interface);
mss = cfg.wan.ipv4.pppoe.mtu - 40;
mss6 = cfg.wan.ipv4.pppoe.mtu - 60;
v6ppp = cfg.wan.ipv6.enable && cfg.wan.ipv6.transport == "ppp";
# Where DHCPv6 replies arrive.
v6dev = if v6ppp then uplink4 else cfg.wan.interface;
dhcpIfaces = map (l: l.device) (lib.filter (l: l.dhcp.enable) lans);
in
{
@@ -212,7 +216,7 @@ in
iifname ${set memberIfaces} icmpv6 type echo-request accept
${lib.optionalString (!pppoe) ''iifname "${cfg.wan.interface}" udp dport 68 accept''}
${lib.optionalString cfg.wan.ipv6.enable ''iifname "${cfg.wan.interface}" udp dport 546 accept''}
${lib.optionalString cfg.wan.ipv6.enable ''iifname "${v6dev}" udp dport 546 accept''}
iifname ${set dnsIfaces} udp dport 53 accept
iifname ${set dnsIfaces} tcp dport 53 accept
@@ -237,8 +241,12 @@ in
${lib.optionalString pppoe ''
# Clamp MSS to the PPP MTU in both directions (never raised).
iifname "${uplink4}" tcp flags syn tcp option maxseg size > ${toString mss} tcp option maxseg size set ${toString mss}
oifname "${uplink4}" tcp flags syn tcp option maxseg size > ${toString mss} tcp option maxseg size set ${toString mss}
iifname "${uplink4}" meta nfproto ipv4 tcp flags syn tcp option maxseg size > ${toString mss} tcp option maxseg size set ${toString mss}
oifname "${uplink4}" meta nfproto ipv4 tcp flags syn tcp option maxseg size > ${toString mss} tcp option maxseg size set ${toString mss}
${lib.optionalString v6ppp ''
iifname "${uplink4}" meta nfproto ipv6 tcp flags syn tcp option maxseg size > ${toString mss6} tcp option maxseg size set ${toString mss6}
oifname "${uplink4}" meta nfproto ipv6 tcp flags syn tcp option maxseg size > ${toString mss6} tcp option maxseg size set ${toString mss6}
''}
''}
${lib.optionalString (
internetIfaces != [ ]
modules/router/networking.nix
+31 -4
View File
@@ -2,6 +2,11 @@
let
cfg = config.router;
v6 = cfg.wan.ipv6.enable;
v6ppp = v6 && cfg.wan.ipv6.transport == "ppp";
# IPv6 on the physical link (not inside the PPP session).
v6native = v6 && !v6ppp;
# The device carrying DHCPv6 / RAs.
v6dev = if v6ppp then cfg.wan.ipv4.interface else cfg.wan.interface;
pppoe = cfg.wan.ipv4.type == "pppoe";
vlan = cfg.wan.ipv4.pppoe.vlan;
vlanDevice = "${cfg.wan.interface}.${toString vlan}";
@@ -20,7 +25,7 @@ let
lib.unique (map (l: l.interface) vlanLans)
);
dhcp = if pppoe then (if v6 then "ipv6" else "no") else (if v6 then "yes" else "ipv4");
dhcp = if pppoe then (if v6native then "ipv6" else "no") else (if v6 then "yes" else "ipv4");
in
{
config = lib.mkIf cfg.enable {
@@ -66,10 +71,15 @@ in
networkConfig = {
DHCP = dhcp;
}
// lib.optionalAttrs v6 {
// lib.optionalAttrs v6native {
IPv6AcceptRA = true;
KeepConfiguration = "yes";
}
// lib.optionalAttrs (pppoe && !v6native) {
# IPv6 (if any) lives inside the PPP session, never on the link.
LinkLocalAddressing = "no";
IPv6AcceptRA = false;
}
// lib.optionalAttrs (pppoe && vlan != null) {
VLAN = [ vlanDevice ];
};
@@ -78,7 +88,7 @@ in
# The router runs its own resolver; never adopt the ISP's DNS for the host.
dhcpV4Config.UseDNS = false;
}
// lib.optionalAttrs v6 {
// lib.optionalAttrs v6native {
# The ISP sends RAs (default route) without DHCPv6 flags, so solicit anyway.
dhcpV6Config = {
WithoutRA = "solicit";
@@ -87,6 +97,23 @@ in
};
};
}
// lib.optionalAttrs v6ppp {
# IPv6 inside the PPP session: pppd (IPv4, IPv6CP link-local) creates the
# device; networkd takes RAs and DHCPv6 (prefix delegation) on it.
"12-wan-ppp" = {
matchConfig.Name = cfg.wan.ipv4.interface;
networkConfig = {
DHCP = "ipv6";
IPv6AcceptRA = true;
KeepConfiguration = "yes";
};
dhcpV6Config = {
WithoutRA = "solicit";
UseDelegatedPrefix = true;
UseDNS = false;
};
};
}
// lib.optionalAttrs (pppoe && vlan != null) {
# pppd configures the session; networkd only has to bring the device up.
"11-wan-pppoe" = {
@@ -118,7 +145,7 @@ in
}
// lib.optionalAttrs lan.ipv6.pd {
dhcpPrefixDelegationConfig = {
UplinkInterface = cfg.wan.interface;
UplinkInterface = v6dev;
SubnetId = 0;
Announce = true;
Assign = true;
modules/router/options.nix
+21 -4
View File
@@ -71,10 +71,27 @@ in
};
};
ipv6.enable = mkEnableOption ''
IPv6 on the uplink: DHCPv6 (soliciting even without RA flags) plus
prefix delegation. The ISP is expected to delegate a single /64
'';
ipv6 = {
enable = mkEnableOption ''
IPv6 on the uplink: DHCPv6 (soliciting even without RA flags) plus
prefix delegation. The ISP is expected to delegate a single /64
'';
transport = mkOption {
type = types.enum [
"native"
"ppp"
];
default = "native";
description = ''
Where IPv6 runs. `native`: untagged on the uplink link, next to a
DHCPv4 or PPPoE (tagged) IPv4 uplink. `ppp`: inside the PPPoE
session (IPv6CP), so DHCPv6 and router advertisements arrive on
the PPP device; needs a PPPoE uplink, and leaves the physical link
free for a LAN.
'';
};
};
};
lans = mkOption {
modules/router/pppoe.nix
+6 -1
View File
@@ -28,7 +28,12 @@ let
defaultroute
replacedefaultroute
noipdefault
nodefaultroute6
${
if wan.ipv6.enable && wan.ipv6.transport == "ppp" then
"+ipv6\n ipv6cp-use-ipaddr"
else
"nodefaultroute6"
}
noccp
mtu ${toString pppoe.mtu}
mru ${toString pppoe.mtu}
tests/assertions.nix
+40
View File
@@ -940,6 +940,45 @@ let
};
expect = [ "not a valid hostname" ];
};
pppTransportValid = {
router = {
wan = {
ipv4 = {
type = "pppoe";
pppoe = {
vlan = 1001;
username = "u";
passwordFile = "/var/lib/pw";
};
};
ipv6 = {
enable = true;
transport = "ppp";
};
};
lans.main = lan {
interface = "eth0";
ipv6.pd = true;
};
};
expect = [ ];
};
pppTransportNeedsPppoe = {
router = {
wan = {
ipv4.type = "dhcp";
ipv6 = {
enable = true;
transport = "ppp";
};
};
lans.main = lan { interface = "eth0"; };
};
expect = [
"needs a PPPoE uplink"
"also used by a LAN"
];
};
nonOverlappingLans = {
router.lans = {
a = lan { };
@@ -988,6 +1027,7 @@ let
"wgValid"
"wanValid"
"sharedLinkValid"
"pppTransportValid"
"dnsValid"
];
tests/onestick.nix
+101 -1
View File
@@ -1,5 +1,6 @@
# Router on a stick: one NIC carries the untagged LAN and, as VLAN 7, the
# PPPoE uplink (IPv4 only), like a single-port box behind a VLAN-tagged ISP.
# PPPoE uplink, like a single-port box behind a VLAN-tagged ISP. IPv6 runs
# inside the PPP session: the ISP sends RAs and delegates a /64 on the PPP link.
#
# upstream (ISP, PPPoE on vlan 7) -- switch -- router (eth1) -- client, peer
#
@@ -68,6 +69,62 @@ pkgs.testers.runNixOSTest {
matchConfig.Name = "eth1.${toString vlan}";
networkConfig.LinkLocalAddressing = "no";
};
# The server's global address, reachable through the session.
systemd.network.networks."05-lo" = {
matchConfig.Name = "lo";
address = [ "2001:db8:1::1/128" ];
};
# RAs and DHCPv6 prefix delegation on the (dynamic) PPP device. Kea
# cannot serve a point-to-point link (it only sees the peer's address),
# so a minimal responder stands in for the ISP's DHCPv6 server. The test
# starts both once the session exists.
environment.systemPackages = [
pkgs.radvd
pkgs.python3
];
boot.kernel.sysctl."net.ipv6.conf.all.forwarding" = 1;
environment.etc."dhcp6-pd.py".text = ''
import socket, struct, subprocess, sys
PREFIX = socket.inet_pton(socket.AF_INET6, "2001:db8:2::")
DEV = sys.argv[1]
ifindex = socket.if_nametoindex(DEV)
sock = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.setsockopt(socket.SOL_SOCKET, 25, DEV.encode() + b"\0") # SO_BINDTODEVICE
sock.bind(("::", 547))
group = socket.inet_pton(socket.AF_INET6, "ff02::1:2")
sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_JOIN_GROUP, group + struct.pack("@I", ifindex))
SERVER_ID = struct.pack("!HH", 3, 0) + b"\x00\x03\x00\x01" + bytes(6)
def options(data):
out, i = {}, 0
while i + 4 <= len(data):
code, length = struct.unpack("!HH", data[i:i + 4])
out[code] = data[i + 4:i + 4 + length]
i += 4 + length
return out
def opt(code, body):
return struct.pack("!HH", code, len(body)) + body
print("dhcp6-pd: listening on", DEV, flush=True)
while True:
data, addr = sock.recvfrom(2048)
mtype, xid, opts = data[0], data[1:4], options(data[4:])
if mtype not in (1, 3) or 25 not in opts: # Solicit, Request with IA_PD
continue
iaid = opts[25][:4]
prefix = opt(26, struct.pack("!IIB", 3000, 4000, 64) + PREFIX)
ia_pd = opt(25, iaid + struct.pack("!II", 1800, 2700) + prefix)
rapid = mtype == 1 and 14 in opts
reply = bytes([7 if (mtype == 3 or rapid) else 2]) + xid
reply += opt(1, opts[1]) + opt(2, SERVER_ID[4:]) + ia_pd
if rapid:
reply += opt(14, b"")
sock.sendto(reply, addr)
print("dhcp6-pd: answered", mtype, "from", addr, flush=True)
'';
environment.etc = {
"ppp/options".text = "";
"ppp/chap-secrets".text = ''
@@ -75,6 +132,7 @@ pkgs.testers.runNixOSTest {
'';
"ppp/pppoe-server-options".text = ''
require-chap
+ipv6
noccp
mtu 1492
mru 1492
@@ -116,6 +174,10 @@ pkgs.testers.runNixOSTest {
enable = true;
wan = {
interface = "eth1";
ipv6 = {
enable = true;
transport = "ppp";
};
ipv4 = {
type = "pppoe";
pppoe = {
@@ -129,6 +191,7 @@ pkgs.testers.runNixOSTest {
interface = "eth1";
internet = true;
ipv4.address = "192.168.10.1";
ipv6.pd = true;
dhcp.pool = {
start = "192.168.10.100";
end = "192.168.10.199";
@@ -213,5 +276,42 @@ pkgs.testers.runNixOSTest {
client.fail("curl -sf --max-time 3 http://192.168.10.1:8080/")
router.succeed("nft list ruleset | grep -q 'iifname \"pppoe-wan\"'")
assert 'iifname { "eth1"' not in router.succeed("nft list ruleset"), "eth1 treated as an uplink"
with subtest("IPv6 comes up inside the PPP session"):
upstream.wait_until_succeeds("ip link show ppp0")
upstream.wait_until_succeeds("ip -6 -o addr show dev ppp0 scope link | grep -q inet6")
upstream.succeed("systemd-run --unit=dhcp6pd python3 /etc/dhcp6-pd.py ppp0")
upstream.succeed("printf 'interface ppp0 {\\n AdvSendAdvert on;\\n AdvDefaultLifetime 1800;\\n};\\n' > /tmp/radvd.conf")
upstream.succeed("systemd-run --unit=radvd6 radvd -n -C /tmp/radvd.conf")
router.wait_until_succeeds("ip -6 route show default | grep -q 'dev pppoe-wan'")
router.wait_until_succeeds("ip -6 addr show dev eth1 scope global | grep -q 2001:db8:2:")
# Nothing IPv6 on the physical link or the PPPoE VLAN.
router.fail("ip -6 addr show dev eth1.${toString vlan} | grep -q inet6")
ll = router.succeed("ip -6 -o addr show dev pppoe-wan scope link | awk '{print $4}' | cut -d/ -f1 | head -1").strip()
assert ll.startswith("fe80:"), ll
# The ISP routes the delegated prefix to the router's link-local address.
upstream.succeed(f"ip -6 route replace 2001:db8:2::/48 via {ll} dev ppp0")
with subtest("the client autoconfigures from the delegated prefix, without NAT"):
client.wait_until_succeeds("ip -6 addr show dev eth1 scope global | grep -q 2001:db8:2:")
client_v6 = client.succeed("ip -6 -o addr show dev eth1 scope global | awk '{print $4}' | cut -d/ -f1 | head -1").strip()
client.wait_until_succeeds("ping -6 -c 1 -W 2 2001:db8:1::1")
out = client.succeed("curl -sf -g --max-time 5 'http://[2001:db8:1::1]/'")
assert out.strip() == client_v6, f"expected {client_v6}, got {out!r}"
with subtest("IPv6 MSS is clamped to the PPP MTU"):
router.succeed("tcpdump -nn -i pppoe-wan -c 1 -w /tmp/syn6.pcap 'ip6 and ip6[6] == 6 and ip6[53] & 0x12 == 0x02' >/dev/null 2>&1 &")
router.wait_until_succeeds("pgrep tcpdump")
router.sleep(1)
client.succeed("curl -sf -g --max-time 5 'http://[2001:db8:1::1]/'")
mss = router.wait_until_succeeds("tcpdump -nn -v -r /tmp/syn6.pcap 2>&1 | grep -o 'mss [0-9]*'")
assert mss.strip() == "mss 1432", f"expected mss 1432, got {mss!r}"
with subtest("the ISP cannot initiate IPv6 to the router or the LAN"):
# Positive control: the ISP really has a route to the client.
upstream.succeed(f"ip -6 route get {client_v6} | grep -q 'dev ppp0'")
upstream.fail(f"ping -6 -c 1 -W 2 {client_v6}")
upstream.fail(f"nc -6 -z -w 2 {client_v6} 22")
upstream.fail(f"nc -6 -z -w 2 {ll}%ppp0 22")
'';
}