the Vimoire

recent

See 69 more →

Migrating home router to Protectli Vault v1211 running OPNsense

Jan 03, 202517 min read

Inspiration

Louis Rossman over at FUTO recently posted these absolute chungus videos, and a written guide. I decided to do this with OPNsense instead of pfSense.

OK but this seems like overkill?

It might be. But I wanted something better than a consumer-grade router being the barrier in between me and the outside world, especially considering I am using Tailscale ubiquitously now.

Setup / goals

Devices on the LAN:

  • OPNsense router handling all wired traffic
  • Dumb switch handling extension of wired traffic (since I only have two ports on the Vault)
  • OpenWRT router acting as a dumb WAP, connected to the switch (since I don’t have wireless capability on the Vault)
  • All other wired devices (NAS, mini PC, etc) connected wired via the switch with static IPs
  • All endpoint devices connected via the OpenWRT WAP or wired via the switch, with IPv4 DHCP

Vault v1211 needs to handle the following:

  • LAN routing (replacing OpenWRT)
  • LAN DHCP (replacing OpenWRT)
  • WAN to LAN firewall (replacing OpenWRT)
  • Local DNS resolution (replacing Pi-hole)
  • Local DNS ad and tracker blocklist (replacing Pi-hole)
  • DNSSEC (replacing OpenWRT)
  • DNS over TLS / DoT (I didn’t yet have this configured, actually)
  • Tailscale VPN for remote LAN access
  • Tailscale DNS for remote devices

Vault v1211 cannot handle the following:

  • Switch for wired connections - will be using a dumb Netgear 10-port switch
  • WAP for wireless connections - will be using my OpenWRT router as a dumb WAP

Step 0: flashing coreboot

Protectli offers their devices from their website configurable: you can have coreboot preinstalled if you would like. I did not have the foresight and simply purchased it from Amazon with no configuration options. So I was left with the stock AMI BIOS, and I wasn’t happy. I really prefer coreboot. I knew if I didn’t flash coreboot now, before I had this as an always-on router critical to my LAN setup, I would never do it. So I did.

The Protectli documentation leaves a lot to be desired. The main thing I was running into was multiple places in their documentation stating that you absolutely had to boot Ubuntu in legacy mode (not UEFI) or you would probably definitely maybe run into issues flashing, and they weren’t responsible for your rash decisions. But their link to instructions on how to boot into legacy mode was 404’d, their link stating how to create a bootable USB drive doesn’t have anything about legacy mode, and any third-party documentation or forum posts I could find simply stated that Ubuntu would boot into legacy mode by default if the BIOS didn’t support UEFI - which wasn’t telling me how to force legacy boot. I dug around in the AMI BIOS for a solid hour trying to get it to boot legacy instead of UEFI, nothing worked for the 1211.

Documentation was also generally outdated. Here it says coreboot for the VXXXX series is “coming soon”, and there were other pages that I can’t be bothered to find again that confused me further by not listing the v1211 at all.

Eventually I said screw it and tried their flashing script, Flashli, anyway. The github page had support for the v1211 listed, and you can buy them with coreboot already installed, so what gives? Tl;dr, it just worked.

Steps:

  • Installed Ubuntu 20.04 onto the NVME drive. (Probably could have done this from a live disk, TBH)
  • Installed flashrom with sudo apt install flashrom
  • Downloaded their script from here
  • Ran the script with sudo ./flashbios
  • After successful flash, shut down, removed power cable for at least 5 seconds, rebooted.

So that was coreboot done in about 2 hours, which could have honestly taken 15 minutes. But I don’t regret being wary. In the end, I don’t regret buying it from Amazon either. Good learning experience.

Installing and configuring OPNsense

The installation process was fairly straightforward, I won’t go through it here.

Here are a couple articles that I referenced:

Also, since OPNsense is FreeBSD-based and I am a FreeBSD baby (all I really use at the moment is Debian headless), here are some helpful references for “mapping” Debian commands to FreeBSD:

  • fetch = wget
  • pkg = apt
  • ifconfig = ip a
  • drill = dig
  • bindkey -v = set -o vi (the most important one)

In terms of configuration, I’ll drop the steps to do so via the GUI below.

I will say that I’m glad I used the approach I did: performed initial setup offline, connected directly into the vault via HDMI or web GUI via ethernet, and set the IP to 10.0.99.1/24 - the then-current IP of my OpenWRT router - so that when I turned off everything, tore down my “rack” and reassembled, and then redeployed everything, all my devices (NAS, mini PC server, etc) that have static IPs set just worked without me having to HDMI into them.

Initial setup after installation and placing in rack

Go through the wizard in System → Wizard for initial setup. We’ll go back and change a lot of this so don’t worry too much about it. The main thing is getting your LAN IP correct so that you don’t need to HDMI in after putting it on your rack.

The one thing I did need to do via HDMI was auto-assign interfaces. Login as root, press “1” to Assign interfaces, then follow the instructions: use ‘a’ for autodetection, connect your port to your modem (WAN) and then connect your port to your switch (LAN).

Afterwards you should be able to log in via web GUI via the IP address you set (in my case, 10.0.99.1) to do the remainder of the configuration.

Note that until you configure DHCP you’ll need to set a static IP on the device you’re using to access the web GUI. My settings were as follows:

  • IP: 10.0.99.253 (doesn’t really matter as long as it isn’t *.1)
  • Subnet mask: 255.255.255.0
  • Gateway: 10.0.99.1
  • DNS: 10.0.99.1

Basic configuration

  • Change root password in System → Access → Users
  • Add a user in System → Access → Users
    • Add to admins group
    • Obviously set a secure password
    • Set login shell to /bin/tcsh
    • Add your local machine’s id_rsa file to Authorized keys
  • Enable SSH for newly added user in System → Settings → Administration
    • Check “Enable Secure Shell”
    • Set login group to wheel, admins
    • Don’t allow root login or password login
  • Enable sudo for newly added user in System → Settings → Administration
    • Ask password
    • Selected groups wheel, admins
  • Enable community repos (note that I did need to do a pkg update after I got internet and DNS working, before doing this step)
    • Login via SSH
    • fetch -o /usr/local/etc/pkg/repos/mimugmail.conf https://www.routerperformance.net/mimugmail.conf
    • pkg update
  • Install the following in System → Firmware → Plugins
    • os-ddclient for Dynamic DNS (will configure later)
    • os-tailscale and os-upnp for Tailscale and NAT-PMP setting for later
    • os-theme-rebellion for dark theme…my poor eyes!
  • Enable dark theme in System → Settings → General (set Theme dropdown to rebellion)
  • Set up DHCP in Services → ISC DHCPv4 → LAN
    • Check “Enable DHCP server on the LAN interface”
    • Set the range (I used *.129 to *.190 at the time, because I was thinking of a too-clever solution for my WAP. Just don’t set this low enough to conflict with any static IPs on your network - e.g. I have devices in the <100 range, so I would set it to above 100. It’s also nice to leave some room at the top of the range, 245-255 is usually the default from what I’ve seen)
    • Everything else should be defaults

At this point you should have internet, and you should be able to wire in to the switch and receive IP configuration via DHCP. SSH and web GUI should be working.

There was also something I needed to with DNS, I think. I may have set an upstream DNS on the Wizard. I may have enabled Unbound DNS early in the process. I’m not certain, but I believe some level of DNS configuration was needed in order to access the internet at this point.

Dumb WAP with OpenWRT

I had tried something similar to this when I tried to spin up a guest network (this was pre-documentation). I didn’t (and still don’t) understand how all the firewall zones, bridge devices, interfaces, etc, interacted, so that project got scrapped and pushed to later. I revisited a lot of what I had been working on in that project at first, and thought I was in for a longer project.

I also was trying to be too clever and was trying to set up a separate subnet of sorts on the WRT router: 10.0.99.192/27, or address space 10.0.99.193-223, and letting the router do the DHCP and route back to the OPNsense. Eventually I found out that this was way too much effort for no real benefit and I should literally neuter the OpenWRT and let the OPNsense router handle all the bells and whistles. It’s there for the antennas and nothing else.

I will eventually loop back around to this to set up a guest network, and I WILL need the firewall functionality of the router, but until then it is simply so that we can connect wirelessly.

I had a working OpenWRT configuration, with the basic structure mostly default. There were a lot of little settings I had edited (e.g. DNSSEC with stubby), but those would all get swept up in the nuke I was about to do.

This took me a solid couple hours of trial and error, but this was the gist of it at the end:

  • Several edits to the LAN interface in Network → Interfaces:
    • Set IP to something on the 10.0.99.0/24 network, NOT in the DHCP range, and NOT the OPNsense IP
    • Set gateway to OPNsense IP (10.0.99.1)
    • In Advanced Settings, set DHCP server to OPNsense IP (10.0.99.1)
    • In DHCP Server, check “Ignore interface”
  • Stop and disable (uncheck “bring up on boot”) the WAN and WAN6 interfaces in Network → Interfaces
  • Disable dnsmasq entirely in Network → DHCP and DNS
  • In Network → Firewall, delete all zones, set Input Output and Forward all to Accept
  • I already had a WLAN configured and didn’t need to adjust anything there.

One thing that tripped me up: you have to plug the cable back to the switch into one of the LAN ports, NOT the WAN port. I had previously messed with the firewall settings and the switch settings to no avail. In the end, I left the switch settings at the OpenWRT defaults, and just disabled the firewall. The switch already has the bridge device enabled, so traffic passes just fine between the LAN port and the WLAN.

Dynamic DNS with DuckDNS

This can work with other providers, but I already had a dynamic DNS setup with my OpenWRT router, so I stuck with that.

You need the os-ddclient plugin installed (which we did in Basic configuration).

A tip from Louis: set your IP in the DuckDNS admin panel to something incorrect, so that you can verify that your config is correct (it should update shortly after you add the following).

In Services → Dynamic DNS, add:

  • Service: duckdns
  • Username: blank
  • Password: paste your token from DuckDNS here
  • Hostname(s): this is only the subdomain of *.duckdns.org (e.g. if your domain is domain.duckdns.org, what you would put here is simply domain).
  • Interface to monitor: WAN
  • Remainder at defaults.

Unbound DNS

One of the goals was to replace my Pi-hole - mainly because I could, but at this point I had also run out of ethernet cables.

This section ties in with the next section regarding Tailscale - I set them up simultaneously as I was troubleshooting - but I’ve separated the sections here.

So again, the goals here:

  • Local DNS resolution (DNS overrides)
  • DNS blackholing
  • DNSSEC and DoT
  • All of this available remotely (when we get to Tailscale)

Basic options (and DNSSEC)

In Services → Unbound DNS → General:

  • Check the box to Enable Unbound
  • This should be the default, but set Network Interfaces to All if it isn’t (I changed this and had to change it back to enable it to serve DNS over Tailscale)
  • Check the box to Enable DNSSEC support
  • Remainder at defaults

Local DNS resolution

In Services → Unbound DNS → Overrides, add an override:

  • Host: *
  • Domain: domain.com
  • Type: A
  • Value: 10.0.99.40

This should enable local DNS for domain.com as I outlined here.

Note for Tailscale config when we get to that, on my Pihole I used the Tailscale IP (100.*) here, so that devices could only connect to the server if they were connected to the Tailscale VPN, but would still be able to connect if they were on a remote LAN. We’ll enable a setting in Tailscale later that gives us the best of both worlds: devices on LAN not connected to Tailscale can access it, and remote devices on Tailscale can access it with the local IP! (I kinda stumbled across this: originally I set the DNS resolution to the local IP for testing purposes only.)

DNS over TLS (DoT)

In Services → Unbound DNS → Query Forwardings, add a forwarding:

  • Domain: leave blank to forward all
  • Server IP: 8.8.8.8
  • Server port: 853
  • Description: Google DNS 1

Repeat, with Server IP 8.8.4.4 description Google DNS 2, as a backup

Note

The above is actually unnecessary. The DoT forwardings below accomplish everything you need for this configuration.

In Services → Unbound DNS → DNS over TLS, add a forwarding:

  • Domain: leave blank to forward all
  • Server IP: 8.8.8.8
  • Server port: 853
  • Hostname: dns.google
  • Description: Google DoT 1

And do the same with 8.8.4.4.

After enabling this, you can monitor the ports with the following tcpdump commands to verify DoT is being used:

  • sudo tcpdump -i <wan-interface> \( dst port 853 or src port 853 \) ← there should be plenty of traffic here.
  • sudo tcpdump -i <wan-interface> \( dst port 53 or src port 53 \) ← there should be no traffic here.

DNS blackhole

Ads and trackers begone!

In Services → Unbound DNS → Blocklist:

  • Check the box to Enable
  • In Type of DNSBL, select all the blocklists you would like to enable

This one was remarkably easy. I was thinking I’d need to use AdGuard or BIND or something else, I even thought about finding an ethernet cable for my RPI5, but I’m perfectly satisfied with these options.

You can also add regexes to whitelist/blocklist, which may come up in the future.

Tailscale

This ended up being more complicated than I expected, but I was also able to set up subnet pass through to LAN like I had previously with my WireGuard bounce server.

Installation

Impeccable timing, but there just recently was an official package added for Tailscale, so I am happy. We already installed that package in Basic configuration.

Set up interface and firewall allow rule

In Interfaces → Assignments:

  • Go ahead and assign tailscale0. I named it [TAIL].
  • Interface can be left at defaults.

In Firewall → Rules → [TAIL], create a new rule:

  • Accept defaults. Rule should look like the following:
  • Putting a description like “Allow all incoming tailscale traffic” is a good idea.

General settings

In VPN → Tailscale → Authentication:

  • Leave login server at default.
  • Paste the “Node key” from the Tailscale admin panel into “Pre-authentication key”. You’ll also want to disable key expiry for the OPNsense box on the Tailscale admin panel side.

In VPN → Tailscale → Settings:

  • Check box for Enabled
  • Listen port at default
  • Accept DNS UNchecked
  • Advertise exit node UNchecked
  • Accept subnet routes UNchecked
  • In the advertised routes tab:
    • Add a route for 10.0.99.0/24

Most of these are at their defaults, but you want to make sure if you’re planning to use Tailscale for DNS for remote devices.

You’ll need to approve changes in the Tailscale admin panel for the newly advertised subnet route.

This is what enables remote LAN access without having to have every device connected to Tailscale (functions like the AllowedIPs line in WireGuard config - in fact Tailscale is probably using that in the backend).

OPNsense specific settings

Couple things we need to change due to how OPNsense handles NAT routing. (I won’t claim to understand fully why, but I trust the article.) Basically how I understand it is reducing latency by not having to hop through DERP relays, and instead make a direct connection - but OPNsense default configuration doesn’t play nice with that.

Note

At the time I enabled these, I was still having big latency issues with DNS which I subsequently resolved by configuring Unbound correctly. But I enabled both of these options. Based on the Tailscale article, it seems like you only need one of the two enabled. I haven’t run into any issues so far with having both enabled, but I will probably disable one of them to test.

Static NAT port mapping

In Firewall → NAT → Outbound

  • Select “Hybrid outbound NAT rule generation”
  • Create a new rule:
    • Protocol: UDP
    • TCP/IP version: IPv4
    • Source address: LAN net
    • Static-port: checked
    • Description: Tailscale IPv4
    • Remainder at defaults
  • Create another rule for IPv6 (edit TCP/IP version and Description)

The documentation isn’t very clear on this: you also need to add randomizeClientPort setting to your access controls (ACLs), in the Tailscale admin panel, here. It wasn’t clear where I needed to add it, so I did so by trial and error. This ended up being the correct configuration:

# ...

	"randomizeClientPort": true,
	"acls": [
	// list of acls
	]

# ...

NAT-PMP

This is why we installed the os-upnp plugin.

In Services → Universal Plug and Play:

  • Simply check “Allow NAT-PMP Port Mapping”

IPv6 configuration

I don’t know if this is even really needed, but it was working with my OpenWRT router and I wanted to get it set up.

Couple helpful posts:

In Interfaces → WAN:

  • IPv6 configuration type: DHCPv6
  • DHCPv6 client configuration:
    • Basic
    • Prefix delegation size: 56
    • Send IPv6 prefix hint: checked

In Interfaces → LAN:

  • IPv6 configuration type: Track Interface
  • IPv6 Interface: WAN
  • IPv6 Prefix ID: 0

These settings populated an IPv6 address, and my devices started receiving IPv6 addresses.

Not that I really use IPv6 at the moment - I’m especially rusty - but I figure it wouldn’t hurt. I’ll probably end up revisiting this later, but who knows - I’m still not sure why we’re still using IPv4 at this point. If it hasn’t changed by now maybe it won’t in my lifetime.

Wrap-up

This post has taken about as long to write as the configuration did - which I’m happy about. I was preparing myself for a weekend-long ordeal, but I’m getting better at this kind of thing.

Right now I know there is one thing I’d like to revisit: a guest network on the OpenWRT WAP. I’ll probably end up messing with firewall rules, monitoring, etc… but I’m good for now. Got way further along than expected, and all my core functionality has been restored, just with a different device.

EOF

Backlinks