Don’t you already have a DNS server?

Well, yes. I spun up an OPNsense machine on a VPS for shits and giggles, really to see if I could, but then it just became a glorified Unbound machine.

It had its issues (primarily, every time it rebooted I had to go into the console and restart the web GUI), and it was $5 a month that was hard to justify paying. It was nice to have my DNS server always on in case my home internet ever went down, but I knew I could do better.

I’d had experience running Pi-hole before, and I also came across Uptime-Kuma. I’d been digging into uptime monitoring and doing a better job giving my servers the TLC they deserve - trying to make that easier for myself with tools such as Cockpit.

I also knew how to spin up a completely locked down server, accessible only via my Tailscale network, and I wanted to run through that process again to make sure I was doing everything right and putting in the sweat equity to know how to do it again, inevitably.

Installation and locking down

I will outline general steps here instead of trying to write out all the commands. One note, I chose to install Debian from a custom ISO so that I don’t have the unknowns that come along with whatever defaults the VPS provider uses.

Mostly from my new Debian VPS reference.

  • Install Debian via console with a custom ISO (vanilla Debian 13 ISO straight from their website)
    • Configure LVM with separate /home, /tmp, and swap partitions (I’d set up LVM before, but not separate /var and /tmp partitions)
    • Install only SSH server and standard system utilities (the more things installed, the more that I have to lock it down)
  • Configure Debian via console
    • Install UFW and enable - immediate first step, this is important. Since I wasn’t using the cloud-init, there was no default firewall blocking anything.
    • Ensure packages are up to date (they should be)
    • Install sudo and add your non-root user to sudo group
    • Change SSH port, allow root user login, restart SSH
    • Grab public IP
    • Add UFW rule for added port, exit
  • SSH into server as root
    • Change root password (only because it’s easier to copy paste from SSH terminal)
    • Disallow root user SSH login, restart SSH
  • SSH into server as user
    • Change password
    • Exit, copy SSH keys, SSH back in
  • Lock down SSH with 2FA

Configuration

Get comfy

Install a bunch of packages:

apt install zsh zsh-syntax-highlighting wget curl git logrotate rsyslog neovim tmux lf fzf locate tcpdump nmap

Then scp dotfiles and chsh to ZSH.

Install and configure necessary services

  • Install Docker
  • Install Nginx and generate certificate (my first post and my second post)
    • apt install python3-certbot-dns-cloudflare python3-pip
  • Create the Cloudflare API token, generate the certificate, create the snippets file and ssl-params file. Then create the /etc/nginx/sites-available server file and link it to /etc/nginx/sites-enabled.
    • Make sure to add a listen 443 ssl; line - mine was previously in the snippets file to listen only on the Tailscale interface, which I eventually put back, but I forgot to add it after I removed it and was wondering why the site wasn’t loading.
  • Install Tailscale (run with --accept-routes=true and operator=$(whoami)), approve in the Tailscale admin panel, disable key expiry. (The reason you’ll want to run with the DNS flag is because of the Uptime-Kuma installation later.) Make sure the Tailscale service is set to run at boot time.
  • Set all services to listen on tailscale0 only
    • For Nginx: edit the line in listen 443 ssl; line to be listen <tailscale-ip>:443 ssl; (easier to add this to the snippets file)
    • For SSH: add the ListenAddress: <tailscale-ip> line in /etc/ssh/sshd_config
  • Set up tailscale-online systemd service (as outlined here)
  • Edit systemd services to wait until Tailscale has an IP (as outlined here)
  • Add UFW rule to allow all traffic on tailscale0 (ufw allow in on tailscale0)

Now reboot and adjust until everything loads correctly.

Set up AdGuard docker container for DNS

I initially went with Pi-hole but ultimately realized it wasn’t a full-fledged recursive DNS server without some additional configuration steps. Also, the fact that I couldn’t add wildcard A records and would need to add each subdomain individually was basically a deal-breaker in itself.

I’d heard of AdGuard before, and after a bit of research I went with that.

After I did the initial install, reverse proxy, and very little setup, I left this running overnight. When I woke up in the morning, I found that I had random DNS requests across the WAN - and they had been fulfilled. I had bound the DNS (and DoT/DoH) ports to 0.0.0.0 instead of to the localhost or Tailscale IP, because I figured my UFW rules would catch them. They didn’t - I’m guessing the DNS port is deemed okay to be open, but that wasn’t apparent to me (and something about UFW that irks me - “uncomplicated” shouldn’t necessarily mean “opaque to the user”). This ultimately spooked me, and so I spent a long time investigating it to make sure it was only accessible to my devices. I ran into some confusion when my DNS requests were being answered (I guess) by my VPS provider, and I thought that for some reason the Docker container wasn’t binding. I did some testing with tcpdump and checked the AdGuard logs - I definitely wasn’t fulfilling those requests from my machine, so it seemed to be working. It did throw me off for a minute, though.

Eventually I ran into the fact that I needed to run it in “host” networking mode so that I could see all the IP addresses for logging, so I ended up having to edit a different configuration to lock it down after that, as I noted below.

  • Set up AdGuard Docker compose (and .env) files such as the following:
services:
  adguard:
    container_name: adguard
    image: adguard/adguardhome:latest
    restart: unless-stopped
    env_file:
      - .env
    network_mode: host
    volumes:
      - ${DATA_DIR}:/opt/adguardhome/work
      - ${CONF_DIR}:/opt/adguardhome/conf
  • Again you’ll want to use host networking mode if you want your client IPs to show up correctly as their Tailscale addresses, instead of a Docker address. After that, and spinning up the container once for initial setup, you’ll want to edit the ${CONF_DIR}/AdGuardHome.yaml file:
    • Bind to 127.0.0.1:${WEBUI_PORT} address instead of 0.0.0.0
    • Bind the DNS host to the Tailscale IP address
  • Add a reverse proxy in Nginx - nothing special that I needed to do to get it up and running, plenty of examples elsewhere on my blog.
  • Do the initial setup in the Web UI:
    • Configure DoT (I used Mullvad’s)
    • Add all local DNS records (“overrides” as Unbound calls them, or “rewrites” as AdGuard calls them)
    • Review blocklists and add any desired, but the defaults are a good baseline for ads and trackers
    • Add names and tags for static Tailscale clients (for logging purposes)

Now you should just be able to select the new server as your DNS in Tailscale’s admin panel.

Set up Uptime-Kuma Docker container for uptime monitoring

Install Uptime-Kuma with Docker compose, set up your reverse proxy, and add some monitors - you’re off to the races.

This one was dead simple to get up and running, but I wanted more out of it - I wanted to be notified when my servers go down and come back up. I had surprisingly few options, given what is available - Uptime-Kuma uses Apprise.

One was Signal, and this is the one I wanted to do. However, it requires either 1) adding the server as a linked Signal device (not comfortable doing that), or 2) creating a new Signal account entirely, which requires a separate phone number, and separate phone (unless there is a way to add multiple Signal accounts in the app that I’m unaware of). So, that was out.

Email wasn’t going to work, since I run ProtonMail bridge on my VPS - if that server goes down, which is the main one I’d like to monitor, then I won’t see my email. I needed it on separate infrastructure.

There are plenty of others, but they all use services such as Gmail (which I just deleted), Telegram, WhatsApp, Discord - all these services that I do not have and am not going to sign up for. So I thought for a moment I was SoL and would need to revisit this when I had the gumption to dive into the Signal setup (which I might end up doing anyway).

Then I remembered I had set up an XMPP client with JMP.chat a while back for a separate phone number - that I hadn’t actually used for its intended purpose, but that’s beside the point - and I figured that might be an option. Lo and behold, Apprise supports XMPP.

However, the Uptime-Kuma Docker container has some issues with that. It’s built on Debian 12, and the latest python3-slixmpp package in that distro (needed for XMPP notifications) is version 1.8 or something. Come to find out, version 1.10+ is needed for XMPP notifications to work properly. So, I got it working eventually, but I had to use a workaround by grabbing the latest python3-slixmpp package from Debian’s FTP (here) then installing it manually. When the container updates, I’m going to have to do it again, alas.

If I come up with a permanent solution, or switch to Signal notifications, I will likely make another post about it.

Where from here

  • Permanent solution for notifications
  • Still need to set up backups to Backblaze
  • Still need to set up Watchtower for automatic container updates (but this will take notifications down silently)

EOF