the Vimoire

recent

See 69 more →

Installing my new Cisco CBS350-16T-2G switch and configuring VLANs with OPNsense

Mar 24, 20257 min read

The saga so far

Helpful links and inspirations

The saga continues

Bit the bullet and bought a CBS350-16T-2G - 16-port non-PoE (almost a $100 difference to get the PoE one) switch from Cisco, didn’t seem like I needed a stupid cloud account to access the web GUI or the CLI. I’m used to older switches, and all CLI configuration, so making the jump to this one was a little bit new to me. Overall, I am happy with the purchase.

Now, jumping into the process. Or rather, for this first bit, a mostly linear and largely inaccurate description of the extremely non-linear process.

Famous last words, if you recall from my previous post - I’ll deal with the firewall rules later.

So, I thought, I had already proved that this would work with my previous configuration. What else to do except flip the power strip off and redo my whole server “rack” (shelf, it’s a shelf.)? What could go wrong?

That’s what I did. I nuked the whole circuit and started rewiring all my machines into the new switch. This process didn’t take too long, but I ended up fiddling entirely too long with cable management for a less-than-stellar result. I turned everything back on, and figured once I set up the trunk port and all the access ports on the switch to the correct VLANs, all should work nicely. I proceeded to spend the next 3 hours banging my head against the wall.

The key thing that I didn’t realize, which should have clicked within 5 minutes: DHCP was working. No other connections were. That’s a dead ringer for firewall. In retrospect, it should have been very obvious what was happening. I’ll get to that in a bit.

The first thing I did after verifying I was receiving an IP, as one does, is check ping. Turns out, it didn’t work. Hm.

At first, I thought it was an issue with the native VLAN, because I had been so worried about those couple posts I found saying not to mix tagged and untagged traffic from one interface on BSD-based systems. If I had done all switch config through the CLI, I would maybe have been less hung up on this aspect, but since I was doing partial CLI and mostly GUI configuration (there are like 6 separate menus for VLAN configuration!), I checked and re-checked that I hadn’t missed something. In the process, I probably fixed some issues that would have come up later down the line, but I most certainly did NOT fix the primary issue.

Then stumped, I figured, it must be some difference in configuration from the previous switch. That one had been working, right? I plugged that one up (luckily hadn’t taken it back to work yet or nuked the configuration). No dice. Turns out, my brilliant ass hadn’t checked ping connectivity on that configuration, or I would have discovered this multiple days prior. Same exact issue: DHCP worked, no ping (or any other connection for that matter).

I then had the brilliant idea to remove the IP address from the physical OPNsense interface, and just leave the VLAN interfaces - maybe that was causing the conflict. I did so. I then did not have access to the OPNsense web GUI for the next hour.

Over that next hour, something clicked. Every once in a while I would step away and do a bit of research on my phone, and I came across this post. There’s a section in there that explains exactly what I did wrong, and what should have been obvious right from the get-go. That section is called Add the Appropriate Firewall Rules.

As it happens, default firewall rules are created automatically when you use the nifty little OPNsense script to set the WAN and the LAN interfaces upon initial install. Now, when you create a custom interface - such as when I created my Tailscale VPN interface - you get the default rules…e.g. allow DHCP…and default deny on (most) everything else. That default deny, by the way, includes ICMP.

I then proceeded to dig myself out of the web GUI hole by running the OPNsense setup script from the CLI (sudo /usr/local/sbin/opnsense-shell, thank you to this forum post) and reassigning the LAN interface IP address, then restoring my config from backup (heralds be blessed that I remembered to back up, prior to beginning any of this). From there, I cloned the default allow firewall rules from the LAN interface. (Of note, in this process I tried to edit a config XML file directly before finding that script - went better than expected, which is to say it didn’t explode my router, but it didn’t work.)

And we had connection!

Actual configuration notes

Most of this follows what I did in the previous posts.

Switch

Trunk port and access ports are basically the same, just translating CLI speak to clicky here speak.

I will note in detail the pains I went through to get rid of the native VLAN traffic as well as general lockdown for the rest of the switchports:

  • All under “VLAN Management”
    • VLAN settings → edit VLAN 1 → uncheck “VLAN Interface State”
    • Port to VLAN → set VLAN 1 to Excluded on ALL interfaces
    • Port VLAN Membership
      • “Join VLAN” on trunk port → set Native VLAN to None
      • Set ALL other interfaces to either 1) a used VLAN with a device connected or 2) an unused “parking” VLAN
  • Under Port Management
    • Port Membership → set all unused ports to Administratively Down and Protected

OPNsense

High level overview only, refer to my previous posts and the helpful links I listed above:

  • Create VLANs
  • Assign VLAN interfaces and IPs/subnets
  • Enable DHCP on new interfaces
  • Enable DNS on new interfaces
  • Create default allow firewall rules on new interfaces

In all its glory

(Not including my NAS in a mid-size tower, sitting on the floor)

Next steps

All these next steps kinda overlap and smush together - modeling my network out doesn’t do me much good unless the setup works with the equipment I have. So I’m mulling that all over. The biggest things are 1) how do I separate server management from server service access with only one physical interface? and 2) how do I serve both the user network and the guest network via my single WAP (2.4gHz and 5gHz)? I feel like both of these things are very achievable.

  • Model out my network - will start small and build as I get more familiar.
  • Figure out how to serve multiple VLANs on a single port (dual trunk and access) - use case, giving my server an IP in the 10.0.20.* range and a management IP in the 10.0.99.* range, as well as guest network? (It’s been a few days since I worked on this, and this thought is making less sense to me now, but I had a lightbulb moment when I was working on it. I’ll need to revisit this.)

Afterwards - meaning after I have the appropriate IPs in the appropriate places - I can see about dicking around with the firewall rules. (It doesn’t feel like procrastinating at this point, it just makes sense to do that last the more I think about it. My concept is now a reality, and it’s just a matter of expanding on it a bit more before jumping into the actual network separation.)

The next steps after that are locking down my raspberry pi as a dedicated jump server, and really taking a look at my Tailscale configuration.

EOF

Backlinks