Attempting and failing to setup VLANs for multiple IP subnets on a switch-chip (hAP)

DrCroubie · March 1, 2025, 3:27am

I've tried reading through the documentation such as https://openwrt.org/docs/guide-user/network/vlan/managed_switch and https://openwrt.org/docs/guide-user/network/vlan/creating_virtual_switches and https://openwrt.org/docs/guide-user/network/vlan/switch_configuration and my brain melts after a few seconds, they're just too technical and don't actually help with what I'm trying to do.

Can someone please explain to me what to do like I'm someone who isn't a network admin for a dayjob?

Here's what I want (assuming the image uploads):

Each LAN port is a different subnet, different firewall zones, different rules etc. No port talks to any other port except the WAN (a whole bunch of custom firewall rules, that part I'm at least mildy competent in so don't need help there).

I tried getting help here trying to find a device that actually had separate eth ports attached to separate busses and no switch chips in between, but seems they don't exist without going back to an old PC with separate cards (like I had 20 years ago with a Pentium1 running IPCOP, or currently have with RPi and multiple USB-Eth dongles). The former just doesn't fit in my space or power budget, and the latter is the spaghetti I'm trying to get rid of now.

So what I've got now is a Mikrotik Routerboard (hAp RB951ui-2nd). After a day of wrangling, it now has 23.0.5 on it, so that's a start.
With another few hours of wrangling I managed to set the IP address of the WANport to what is now my local subnet, 192.168.64.x, so at least I can both talk to Luci and be on the internets at the same time without messing with cables or ip addresses.

But now I can't get the other ports to change IP addresses or anything. At the very least I want to set port 4 to 192.168.64.2 and then connect to Luci that way, then change port 1 to WAN so I can create some firewall rules between them, but I can't even do that.

So what I've got now is a mess that looks like:

And if all the rest of these upload, maybe they're useful too to see what I've gone through so far:

If eth0 is the GbE to the CPU (which is what the VLAN page says) and the other four 100M ports are connected to the switch chip separately on eth1, why is the virtual eth0.1 and not eth1.1? Or is the VLAN page wrong?

(I probably don't need VLAN6 for Wifi, but it's good to keep it there as a reminder)
Also have no idea what the difference is between tagged or untagged or what I actually need, but at least this shows what I want connected to what.

Attempted to create a new bridge device for Port5, is that what to do?

If I try to remove eth1 from the (working / default / inbuilt) bridge, I lose all connection and have to wait for it to rollback, even though the cable is connected to port1, which (should be) the WAN / eth0

I don't even know what that means

Is this even remotely close to what I should be doing?
Hang on, are these not backwards from the VLAN page? That page said that the GbE WANport was eth0 (so by elimination the switch is eth1). This page seems to agree with the sys/class/net that says the switch is eth0 (by elimination making the GbE WANport eth1)

Maybe? But it says 'no link' even though the cable is plugged into port 5.

64.46 is my current connection, the other 2 do squat no matter what I change.

I can't screw with this one yet, it's my only connection to the device through the GbE / WAN / Port1

You'd think I would be able to plug a cable into here and ping .64.2 yeah? Think again...

Or maybe I could ping 64.3? As they say in my country, "yeah nah".

Hey if the switch is eth1 maybe I can ping that?

Or if it's backwards I can ping eth0? Surely at least one of these last 3 would work? nope nopity nopenope.

I'm all out of ideas. Hope someone else has some...?

psherman · March 1, 2025, 3:33am

This post is really long, so I didn't read it in detail.... but what I can see immediately is that you have a swconfig device. You seem to be mixing swconfig and DSA syntax - this technique won't work. Further, it seems that you've got some issues because you're using dotted notation where it doesn't belong.

So that said, I'd recommend that you reset your device to defaults and then post your configs in text from. Additionally, please specify what subnet you what to use on each port (you said that each port will be its own subnet). From there, we can make this all work.

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:

Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/dhcp
cat /etc/config/firewall

DrCroubie · March 1, 2025, 3:36am

And one more confusion for good measure. Seems /etc/config/network names the 'option ports' completely backwards to the Luci VLAN page too, just for funsies:

At the very least, the vlan / vid numbers 1-5 correspond with the numbers 1-5 printed on the device case itself (FYI 1 is the WAN port, 2-5 are the LAN ports as printed on the case), I've at least verified this by plugging in a cable and watching the 'cable present' logo come up...

DrCroubie · March 1, 2025, 3:47am

I don't know what that means.

I also don't know what that means.

Explain it to me like I'm not a network admin for a dayjob.

These pretty much are the defaults, all I've done is manage to change the IP address of the WANport to 192.168.64.46 (which by default was what is called br-lan in the screenshots) so that I can talk to it at all. If I factory reset it, it goes back to Mikrotik RouterOS.

Repeating desired network diagram for clarity:

Or in text form:
wan subnet is 192.168.4.x/24
one lan subnet is 192.168.8.x/24
different lan subnet is 192.168.16.x/24
another different lan subnet is 192.168.32.x/24
yet another different lan subnet is 192.168.64.x/24 (the one my current PC is on)
wifi subnet is 192.168.128.x/24

cat /etc/config/network I just did in post above.
Wireless is disabled and I don't care about it yet.
DHCP is disabled and forever shall be.
Firewall isn't set up yet. once I've got the IP addresses set up I'll start on that. First thing is I just want to be able to set the 4 LAN ports to the right addresses on the right subnets, that's what I'm stuck on now.

psherman · March 1, 2025, 3:52am

The config you shared is not even close to defaults.

Reset and then post again. Please use text, not screenshots for all text based output.

DrCroubie · March 1, 2025, 3:58am

if I delete everything from the vlan page all I'm left with is this
I can guarantee I didn't write anything to do with the bridge, or br-lan, so besides the lines that say 'option ipaddr / gateway / dns', the rest is the default right there

root@OpenWrt:~# cat /etc/config/network 

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdf7:9d2c:2359::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0'
        list ports 'eth0.1'
        list ports 'eth1'
        option ipv6 '0'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.64.46'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option gateway '192.168.64.1'
        list dns '192.168.64.1'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '0'

psherman · March 1, 2025, 4:03am

The config is still messed up.

Please reset to defaults using the following command: firstboot -y && reboot

DrCroubie · March 1, 2025, 4:28am

... and now I can't connect to it at all. great.

note that these are the defaults after following the procedure of https://openwrt.org/toh/mikrotik/common with the whole dnsmasq dhcp tftp boot and flashing nand and whatever. All I did after following that was change the IP address I was connected to to 64.46.

And now I don't know where it is.
OK, managed to find it on 192.168.1.1, and now it's on port 2 instead of port 1.
So I've sshed in and changed it back to 64.46 so I can keep connection to here too. Now:

root@OpenWrt:~# cat /etc/config/network 

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd4c:f2e3:fdf3::/48'

config device
        option name 'eth1'
        option macaddr 'c4:ad:34:7c:70:6e'

config interface 'wan'
        option device 'eth1'
        option proto 'dhcp'

config interface 'wan6'
        option device 'eth1'
        option proto 'dhcpv6'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0.1'

config device
        option name 'eth0.1'
        option macaddr 'c4:ad:34:7c:70:6f'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.64.46'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option gateway '192.168.64.1'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '1 2 3 4 0t'

Yes, you're right, that is a completely different configuration to before. Looking more like what you were expecting?

But don't look at me, I'm just following instructions from here: https://openwrt.org/toh/mikrotik/common and here: https://openwrt.org/toh/mikrotik/rb941-2nd using the pre-canned firmware from here: https://downloads.openwrt.org/releases/23.05.5/targets/ath79/mikrotik/
That config (besides the failed attempts at editing the vlan page) is exactly how I found it.
Even before that, there was never an 'interface wan' nor 'interface wan6' before, nor was there ever a 'config switch' or 'config switch_vlan'.

If some defaults are different to other defaults then someone might want to look into changing something somewhere to make all the defaults the same as all the other defaults.

So now that it's been reset to a completely different default than the previous default, now what?

psherman · March 1, 2025, 6:35am

hmmm... looking at the link, I don't see anything that would have instructed you to change the OpenWrt config to the state you presented initially. The problem with the config was related to the stuff you were doing trying to setup VLANs and the like. But no worries... looks like it's back to a proper default condition now.

Yup... default OpenWrt address.

I don't know about the physical port labeling, but eth1 (CPU port) is the wan, and and eth0 (CPU port) is connected to the switch where there are 4 more ports.

FWIW, there are actually a bunch of devices like that, but maybe they don't have enough ports for your needs?? Anyway, it's not necessary since you can replicate that situation with a switch + VLANs.

Yup. Thanks for resetting it.

Now we can get to work making it do what you want.

You haven't proved specifics about the subnets per port, so I'll make a template and you can either use it directly or you can adjust it if you want.

First thing is you'll want to setup additional VLANs on the switch, each with just one physical/logical port plus the CPU (which is logical port 0).

So... edit VLAN 1 on the switch so it looks like this:

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '1 0t'

Now you can add the other VLANs:

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '2 0t'

config switch_vlan
        option device 'switch0'
        option vlan '3'
        option ports '3 0t'

config switch_vlan
        option device 'switch0'
        option vlan '4'
        option ports '4 0t'

Next, we'll make bridges for each of these VLANs:

config device
        option name 'br-vlan2'
        option type 'bridge'
        list ports 'eth0.2'

config device
        option name 'br-vlan3'
        option type 'bridge'
        list ports 'eth0.3'

config device
        option name 'br-vlan4'
        option type 'bridge'
        list ports 'eth0.4'

And finally we can make the new network interfaces for them:

config interface 'vlan2'
        option device 'br-vlan2'
        option proto 'static'
        option ipaddr '192.168.2.1'
        option netmask '255.255.255.0'

config interface 'vlan3'
        option device 'br-vlan3'
        option proto 'static'
        option ipaddr '192.168.3.1'
        option netmask '255.255.255.0'

config interface 'vlan4'
        option device 'br-vlan4'
        option proto 'static'
        option ipaddr '192.168.4.1'
        option netmask '255.255.255.0'

You can optionally enable DHCP servers for each network (I think you said you don't want them). You'll need to put these networks into a firewall zone -- there are several potential approaches, but it sounds like you might want to simply put each into its own zone (you can actually put them into a single zone and set the zone forward rule to REJECT -- that has the same effect on the inter-vlan allowances, but the choice will also depend on your routing plan for upstream/internet access). For you initial verification that the networks are functioning, you might just put them all in the lan firewall zone, and then separate them once you know the fundamentals work.

With these changes, each port will have its own subnet. Reboot after you make the changes and then test.