I've been trying to segment my home network into VLANs as a fun learning project. (It hasn't really been fun though )
This is my topology overview:
Bare-metal hypervisor (ESXi) with two physical network interfaces. WAN (from cable modem) is on vmx0, and LAN on vmx1.
pfSense virtual machine is the router for my entire network, using the above physical interfaces.
From the vmx1 port goes to my Mikrotik hAP AC with OpenWrt 19.07.6. It is configured as a dummy AP.
From the AP/switch, I have wired connection to two computers and wireless to my phone and smart home devices.
I successfully set up a VLAN 24 for my smart devices and a management VM. Next, I wanted to cut over all my "normal" devices to their own VLAN 103. I do this by enabling the VLAN 103 interface on pfSense, and configure the appropriate DHCP settings and firewall rules. Additionally, I disable the DHCP server on the default LAN interface and setting the interface IP to a /32.
When I do that, none of my wired devices can get DHCP from pfSense. But my phone successfully connects and is in the new subnet, with Internet access and all. Since my phone works, I think there might be some misconfiguration on the switch.
Reverting my changes on pfSense restores my network connectivity for wired devices, but now my phone is the one that does not get DHCP.
Currently, this is how my network is configured:
Smart home devices + management VM on 192.168.24.0/24 - gateway 192.168.24.5
All other devices on 192.168.1.0/24 - gateway 192.168.1.5
The LAN port group in ESXi is configured with VLAN ID 4095, which is supposed to trunk it
Side note, the ESXi server is configured with static IP on 192.168.1.0/24 and gateway 192.168.1.5. I was thinking the hard coded gateway might have been an issue, but probably not since my phone was working fine with the new subnet.
Here are a couple screenshots of my OpenWrt configuration. If anyone has ideas, it would be really appreciated.
Thanks for your response. The wired hosts have NICs capable of understanding 802.11Q tagging. My intention is to have all wired hosts be part of VLAN 103. I kept VLAN 1 active in the config above since I assumed it was necessary in order to access the management interface of the switch (this is one part I'm not sure about). I should also clarify that "LAN 1" in the config is supposed to be the trunk port going to the vmx1 interface on the hypervisor.
VLAN ID 4095 holds special meaning for ESXi. When I configure a port group with this tag, it means that port group is a trunk.
Perhaps not. I meant to say that I did this in the pfSense configuration. Here my intention was to also leave a management interface accessible, so I wanted to keep the pfSense LAN interface with IP 192.168.1.5, without allowing other IPs on this subnet (hence the /32).
I assume that I also have to keep the pfSense LAN interface here in order to manage it. Otherwise if I could somehow "convert" the LAN interface to VLAN 103, that is essentially my goal. I feel like I was very close to getting it working correctly. How could it be that my phone was able to get DHCP on the new subnet but none of the wired devices? Should I also be bridging the VLAN1 interface in OpenWrt to the VLAN103? Perhaps my tagging configuration is incorrect.
I assume ESXI and pfSense are working properly since that's out of scope of this forum. You could monitor your trunk cable with a laptop running Wireshark to see that properly tagged packets are present.
The switch in an OpenWrt router is managed out of band (mdio bus) so any way to log into OpenWrt can manage the switch. It isn't necessary to leave a VLAN 1 present in the switch etc. On a device that has WiFi I recommend temporarily setting up a WiFi management network so you don't lose access if the Ethernet is misconfigured.
If a wired PC etc is to be on one VLAN, make it an access port: the port untagged in that VLAN and off in all the others. Sending tagged traffic of a single VLAN is not necessary. Sending multiple VLANs to an endpoint device and expecting it to ignore those that don't apply to that device is a security risk.
Some switch hardware only supports 16 or 128 VLANs directly and special configuration is needed to have VLAN numbers higher than that.
If there is no specific need, use smaller vlan ids
Then there is no point in assigning an IP.
Other than that, OpenWrt can be managed by any IP in any interface. For your ease, assign the vlan subinterface name in lan. This way the firewall zone settings will be correct. Also remove the vlan1 if you don't use it, after you have verified that you can connect from another interface.