Main router: raspberry pi4b; 192.168.10.1, wifi is disabled.
2nd router: RT-AC68U running merlin firmware192.168.10.2, connect to main router by cable;
3rd router: 192.168.10.3 running openwrt firmware, connect to 2nd router by cable;
4th router (Netgear wifi externder ): 192.168.10.4 running official firmware, connect to 2nd router by cable.
2nd to 4th routers have wifi enabled for wireless devices, they are acting as wired and wireless AP. All devices connected to them are in the same subnet.
Now I want to enable guest wifi on 2nd and 4th router. There is option to add guest wifi on merlin firmware and Netgear firmware, see snapshot for setup on each router:
Probably won't work the way you want unless you install OpenWrt or a firmware that gives you direct control over the VLANs and SSIDs. The guest networks on those devices is does not give you the ability to associate it with a VLAN of your choosing... instead, it creates its own (isolated) subnet internally and connects it to an SSID for guests.
If you enable the guest wifi via the firmware on those devices, they will not be related to the guest network you've setup via your main router and your other OpenWrt device. Instead, you'll basically have 3 independent guest networks (which will not be a good experience for guests).
It appears that you can install OpenWrt on the RT-AC68U and this would make it pretty straightforward on that device.
The netgear device may not be so simple... what is the model?
By far, the easiest option here is to setup 2 independent networks on your main router (lan and guest) and then setup a guest wifi and a normal wifi using VLANs on the APs. It seems to me that this should be simple to do with the first 3 devices (provided that you install OpenWrt on the AC68U).
You can then either
use the Netgear device for just one of the networks (likely your main lan)
replace the Netgear device with something that can either run OpenWrt or gives you specific control over the VLANs and SSIDs.
hmm, Netgear is my only Wifi6 device, so I need for both networks. Also the openwrt wiki says wifi does not work on RT-AC68U. That means I need to replace two of them for VLAN approach. I already have a list of the known mac addresses (about 40 devices), I am thinking to try 2nd thought first before spending money (budget is tight)
If you run one network (just a regular lan), you will not be able to isolate devices with a MAC address based system. You might be able to restrict the internet access, but that is very different than isolating the guests so that they cannot reach your trusted lan devices.
Nope, that is not possible. Unless, of course, you use enterprise methods (802.1x and WPA2-Enterprise). I would highly doubt that your Asus and Netgear devices support this. But even if they do, it is seriously overkill.
If it supports VLANs (I don't know if it does or not), you need to set it up such that it can link a VLAN with an SSID, but that's all it should do. It should not perform any routing or DHCP... just what is known as a dumb AP (bridge).