Apple Home, VLAN and isolated WLAN

I’m using all my IoT devices inside the 2.4ghz WLAN but I discovered that Apple HomeKit devices work very slow if you isolated the clients, and if I put the clients in a VLAN I need to forward some ports (read this https://medium.com/@gepeto42/using-homekit-devices-across-vlans-and-subnets-aa5ae1024939) at this point the privacy is gone.

There’s another way or a workaround to use the HomeKit devices inside an isolated VLAN/WLAN? Im trying to think about it but I can’t find a solution. Any suggestion?

Thanks!