Apple Home, VLAN and isolated WLAN

I’m using all my IoT devices inside the 2.4ghz WLAN but I discovered that Apple HomeKit devices work very slow if you isolated the clients, and if I put the clients in a VLAN I need to forward some ports (read this at this point the privacy is gone.

There’s another way or a workaround to use the HomeKit devices inside an isolated VLAN/WLAN? Im trying to think about it but I can’t find a solution. Any suggestion?