I just want to update this, as the true solution may be relevant to others.
The delay/latency problems returned and I had to investigate further.
I found an other mail describing the same problem and the solution:
In a vlan 4 extra bytes are added to the frame header. On consumer routers like the R7800 this can cause problems if the total link layer frame length goes from 1518 to 1522 bytes.
Solution is to reduce the MTU size in the interface definition(advanced settings) from the default 1500 to 1496 to compensate.
I now have a stable vlan based segmentation of IOT, Guest and LAN traffic!
The related post:
The ticket with more details on MTU size impact:
https://dev.archive.openwrt.org/ticket/18590#comment:1
Kind regards,
Bert