Speed drops to 100mbit accross VLANs

It's not exactly clear what you mean: You've set up the router ports as untagged and use unmanaged switches for the untagged traffic? Then it's fine.
Using tagged VLANs over unmanaged switches is not recommended (it works with some switches, doesn't with others).

This traffic has to reach the CPU of the router and needs to be handled in software. Since most routers (didn't check on yours) only have a single connection between the internal switch and the CPU, your traffic needs to pass this interface twice, halving the available bandwidth (so a theoretical 500M). The CPU needs to handle this routing speed, which I doubt it does. You can check the CPU usage of the router while you perform such a speed test - it's probably maxed out.

See here for a similar discussion with other hardware.