I have two subnets, a normal one for internal use (192.168.1.0/24) and a dmz with a webserver on it at 192.168.2.0/24. I was ssh'ing to the webserver from inside my network and it's so slow it's unuseable. I move the webserver back into the internal subnet (both wireless) and it's fine, really responsive. From this I take it that the problem is in the OpenWRT box networking setup. Only problem is, I've no real idea about how to start debugging this, except for turning things off until I find the problem. If anyone has any suggestions, please go for it.
Between the subnets requires routing. Perhaps your routers cpu is insufficient. Or perhaps you have an MTU issue.