Guessing a bit here, but perhaps the responses from the server are being routed over the WAN or VPN rather than going locally?
For example, using VPN Policy-Based Routing I append ! -d 192.168.0.0/16 so that when the source is one of my local subnets, the traffic stays local.
Apologies if this is more 'wild goose' than 'golden goose', but maybe it'll help.