Have I understood this correctly:
You want the traffic to have the same Source IP, once it hits the server on your inside VLAN, as the traffic did when it arrived on the WAN side of your OpenWrt box?
But the Source IP is being translated by OpenWrt? And when it leaves OpenWrt / arrives at the server, instead it has some IP address that OpenWrt is configured with in that VLAN?
Hmm.
If you go to the Firewall screen, and at the bottom find the relevant Zone (the one that contains the inside/server VLAN) on the further-most left-hand side... Does it have the "Masquerade" flag checked?
Because that would mean that you asked OpenWrt to apply masquerading to all traffic going into that VLAN.
Just going for the simple stuff first here, worth checking...
If that's the case, you can make your own masquerade-or-not rules here, called a NAT rule:
Choose your VLAN as the outbound zone and ACCEPT to disable masquerading.
LUCI allows you to do this for a single Destination IP but not for a single Destination Port.
It comes with some extra difficulty if you want to match up a NAT rule and a Port Forward rule.
To accomplish this in LUCI, I think it is necessary to also create a matching Traffic Rule:
In this Traffic Rule, of course configure the IP address and Port to match your other rules; then set the Action to "apply firewall mark" (and just choose 1234 or whatever for that mark).
Because then you can adjust the Advanced Settings in the NAT Rule:
And instead of matching on the Destination Port, you just match on that firewall mark (1234) instead.
Have not tried it, but I can't think of a simpler way to (1) Accept traffic on a specific WAN IP and Port, (2) Translate IP and/or Port the way you decide, and (3) Route or ARP that traffic into the relevant inside VLAN.
