Router Cascade Port Forwarding Issue

egc · February 4, 2024, 1:31pm

Could it be that your main router only NAT's its own subnet out via the WAN and not the subnet of lan 2?

I have some broadcom routers which runs DDWRT which operate like this

apo440 · February 4, 2024, 2:14pm

@egc Don't think so. It used to work with another openWRT box, and there's plenty people running a very similar setup with openWRT boxes behind FritzBoxes.

For that ICMP packet filtered error, it must be some setting somewhere. Might as well be on the FritzBox, though I have not changed anything there. Still looking around, read something on MTU sizes which might be related, not clear to me yet.

psherman · February 4, 2024, 3:41pm

Delete everything below the ipaddr line:

apo440:

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ip6assign '60'
        list ipaddr '10.0.10.1/24'
        option gateway '10.0.0.1'
        list dns '10.0.0.1'
        list dns_search 'fritz.box'

And delete all of these rules:

apo440:

config rule
        option name 'Allow_Web+SSH_WAN-to-LAN-IP'
        list proto 'tcp'
        option src 'wan'
        option dest 'lan'
        list dest_ip '10.0.10.1'
        option dest_port '22 80 443'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow_WebIF_on-WAN-IP'
        list proto 'tcp'
        option src 'wan'
        option dest_port '22 80 443'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow_VM-105_pihole'
        option src 'wan'
        option dest 'lan'
        option dest_port '22 53 80 443 '
        option target 'ACCEPT'
        option family 'ipv4'
        list dest_ip '10.0.10.105'
        list proto 'tcp'

config rule
        option name 'Allow_Ping_to_LAN'
        option family 'ipv4'
        list proto 'icmp'
        option src 'wan'
        option dest 'lan'
        option target 'ACCEPT'

Then reboot and test again.

apo440 · February 4, 2024, 4:16pm

Done. No change.

Wouldn't have understood what the rules punching holes to dedicated ports on the the openWRT box or boxes behind it should have broken, but anyhow: Deleted, reboot done, no change.

(For clarity: in /etc/config/network, I've only deleted the lines belonging to interface 'lan' below the 'list ipaddr' line, but not the lines following thereafter, which are for 'wan' etc. - trust we do need all those other interfaces )

psherman · February 4, 2024, 4:18pm

Do you have a static route configured on the main router?

If so, please show that to us.

If not, re-enable nat masquerading on the wan zone.

apo440 · February 4, 2024, 4:29pm

You mean from the FritzBox Main Router towards the 10.0.10.x/24 subnet, via the openWRT WAN IP 10.0.0.2? Sure. Nothing ever changed there.

FritzBox_static_route

Been in place for ages.

I can capture traffic on the FritzBox main router, and there I can see that the ping packages to 8.8.8.8 are getting to the FritzBox, and obviously (?) the responses ("packet filtered") are getting back from there (is that error thrown by the FritzBox, or by the next hop, i.e. the inbound router of my ISP?) to the host on the .10.x subnet where the ping originated - otherwise, those results/errors wouldn't be visible there, but we'd only see a time out (ie total packet loss) - right?

psherman · February 4, 2024, 4:30pm

That looks fine. Let’s double check the lan config on the main router - can you show that to us?

apo440 · February 4, 2024, 4:36pm

Not much we can set there, but here you go. There's no text based interface, as there's no shell access to these boxes unfortunately, all GUI only

psherman · February 4, 2024, 4:38pm

Looks fine. Wanted to verify that we were looking at a /24 for each network. All good.

Is there any chance that there is another device using the same 10.0.0.2 address as your openwrt router? An address conflict would certainly account for these problems.

apo440 · February 4, 2024, 4:38pm

Oh, there's one thing a little odd (maybe).
If I run ping 8.8.8.8 from a host behind openWRT, I get the packet filtered response.
If I run same ON the openWRT box itself, ping just sits there and does nothing until I give it a Ctrl+c.
May be due to different versions of ping, or maybe relevant?

apo440 · February 4, 2024, 5:26pm

None. Been checking various things (took me a while and several reboots to make tripple-sure), on both routers, no, there isn't. This IP was assigned to that WAN interface of that openWRT router for ages, isn't part of any DHCP range, nothing. To be sure, I now purged by rebooting anything the FritzBox may have in memory AND changed the MAC address for the openWRT WAN port to a different XX:YY in the end which was never ever seen in the house before. Checked that this is now known also the only thing the Fritzbox known about Mr. openWRT.

psherman · February 4, 2024, 5:29pm

And I'm assuming it is still not working, right?

Does it start to work if you re-enable masquerading on the wan zone (on OpenWrt)?

apo440 · February 4, 2024, 5:43pm

Yes, it does (and did) not work with all above, I'd jump and shout immediately if it ever does, promise

And YES, once I re-activate MASQ on WAN I can ping 8.8.8.8 now. Not sure why that made no difference earlier today, not sure what was different then.

Yet, it should (or at least: was at some point in time) be doable without MASQ, there's plenty threads all around where people say "don't do double-NAT, just do proper subnet routing".

I have an IT magazine (you know, stuff on paper) in front of me where those guys took a "fritzbox 4040", flashed it with openWRT, did the entire two-networks-setup and routing-entries, AND explicitly turned of "MASQ" and "MSS clamping" - and got it to work, no mention of any problems there.

Found another hint somewhere that MTU size might need to be reduced to as little as 1472 for a router cascade for cases where MTU size discovery along the path (or what the right terms are) isn't working. Tried that earlier, reduced the MTU size on all openWRT interfaces to 1472, but didn't help. Left it there, nevertheless. If I understood correctly that might eventually result in performance degradation, but hey, getting out there slow would be a starting point

psherman · February 4, 2024, 5:57pm

Would you be willing to try resetting the device to defaults again. Then make only two changes:

lan IP address/subnet
-- after changing, test to make sure connectivity works (it should)
disable wan masquerading
-- and then test this (don't know if this will work or not -- it theoretically should, but this is the purpose of the test).

This will reduce all the variables to those two, plus the config on the FB main router. If things break after wan masquerading is disabled, we should look more about what could be happening on the FB.

mk24 · February 4, 2024, 6:00pm

Also in OpenWrt, must add allow wan->lan forward in the firewall (lan->wan is confugred by default, leave that in place and also add wna->lan). This will alow free connections between the networks, all Ps and ports. Later once that works you can go back to writing rules only for specific servers.

Masquerading in the second router means that a connection from a 10.0.10.X machine will be source-NATd to appear to come from 10.0.0.2. Then the Fritzbox answers it as if it is on its default LAN. If masquerade is turned off, the packet will retain its 10.0.10.X IP, and the Fritzbox needs the static route to know how to route a reply back to the other network.

psherman · February 4, 2024, 6:03pm

But to be clear, wan > lan forwarding is not required for general connectivity from behind the OpenWrt router. It won't hurt to add it, but the OpenWrt lan hosts should be able to reach the internet even if wan > lan forwarding is not enabled.

So, I'd recommend not touching the firewall except to disable masquerading so that we keep all other variables constant.

apo440 · February 4, 2024, 8:27pm

Guys, I think (!) I've found the problem. Its a FritzBox-Thing I think now.

Let me explain.

FritzBoxes are nice "home" devices designed with family live in mind. Therefore they provide some "child protection / limitation" ("Kindersicherung") features: Devices can be assigned "access profiles" ("Zugangsprofile") for which the online time can be limited (by time of day, by total online time per 24 hrs, different per weekday, summing up devices in a group for each user etc etc). Quite flexible. And to ensure that no "new" devices (new or changed MAC) slips through, the "default" profile (which gets assigned automatically to every new device/MAC the box sees on her network) can be set to "NO ACCESS".

Now, with that, when openWRT does NAT, and the FritzBox sees openWRT on HER network (openWRT WAN IP & MAC), and I have assigned a "good to go"-profile, then openWRT and all boxes behind it are granted internet access.

However, when openWRT does NOT do NAT, then every box behind it shows up with its own MAC&IP at the FritzBox - and they are all "new" and get restrictions applied.
/Edit: confusing thing is that the FritzBox "rejects" (filters) those packets, but on the reply it sends back, it gives its own EXTERNAL IP address. Would have been more helpful if the response came from its INTERNAL IP to rule out any outside-world-issues. Edit/

Should be simple to solve, but I think I found a bug/incomplete feature for all FritzBoxes today. Those devices from the openWRT-subnet (LAN2) are NOT showing up on the FritzBox for administration of their access profile, i.e. I can't tell that damn FritzBox that they are "good kids" and may browse as much as they want.

Two options:
(a) change the blocking approach on Fritzbox to never restrict "new" or "unknown" devices - wait, no, can't do that, because of the kids (they already learned what MAC addresses are).
(b) live with NAT on openWRT until I manage to convince the makers of FritzBox that there's something to be fixed on their end. Read: "likely never"?

SO, I guess it has to be NAT then, for a while.
I'll work on setting it up further with NAT enabled, and will eventually have to get back to understanding how I can make those outside-world requests go through to the machines behind the openWRT. But that's not gonna happen in the next 3 days or so. Will have to roll back to the "semi-broken but mostly working" config for now.

psherman · February 4, 2024, 8:42pm

Actually, the problem with your theory is here:

MAC addresses are a layer 2 concept, and will not pass through a router which operates at L3. What your Fritzbox is seeing is IP addresses on a network that it doesn't control -- it cannot see the MAC addresses of the devices on the OpenWrt network.

But yes, in theory, the rest of the general idea does hold insofar as the FritzBox device is indeed restricting that traffic in some way.

I'd bring up this issue/question on other sites that deal with the FritzBox devices and their FritzOS to find out if there is a way you can set the appropriate profile for the entire subnet that sits behind OpenWrt.

Meanwhile, that's why I asked you to run a default config and turn masquerading on and off... when it failed to function with masquerading off, it pointed squarely at the Fritz device as the issue.

Although not entirely solved, your problem is now identified to be outside of the realm of OpenWrt. With that in mind...

If your problem is solved, please consider marking this topic as [Solved]. See How to mark a topic as [Solved] for a short how-to.
Thanks!

apo440 · February 4, 2024, 9:16pm

L2 / L3: True, forgot about that.

Issue solved - not sure yet. In the first place, it didn't work with NAT enabled and I'm not there yet to rebuild that setup. Once I have, and got it to work (or not), I'll be back and report - and close, if solved, or hope for more of your great support - Big Thanks for that!