aMule behind 2 NATs not working

laurentppol · April 23, 2021, 11:51am

Hi there,
I want to make my a(e)Mule to work behind 2 NATs (behind one it works with no problem):

working case:
: laptop 192.168.1.200, router 192.168.1.1 (TP-Link ADSL router with factory firmware),
redirect: 4662-4672 192.168.1.200 4662-4672 TCP or UDP Enabled pppoa_0_35_2_d
default route 192.168.1.1.
WORKS.

Now I am trying to make it work behind SECOND NAT (TP-Link MR-3220 OpenWRT 14.07, 192.168.1.60, "central router" (with backup MWAN3 configured and working), I know I should update to 1043ND -> 18.09 at least).
Default route 192.168.1.60 (on PC).
I change a redirect rule on ADSL router, to be:
4662-4672 192.168.1.60 4662-4672 TCP or UDP Enabled pppoa_0_35_2_d

On MR3220 I use forwarding rule:

aMule_TCP IPv4-TCP
From IP any in lan
Via any router IP at port 4662 IP 192.168.1.200, port 4662 in lan
Also enabled NAT in rc.local
: iptables -t nat -A POSTROUTING -o br-lan ! -d 192.168.0.0/22 -j MASQUERADE

And this does NOT work, aMule shows "behind firewall" status.

What am I doing wrong and how to correct it?

Laurent

trendy · April 23, 2021, 12:07pm

Better make it 19.07

This doesn't sound like the correct way. I don't see the wan and why are you masquerading the br-lan?

laurentppol · April 23, 2021, 1:21pm

To have ADSL router "see" only 3220 address.

This was for an ADSL router w/o static routes (previous. this one has).
My config is:

ADSL -> Eth routing + NAT (TP-LINK ADSL),
WiFi / LAN / 3G router "main" (TP-LINK 3220).

So packets are from PC -> 3220 -> ADSL (if ADSL is up, if not to WiFi / 3G backups).
Opposite direction: WAN -> ADSL (redir) -> 3220 (redir) -> laptop.

"Double NAT" WORKS for accessing WWW servers from WAN side. Only aMule doesn't :(.

trendy · April 23, 2021, 1:36pm

Better run the following commands (copy-paste the whole block) and paste the output here, using the "Preformatted text </> " button:

Remember to redact passwords, MAC addresses and any public IP addresses you may have

uci export network;  uci export firewall; \
head -n -0 /etc/firewall.user; \
ip -4 addr ; ip -4 ro li tab all ; ip -4 ru

laurentppol · April 23, 2021, 3:17pm

Ok:

root@Laurent_Home_OpenWrt_3220_phone_gateway:~# uci export network
package network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdb9:e615:11e4::/48'

config interface 'lan'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option netmask '255.255.255.0'
        option delegate '0'
        option ipaddr '192.168.1.60'
        option gateway '192.168.1.1'
        option aliases '192.168.2.60/24'
        option dns '194.204.152.34 194.204.159.1'
        option _orig_ifname 'eth0 wlan0'
        option _orig_bridge 'true'
        option ifname 'eth0 eth1'
        option metric '10'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 4'

config interface 'wwan_phone'
        option force_link '1'
        option ifname 'wlan0'
        option _orig_ifname 'radio0.network2'
        option _orig_bridge 'false'
        option proto 'static'
        option ipaddr '192.168.43.60'
        option netmask '255.255.255.0'
        option gateway '192.168.43.1'
        option dns '194.204.159.1 8.8.8.8'
        option delegate '0'
        option metric '20'

config route
        option interface 'lan'
        option target '192.168.10.0'
        option netmask '255.255.255.0'
        option gateway '192.168.1.30'
        option metric '10'

config route
        option interface 'lan'
        option target '192.168.20.0'
        option netmask '255.255.255.0'
        option metric '10'
        option gateway '192.168.1.70'

config interface 'wwan_Aero2'
        option proto '3g'
        option device '/dev/ttyUSB0'
        option apn 'internet'
        option delegate '0'
        option peerdns '0'
        option keepalive '10 5'
        option metric '30'
        option service 'umts'

config route
        option interface 'lan'
        option target '192.168.15.0'
        option netmask '255.255.255.0'
        option gateway '192.168.1.110'
        option metric '10'

package firewall

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option drop_invalid '1'
        option forward 'ACCEPT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan wwan'
        option masq '1'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan wan6 wwan_phone'
        option forward 'ACCEPT'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fe80::/10'
        option src_port '547'
        option dest_ip 'fe80::/10'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config include
        option path '/etc/firewall.user'

config zone
        option input 'ACCEPT'
        option forward 'ACCEPT'
        option output 'ACCEPT'
        option name 'IoT_to_lan'
        option network 'IoT'
        option family 'ipv4'

config forwarding
        option dest 'lan'
        option src 'IoT_to_lan'

config forwarding
        option dest 'IoT_to_lan'
        option src 'lan'

config zone
        option name 'wwan_Aero2'
        option input 'ACCEPT'
        option forward 'ACCEPT'
        option output 'ACCEPT'
        option network 'wwan_Aero2'

config redirect
        option target 'DNAT'
        option dest 'lan'
        option src_dport '4662'
        option dest_ip '192.168.1.200'
        option dest_port '4662'
        option name 'aMule_TCP'
        option proto 'tcp'
        option src 'lan'
        option src_ip 'any'
        option reflection '0'

config redirect
        option target 'DNAT'
        option dest 'lan'
        option src_dport '8080'
        option dest_ip '192.168.1.200'
        option dest_port '80'
        option name 'laptop_WWW_over_1.60'
        option proto 'tcp'
        option src 'lan'
        option src_ip 'any'
        option reflection '0'

config redirect
        option target 'DNAT'
        option dest 'lan'
        option src_dport '666'
        option dest_ip '192.168.1.201'
        option dest_port '80'
        option name 'NodeMCU_WWW_over_1.60'
        option proto 'tcp'
        option src 'lan'
        option src_ip 'any'
        option reflection '0'

config redirect
        option target 'DNAT'
        option dest 'lan'
        option src_dport '8444'
        option dest_ip '192.168.1.110'
        option dest_port '8080'
        option name 'OpenHAB2_noSSL_OPI3_over_1.60'
        option proto 'tcp'
        option src 'lan'
        option src_ip 'any'
        option reflection '0'
        option enabled '0'

root@Laurent_Home_OpenWrt_3220_phone_gateway:~# head -n -0 /etc/firewall.user
# This file is interpreted as shell script.
# Put your custom iptables rules here, they will
# be executed with each firewall (re-)start.

# Internal uci firewall chains are flushed and recreated on reload, so
# put custom rules into the root chains e.g. INPUT or FORWARD or into the
# special user chains, e.g. input_wan_rule or postrouting_lan_rule.

# iptables -t nat -A zone_lan_postrouting -j MASQUERADE

root@Laurent_Home_OpenWrt_3220_phone_gateway:~# ip -4 addr ; ip -4 ro li tab all ; ip -4 ru
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
5: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    inet 192.168.1.60/24 brd 192.168.1.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet 192.168.2.60/24 brd 192.168.2.255 scope global br-lan:1
       valid_lft forever preferred_lft forever
10: wlan0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
    inet 192.168.43.60/24 brd 192.168.43.255 scope global wlan0
       valid_lft forever preferred_lft forever
14: 3g-wwan_Aero2: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 3
    inet 100.82.84.98 peer 10.64.64.64/32 scope global 3g-wwan_Aero2
       valid_lft forever preferred_lft forever
default via 192.168.1.1 dev br-lan  table 1 
default via 10.64.64.64 dev 3g-wwan_Aero2  table 3 
default via 192.168.1.1 dev br-lan  proto static  metric 10 
default via 192.168.43.1 dev wlan0  proto static  metric 20 
default via 10.64.64.64 dev 3g-wwan_Aero2  proto static  metric 30 
10.64.64.64 dev 3g-wwan_Aero2  proto kernel  scope link  src 100.82.84.98 
192.168.1.0/24 dev br-lan  proto static  scope link  metric 10 
192.168.2.0/24 dev br-lan  proto kernel  scope link  src 192.168.2.60 
192.168.10.0/24 via 192.168.1.30 dev br-lan  proto static  metric 10 
192.168.15.0/24 via 192.168.1.110 dev br-lan  proto static  metric 10 
192.168.20.0/24 via 192.168.1.70 dev br-lan  proto static  metric 10 
192.168.43.0/24 dev wlan0  proto static  scope link  metric 20 
local 100.82.84.98 dev 3g-wwan_Aero2  table local  proto kernel  scope host  src 100.82.84.98 
broadcast 127.0.0.0 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
local 127.0.0.0/8 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
local 127.0.0.1 dev lo  table local  proto kernel  scope host  src 127.0.0.1 
broadcast 127.255.255.255 dev lo  table local  proto kernel  scope link  src 127.0.0.1 
broadcast 192.168.1.0 dev br-lan  table local  proto kernel  scope link  src 192.168.1.60 
local 192.168.1.60 dev br-lan  table local  proto kernel  scope host  src 192.168.1.60 
broadcast 192.168.1.255 dev br-lan  table local  proto kernel  scope link  src 192.168.1.60 
broadcast 192.168.2.0 dev br-lan  table local  proto kernel  scope link  src 192.168.2.60 
local 192.168.2.60 dev br-lan  table local  proto kernel  scope host  src 192.168.2.60 
broadcast 192.168.2.255 dev br-lan  table local  proto kernel  scope link  src 192.168.2.60 
broadcast 192.168.43.0 dev wlan0  table local  proto kernel  scope link  src 192.168.43.60 
local 192.168.43.60 dev wlan0  table local  proto kernel  scope host  src 192.168.43.60 
broadcast 192.168.43.255 dev wlan0  table local  proto kernel  scope link  src 192.168.43.60 
0:      from all lookup local 
1001:   from all iif br-lan lookup main 
1003:   from all iif 3g-wwan_Aero2 lookup main 
2001:   from all fwmark 0x100/0xff00 lookup 1 
2003:   from all fwmark 0x300/0xff00 lookup 3 
2253:   from all fwmark 0xfd00/0xff00 blackhole
2254:   from all fwmark 0xfe00/0xff00 unreachable
32766:  from all lookup main 
32767:  from all lookup default

Here it is. I do direct files edit (/etc/config) or LuCI (if I don't know syntax, ex for MWAN3).

trendy · April 23, 2021, 4:31pm

This is rather messy setup and in addition to the old code it's not worthy to troubleshoot it.
Upgrade to latest supported code first, distinguish the lan from the wan, setup mwan3 or vpn-policy-routing for the source routing, and then you'll have more chances to have it fixed.

laurentppol · April 23, 2021, 5:36pm

I will upgrade it, but need to migrate (MWAN3 + 3G link) to TP-LINK 1043ND (8/32) to have 18.06.
Current device is MR3220, an 4/32 device, has not enough flash to support 18.06.

You are right, it is rather complicated setup, it involves 2 routing stages and (not shown for clearity) 2 bridges, a WiFi links... Main link is trough ADSL router, and there are 2 backup links, one over WiFi (to a LTE/3G Android phone) as client, 2nd using 3G modem ("last ressort").

It is working for WWW serving exposed to Internet (only if ADSL works, as 3G are not public IPs), 2 servers under 2 port numbers.

laurentppol · June 5, 2021, 11:59pm

Ok, (almost) migrated:

flashed 1043 with 18.06 "factory" firmware, keeping settings,
copied /etc/config contents,
adjusted /etc/config/network for switch config:

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '0'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 4 5'

adjusted /etc/config/wireless

        option path 'platform/ath9k'

Now routing works as expected, ONLY in status page I read:
"Status: Interface DISABLED"... for all 3 defined (for mwan3) interfaces.

What am I missing?

trendy · June 6, 2021, 12:44am

Have you got different metrics on all wan interfaces?

laurentppol · June 6, 2021, 12:00pm

Yes, should I include /etc/config/mwan3 and /etc/config/network ?

trendy · June 6, 2021, 7:13pm

Yes, let's see them all

uci export network; uci export mwan3; mwan3 status

laurentppol · June 7, 2021, 3:50am

root@Laurent_Home_OpenWrt_1043ND_retired:~# uci export network
package network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fdb9:e615:11e4::/48'

config interface 'lan'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option netmask '255.255.255.0'
        option delegate '0'
        option ipaddr '192.168.1.70'
        option gateway '192.168.1.1'
        option aliases '192.168.2.70/24'
        option dns '194.204.152.34 194.204.159.1'
        option _orig_bridge 'true'
        option ifname 'eth0'
        option metric '10'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '0'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '0 1 2 3 4 5'

config interface 'wwan_phone'
        option force_link '1'
        option ifname 'wlan0'
        option _orig_ifname 'radio0.network2'
        option _orig_bridge 'false'
        option proto 'static'
        option ipaddr '192.168.43.70'
        option netmask '255.255.255.0'
        option gateway '192.168.43.1'
        option dns '194.204.159.1 8.8.8.8'
        option delegate '0'
        option metric '20'

config route
        option interface 'lan'
        option target '192.168.10.0'
        option netmask '255.255.255.0'
        option gateway '192.168.1.30'
        option metric '10'

config interface 'wwan_Aero2'
        option proto '3g'
        option device '/dev/ttyUSB0'
        option apn 'internet'
        option delegate '0'
        option peerdns '0'
        option keepalive '10 5'
        option metric '30'
        option ipv6 'auto'
        option service 'umts_only'

config route
        option interface 'lan'
        option target '192.168.15.0'
        option netmask '255.255.255.0'
        option gateway '192.168.1.110'
        option metric '10'

root@Laurent_Home_OpenWrt_1043ND_retired:~# uci export mwan3
package mwan3

config rule 'no_mule_wan_rule_tcp'
        option dest_port '4662,4672'
        option proto 'tcp'
        option use_policy 'lan_only'

config rule 'no_mule_wan_rule_udp'
        option dest_port '4662,4672'
        option proto 'udp'
        option use_policy 'lan_only'

config rule 'no_yt_wan_rule_tcp'
        option dest_ip '172.217.18.110/24'
        option proto 'all'
        option use_policy 'lan_only'

config rule 'no_radio_wwan2'
        option dest_ip '217.74.72.4'
        option proto 'all'
        option use_policy 'lan_wan'

config rule 'default_rule'
        option dest_ip '0.0.0.0/0'
        option proto 'all'
        option use_policy 'lan_wan_wan2'

config member 'lan_m1_w3'
        option interface 'lan'
        option metric '1'
        option weight '3'

config member 'lan_m2_w3'
        option interface 'lan'
        option metric '2'
        option weight '3'

config member 'wwan_phone_m1_w2'
        option interface 'wwan_phone'
        option metric '1'
        option weight '2'

config member 'wwan_phone_m2_w2'
        option interface 'wwan_phone'
        option metric '2'
        option weight '2'

config policy 'lan_only'
        list use_member 'lan_m1_w3'

config policy 'wan_only'
        list use_member 'wwan_phone_m1_w2'

config policy 'balanced'
        list use_member 'lan_m1_w3'
        list use_member 'wwan_phone_m1_w2'

config policy 'lan_wan'
        list use_member 'lan_m1_w3'
        list use_member 'wwan_phone_m2_w2'

config policy 'wan_lan'
        list use_member 'lan_m2_w3'
        list use_member 'wwan_phone_m1_w2'

config interface 'lan'
        option enabled '1'
        option reliability '2'
        option count '1'
        option timeout '2'
        option down '3'
        option up '3'
        list track_ip '194.204.159.1'
        list track_ip '194.204.152.34'
        list track_ip '8.8.8.8'
        option interval '1'

config interface 'wwan_phone'
        option enabled '1'
        list track_ip '8.8.8.8'
        list track_ip '208.67.220.220'
        option reliability '1'
        option count '2'
        option timeout '5'
        option interval '5'
        option down '3'
        option up '3'

config interface 'wwan_Aero2'
        option enabled '1'
        list track_ip '8.8.8.8'
        list track_ip '194.204.159.1'
        option reliability '1'
        option down '3'
        option timeout '5'
        option count '2'
        option interval '30'
        option up '3'

config member 'wwan_Aero2_m1_w2'
        option interface 'wwan_Aero2'
        option metric '1'
        option weight '2'

config member 'wwan_Aero2_m2_w2'
        option interface 'wwan_Aero2'
        option metric '2'
        option weight '2'

config policy 'wan2_only'
        list use_member 'wwan_Aero2_m1_w2'
        option last_resort 'unreachable'

config policy 'lan_wan_wan2'
        list use_member 'lan_m1_w3'
        list use_member 'wwan_phone_m1_w2'
        list use_member 'wwan_Aero2_m1_w2'
        option last_resort 'unreachable'

root@Laurent_Home_OpenWrt_1043ND_retired:~# mwan3 status
**/sbin/uci: Invalid argument**
Interface status:
 interface lan is offline and tracking is down
 interface wwan_phone is offline and tracking is down
 interface wwan_Aero2 is offline and tracking is down

Current ipv4 policies:

Current ipv6 policies:

Directly connected ipv4 networks:

Directly connected ipv6 networks:

Active ipv4 user rules:

Active ipv6 user rules:

Looks something changed in mwan3 config from 14.07 to 18.06 and this breaks things (I have simply copied /etc/config/mwan3 from old router).

root@Laurent_Home_OpenWrt_1043ND_retired:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    10     0        0 br-lan
0.0.0.0         192.168.43.1    0.0.0.0         UG    20     0        0 wlan0
0.0.0.0         10.64.64.64     0.0.0.0         UG    30     0        0 3g-wwan_Aero2
10.64.64.64     0.0.0.0         255.255.255.255 UH    0      0        0 3g-wwan_Aero2
192.168.1.0     0.0.0.0         255.255.255.0   U     10     0        0 br-lan
192.168.10.0    192.168.1.30    255.255.255.0   UG    10     0        0 br-lan
192.168.15.0    192.168.1.110   255.255.255.0   UG    10     0        0 br-lan
192.168.43.0    0.0.0.0         255.255.255.0   U     20     0        0 wlan0

trendy · June 7, 2021, 8:09am

Nope, don't do that when migrating between major releases.
I suggest to wipe the router clean and start configuring from scratch.

laurentppol · June 7, 2021, 10:14am

Could You post (here) contents of "factory fresh" mwan3 and network config files? (for 18.07). Will compare this line by line.

trendy · June 7, 2021, 3:22pm

It should be already in the config folder as mwan3-opkg

laurentppol · June 7, 2021, 4:48pm

There is only "mwan3" file :(.
Anyone help?

trendy · June 7, 2021, 5:11pm

Reinstall it and it will appear.

laurentppol · June 7, 2021, 7:04pm

Did another (less destructive) thing:

downloaded mwan3.ipk (as pointed in /etc/opkg/distfeeds.conf "packages" line),
changed mwan3 (from 14.07 MR3220),
added:

config globals 'globals'
        option local_source 'lan'

EDIT: oops, had to remove "option local_source 'lan'" as it made crazy default route (0.0.0.0) to router itself (192.168.1.60) by LO interface, which of course didn't work.

and

        option family 'ipv4'

in "interface"s sections.
Now it shows status correctly.

ALMOST done:
detailed statistics page shows:

Interface status:
 interface lan is online and tracking is active
 interface wwan_phone is offline and tracking is active
 interface wwan_Aero2 is online and tracking is active

Current ipv4 policies:
balanced:
 lan (100%)

lan_only:
 lan (100%)

lan_wan:
 lan (100%)

lan_wan_wan2:
 wwan_Aero2 (40%)
 lan (60%

And "lan_wan_wan2" should be as follows:

lan if available, 100%, [NO wwan_Aero2 40%!]
wan if not lan available, 100%,
wan2 if no lan and wan, 100%.
How to achieve this?

trendy · June 8, 2021, 8:04am

Reinstall is not destructive. If there is already a configuration file, it copies the package version into mwan3-opkg.

Use different metrics, currently they all have metric 1, so they are load balanced.

laurentppol · June 8, 2021, 10:21am

In (as You may see above) output of "route -n" the all 3 paths (def gateways) have different metrics (10, 20, 30). And packet counters show that (LAN gateway working) all traffic is going trough LAN, on 3g-wwan_Aero2 there is only (I suppose) PING trafic.

So what metric setting do You mean? In which part of config file?