[SOLVED] Ping external host on interface working, but curl does not

oppsig · May 8, 2019, 11:25am

When i ping from 192.168.1.2 (client br-lan) to 10.100.100.2 connected to wg1 interface on router, everything works fine.
But when i try to curl a service running on that same ip 10.100.100.2 on port 5000 the response does not get routed back to the client that asked for it.
Masquarade is on for the zone.

➜  ~ ping -c1 10.100.100.2 
PING 10.100.100.2 (10.100.100.2) 56(84) bytes of data.
64 bytes from 10.100.100.2: icmp_seq=1 ttl=63 time=35.6 ms
--- 10.100.100.2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 35.688/35.688/35.688/0.000 ms

➜  ~ curl 10.100.100.2:5000
curl: (7) Failed to connect to 10.100.100.2 port 5000: Connection refused

root@openwrt:~# tcpdump -i wg1
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on wg1, link-type RAW (Raw IP), capture size 262144 bytes
13:14:50.469236 IP 10.100.100.1 > 10.100.100.2: ICMP echo request, id 27326, seq 1, length 64
13:14:50.506108 IP 10.100.100.2 > 10.100.100.1: ICMP echo reply, id 27326, seq 1, length 64
13:14:51.783712 IP 10.100.100.1.43802 > 10.100.100.2.5000: Flags [S], seq 56625904, win 29200, options [mss 1460,sackOK,TS val 842812138 ecr 0,nop,wscale 7], length 0
13:14:51.818168 IP 10.100.100.2.5000 > 10.100.100.1.43802: Flags [R.], seq 0, ack 56625905, win 0, length 0

Here is a tcpdump on the client (192.168.1.2) with ping first then curl.

sudo tcpdump -vv -i eth1 -n host 10.100.100.2
tcpdump: listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
13:38:23.693982 IP (tos 0x0, ttl 64, id 15668, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.1.2 > 10.100.100.2: ICMP echo request, id 30901, seq 1, length 64
13:38:23.728750 IP (tos 0x0, ttl 63, id 48793, offset 0, flags [none], proto ICMP (1), length 84)
    10.100.100.2 > 192.168.1.2: ICMP echo reply, id 30901, seq 1, length 64
13:38:25.303043 IP (tos 0x0, ttl 64, id 41152, offset 0, flags [DF], proto TCP (6), length 60)
    192.168.1.2.44126 > 10.100.100.2.5000: Flags [S], cksum 0x04e7 (correct), seq 2543336090, win 29200, options [mss 1460,sackOK,TS val 844225677 ecr 0,nop,wscale 7], length 0
13:38:25.338708 IP (tos 0x0, ttl 63, id 0, offset 0, flags [DF], proto TCP (6), length 40)
    10.100.100.2.5000 > 192.168.1.2.44126: Flags [R.], cksum 0xeda5 (correct), seq 0, ack 2543336091, win 0, length 0

mikma · May 8, 2019, 12:00pm

The server at 10.100.100.2 on port 5000 seems to be rejecting the connection attempt since you receive a RST from 10.100.100.2:5000. You should check the server's firewall.

BTW what's the reason to use masquerade inside your network?

trendy · May 8, 2019, 12:01pm

Saves in static routes, but could be a pain with forwardings.

oppsig · May 8, 2019, 12:31pm

Here is my iptables rules for the external server running on 10.100.100.2.

Counter for the output chain get's activated everytime i do a curl.

Short paste:

-A INPUT -s 10.100.100.0/24 -i wg0 -p tcp -m tcp --dport 5000 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -o wg0 -p tcp -m tcp --sport 5000 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

All rules

sudo iptables -S                                                                                                            
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-ISOLATION-STAGE-1
-N DOCKER-ISOLATION-STAGE-2
-N DOCKER-USER
-N f2b-sshd
-A INPUT -p tcp -m multiport --dports 22 -j f2b-sshd
-A INPUT -i lo -j ACCEPT
-A INPUT -s 127.0.0.0/8 ! -i lo -j REJECT --reject-with icmp-port-unreachable
-A INPUT -p icmp -m state --state NEW -m icmp --icmp-type 8 -j ACCEPT
-A INPUT -p icmp -m state --state RELATED,ESTABLISHED -j ACCEPT
-A INPUT -i docker0 -p tcp -m state --state ESTABLISHED -j ACCEPT
-A INPUT -i eth0 -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 22 -j ACCEPT
-A INPUT -i eth0 -p udp -m state --state NEW,ESTABLISHED -m udp --dport 1194 -j ACCEPT
-A INPUT -i eth0 -p udp -m state --state ESTABLISHED -m udp --sport 53 -j ACCEPT
-A INPUT -i eth0 -p tcp -m state --state ESTABLISHED -m tcp --sport 80 -j ACCEPT
-A INPUT -i eth0 -p tcp -m state --state ESTABLISHED -m tcp --sport 443 -j ACCEPT
-A INPUT -i tun0 -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -p udp -m udp --dport 51820 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.100.100.0/24 -p tcp -m tcp --dport 53 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.100.100.0/24 -p udp -m udp --dport 53 -m conntrack --ctstate NEW -j ACCEPT
-A INPUT -s 10.100.100.0/24 -i wg0 -p tcp -m tcp --dport 5000 -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT
-A INPUT -i eth0 -p udp -m state --state NEW,ESTABLISHED -m udp --dport 51820 -j ACCEPT
-A INPUT -m limit --limit 3/min -j LOG --log-prefix "iptables_INPUT_denied: "
-A INPUT -j REJECT --reject-with icmp-port-unreachable
-A INPUT -i eth0 -p udp -m state --state NEW,ESTABLISHED -m udp --dport 51820 -j ACCEPT
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o docker0 -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -i wg0 -o wg0 -m conntrack --ctstate NEW -j ACCEPT
-A FORWARD -m limit --limit 3/min -j LOG --log-prefix "iptables_FORWARD_denied: "
-A FORWARD -j REJECT --reject-with icmp-port-unreachable
-A OUTPUT -o lo -j ACCEPT
-A OUTPUT -p icmp -j ACCEPT
-A OUTPUT -o docker0 -p tcp -m state --state NEW,ESTABLISHED -j ACCEPT
-A OUTPUT -o eth0 -p tcp -m state --state ESTABLISHED -m tcp --sport 22 -j ACCEPT
-A OUTPUT -o eth0 -p udp -m state --state ESTABLISHED -m udp --sport 1194 -j ACCEPT
-A OUTPUT -o eth0 -p udp -m state --state NEW,ESTABLISHED -m udp --dport 53 -j ACCEPT
-A OUTPUT -o eth0 -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 80 -j ACCEPT
-A OUTPUT -o eth0 -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 443 -j ACCEPT
-A OUTPUT -o tun0 -j ACCEPT
-A OUTPUT -o eth0 -p udp -m state --state NEW,RELATED,ESTABLISHED -m udp --dport 51820 -j ACCEPT
-A OUTPUT -o wg0 -p tcp -m tcp --sport 5000 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A OUTPUT -o eth0 -p udp -m state --state RELATED,ESTABLISHED -m udp --dport 51820 -j ACCEPT
-A OUTPUT -m limit --limit 3/min -j LOG --log-prefix "iptables_OUTPUT_denied: "
-A OUTPUT -j REJECT --reject-with icmp-port-unreachable
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
-A f2b-sshd -j RETURN

oppsig · May 8, 2019, 1:12pm

Ok, sorry for wasting anyone's time.
The flask service that was running on external server on port 5000 defaulted to listening on localhost.
Didn't think about this
So when i bound it to 10.100.100.2 instead everything was working fine.

Just have one question, if i want to put that same flask service into a docker container, how would i change my iptables rules to make that work?

system · May 18, 2019, 1:12pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.