Losing internet since nf_conntrack: nf_conntrack: table full

Installing and Using OpenWrt
1

Hi I’m having a very weird problem. I use my OpenWrt device as a second router, which get an ip address from the parent router. Just from yesterday the parent router could no longer provide internet to the devices connected to it (including my OpenWrt). After checking the log on OpenWrt I found out that the conntrack table is full. I saw the OpenWrt device (192.168.0.6 is the ip assigned by the parent router) was constantly sending requests to a host 203.107.63.46 (not always to this, it can vary time to time) despite it failed all the time. It repeated like more than 50000 times in a short time. I think the parent router was thus overloaded and out of work.

After checking netstat -n -p I still can’t found out which process was doing these flooding requests.

Firmware: https://downloads.openwrt.org/releases/23.05.0/targets/ramips/mt7621/openwrt-23.05.0-ramips-mt7621-d-team_newifi-d2-squashfs-sysupgrade.bin

No changes have been made to the OpenWrt device during the last 2 weeks I would say

Hope you can help
Thanks

root@OpenWrt:~# conntrack -L
~~~~~~
tcp      6 53 SYN_SENT src=192.168.0.6 dst=203.107.63.46 sport=11204 dport=443 packets=1 bytes=1452 [UNREPLIED] src=203.107.63.46 dst=192.168.0.6 sport=443 dport=11204 packets=0 bytes=0 mark=0 use=1
tcp      6 52 SYN_SENT src=192.168.0.6 dst=203.107.63.46 sport=9049 dport=443 packets=1 bytes=1452 [UNREPLIED] src=203.107.63.46 dst=192.168.0.6 sport=443 dport=9049 packets=0 bytes=0 mark=0 use=1
tcp      6 11 SYN_SENT src=192.168.0.6 dst=203.107.63.46 sport=14735 dport=443 packets=1 bytes=1452 [UNREPLIED] src=203.107.63.46 dst=192.168.0.6 sport=443 dport=14735 packets=0 bytes=0 mark=0 use=1
tcp      6 49 SYN_SENT src=192.168.0.6 dst=203.107.63.46 sport=46009 dport=443 packets=1 bytes=1452 [UNREPLIED] src=203.107.63.46 dst=192.168.0.6 sport=443 dport=46009 packets=0 bytes=0 mark=0 use=1
conntrack v1.4.8 (conntrack-tools): 55628 flow entries have been shown.

root@OpenWrt:~# netstat -n -p
netstat: showing only processes with your user ID
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 192.168.1.1:22          192.168.1.223:60506     ESTABLISHED 5076/dropbear
raw        0   2240 192.168.0.6:255         203.107.63.46:*         255         -
raw        0   2240 192.168.0.6:255         203.107.63.46:*         255         -
2

50K requests sounds like one of your client machines is trying to DOS that site. Do you have malware on something?

1 Like
3

This address belongs to

203.107.63.0/24
Aliyun Computing Co., LTD

that is a part of

# 203.107.46.0/23
## AS37963 · Hangzhou Alibaba Advertising Co.,Ltd.
4

The destination ip address is really random? Last time it was pointing to an ip of Microsoft and sometimes to an ip in South Africa.

5

Since the request is sent from 192.168.0.6 (the ip of OpenWrt itself). If it was sent from one of the clients the ip should be 192.168.1.x (my OpenWrt LAN)?

6

You can probably identify the most talkative client on OpenWrt LAN by checking the list in Status - Realtime Graphs - Connections.

7

No, the 192.168.0.6 router is NATting the clients behind it.

2 Likes
8

Like Dave says, you're seeing the NATted version. To see where it's coming from, you could tcpdump on the pre-NAT interface. Find one of those IPs that's getting hit right now and do something like this (example uses the IP of that alibaba host you give above):

$ opkg update ; opkg install tcpdump

$ tcpdump -vvn  -i br-lan  host 203.107.63.46
tcpdump: listening on br-lan, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:00:31.989123 IP (tos 0x0, ttl 128, id 11622, offset 0, flags [none], proto ICMP (1), length 60)
    192.168.1.99 > 203.107.63.46: ICMP echo request, id 1, seq 30, length 40
...
1 Like
9

thanks! I did use tcpdump when the router got stuck.
Here’re some records out of the tons of results:

No. Time Source Destination Protocol Length Info
69981 6.903496 192.168.0.6 203.107.63.46 SSL 1466 Continuation Data
69982 6.903644 192.168.0.6 203.107.63.46 SSL 1466 Continuation Data
69983 6.903693 192.168.0.6 203.107.63.46 SSL 1466 [TCP Port numbers reused] , Continuation Data
69984 6.903749 192.168.0.6 203.107.63.46 SSL 1466 [TCP Port numbers reused] , Continuation Data
69985 6.903846 192.168.0.6 203.107.63.46 SSL 1466 Continuation Data

Seems like it is just OpenWrt who was sending these requests?

10

Can’t you login to the LAN side of 192.168.0.6 and look there for traffic to 203.107.63.46? Is the LAN IP 192.168,1.1?

1 Like
11

Correct it is 192.168.1.1.
Luci has been so overloaded during these suspicious traffic that no realtime connections can be listed. But the cli works.