VBox OpenWRT setup experiencing tcp re-transmissions, dup ack, and out-of-order

Installing and Using OpenWrt Network and Wireless Configuration
1

Hello,

I have setup OpenWRT on a Ubuntu 20.04.2 Laptop (Acer v3-471g, core i5, 4GB Ram) using VirtualBox. The below is the diagram of setup:

Untitled Diagram-Page-2
Untitled Diagram-Page-22148×2820 394 KB

Description of the diagram:

  • The laptop has an ethernet PCI port which is connected to OpenWRT eth1 using VBox Bridged Adapter and promisuous mode: allow all. This helped in giving the laptop an IP address 192.168.1.10 and allowing for docker to leverage the same ethernet for connecting PiHole with IP address 192.168.1.11 using Docker MacVlan Network or having NtopNG to monitor all traffic passing through the port using Docker Host Network. The PiHole, Laptop, OpenWRT can all ping each other or "see" other without any issues.
  • A USB 3.0 HUB with usb ASIX ethernet adapter is connected to the Laptop USB 3.0 port. The usb ASIX ethernet adapter was passed through to the OpenWRT VBox as eth2 such that the host laptop no longer sees it.
  • OpenWrt br-lan for LAN was created using eth1 and eth2 and has the subnet 192.168.1.0/24.
  • A 4G usb modem was connected to the USB 3.0 HUB and is setup with ModemManager on the Host Laptop for internet access and appears as wwan0 on Host Laptop.
  • An additional VBox NAT adapter has been connected to the OpenWRT VM which is configured as WAN and automatically takes an IP from the VBox Network DHCP. All public traffic is then routed through this NAT adapter to the public internet via wwan0.
  • The ethernet ports each have a dummy AP/switch (basically are routers with DHCP settings switched off), and each have a client.

My problem lies in the traffic which between the personal laptop 192.168.1.222 and the TV Receiver 192.168.1.216 or even the Dummy AP/Switch 192.168.1.3. It seems when the traffic is being routed over the br-lan between its two ethernets, the traffic faces issues and timeouts. I have run wireshark/tcpdump on the following places:

  • On the Host Laptop for eth0
  • On OpenWRT br-lan
  • On personal laptop 192.168.1.222

Host Laptop for eth0

image
image1247×263 163 KB

TCP issues were detected

OpenWRT br-lan

root@LEDE:~# tcpdump -XX -n -l host 192.168.1.3 and host 192.168.1.222
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:18:54.146834 IP 192.168.1.222.1025 > 192.168.1.3.80: Flags [S], seq 1886495236, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
13:18:54.148158 IP 192.168.1.3.80 > 192.168.1.222.1025: Flags [S.], seq 1218033837, ack 1886495237, win 5840, options [mss 1460,nop,nop,sackOK,nop,wscale 0], length 0
13:18:54.151417 IP 192.168.1.222.1025 > 192.168.1.3.80: Flags [.], ack 1, win 256, length 0
13:18:54.176643 IP 192.168.1.222.1025 > 192.168.1.3.80: Flags [P.], seq 1:524, ack 1, win 256, length 523: HTTP: GET /setup.cgi?next_file=start.htm HTTP/1.1
13:18:54.178648 IP 192.168.1.3.80 > 192.168.1.222.1025: Flags [.], ack 524, win 6432, length 0
13:18:54.184046 IP 192.168.1.3.80 > 192.168.1.222.1025: Flags [P.], seq 1:1035, ack 524, win 6432, length 1034: HTTP: HTTP/1.1 401 Unauthorized
13:18:54.185703 IP 192.168.1.3.80 > 192.168.1.222.1025: Flags [F.], seq 1035, ack 524, win 6432, length 0
13:18:54.187220 IP 192.168.1.222.1025 > 192.168.1.3.80: Flags [.], ack 1035, win 252, length 0
13:18:54.206147 IP 192.168.1.222.1025 > 192.168.1.3.80: Flags [.], ack 1036, win 252, length 0
13:18:54.211346 IP 192.168.1.222.1025 > 192.168.1.3.80: Flags [F.], seq 524, ack 1036, win 252, length 0
13:18:54.212196 IP 192.168.1.3.80 > 192.168.1.222.1025: Flags [.], ack 525, win 6432, length 0
^C
11 packets captured
11 packets received by filter
0 packets dropped by kernel

No issues were observed

Personal laptop 192.168.1.222

image
image1232×250 15.7 KB

No issues were observed

In case the personal laptop was connected to Dummy AP/Switch 192.168.1.3, the traffic does not experience any issues and everything works just fine. It is only when the devices are connected to different Ethernets of the br-lan in OpenWRT where the traffic between them faces issues and things do not work as they should be, such as receiver application unable to connect to TV receiver or Dummy AP/Switch 192.168.1.3 settings page not loading.

Is the issue something caused by VBox or OpenWRT? How can I troubleshoot it and resolve it?

2

i'm not seeing anything particularly odd at first glance...

other than it seems you are running lede-17.01?

(i'd be looking at the host / hypervisor interface properties if I were you)

3

You're right, I felt that his might be noticed directly. I'm very worried about upgrading something that is working because I don't have a redundant setup to keep the family connected while trying out upgrades and changes. Do you think that upgrading would probably solve the problem though?

I thought so too, what particular detail should I look at? This is my first time dealing with it.

I forgot to mention that VirtualBox I'm using is 6.1.16.

4

doubt it...

promisc settings, driver versions, host level loops, forgotten low level apps etc. etc. (docker subsystems?)

5

I've done some more digging and found out that the traffic observed on my personal laptop 192.168.1.222 in fact does face tcp issues similar to what is observed on host laptop eth0.

Also, if I try accessing 192.168.1.3 or 192.168.1.216 from host laptop 192.168.1.10, there are no traffic issues, and everything works smoothly. Does this mean that the setup on the host laptop is OK while the actual problem is somehow related to the dummy AP/switch 192.168.1.15?

6

your issue is not related to openwrt...

keep simplifying your topology until you isolate the issue... it is not difficult... upper layer testing on the host is rarely a good indicator of actual end-to-end network operation...

1 Like