Intermittent packet loss - OpenWrt behind orange flybox 4g

Hi, could someone shed some light on what's happening here?

I recently swapped out my DSL modem/router for a 4G "flybox" from orange. I left all wrt settings as before except for changing the WAN on the wrt box from pppoe to dhcp. (I know this is a double NAT setup but AFAIK this is not relevant here).
in a nutshell the setup looks like this:

4G flybox router 192.168.1.1-> 192.168.1.101 wrt router 10.0.0.1 -> LAN devices.
When connecting a laptop directly (via wifi) to the flybox the connection is fine, i.e. no packet loss - so I'm assuming the 4G connection itself or the flybox is not the problem.

However when connecting the laptop via the WRT router lots of packets are dropped.
running ping on the WRT box itself shows a regular pattern of dropped packets. (pinging the flybox from WRT shows no loss, so the wired connection should also not be the problem).


64 bytes from 139.59.209.225: seq=19 ttl=52 time=26.354 ms
64 bytes from 139.59.209.225: seq=20 ttl=52 time=32.187 ms
64 bytes from 139.59.209.225: seq=21 ttl=52 time=24.965 ms
64 bytes from 139.59.209.225: seq=22 ttl=52 time=24.811 ms
64 bytes from 139.59.209.225: seq=23 ttl=52 time=24.581 ms
64 bytes from 139.59.209.225: seq=51 ttl=52 time=530.124 ms
64 bytes from 139.59.209.225: seq=52 ttl=52 time=27.809 ms
64 bytes from 139.59.209.225: seq=53 ttl=52 time=23.713 ms
64 bytes from 139.59.209.225: seq=54 ttl=52 time=26.434 ms
64 bytes from 139.59.209.225: seq=55 ttl=52 time=26.282 ms
64 bytes from 139.59.209.225: seq=56 ttl=52 time=26.041 ms
64 bytes from 139.59.209.225: seq=57 ttl=52 time=23.882 ms
64 bytes from 139.59.209.225: seq=58 ttl=52 time=27.696 ms
64 bytes from 139.59.209.225: seq=59 ttl=52 time=28.508 ms
64 bytes from 139.59.209.225: seq=86 ttl=52 time=2004.813 ms
64 bytes from 139.59.209.225: seq=87 ttl=52 time=1004.747 ms
64 bytes from 139.59.209.225: seq=88 ttl=52 time=24.721 ms
64 bytes from 139.59.209.225: seq=89 ttl=52 time=27.549 ms
64 bytes from 139.59.209.225: seq=90 ttl=52 time=28.335 ms
64 bytes from 139.59.209.225: seq=91 ttl=52 time=26.121 ms
64 bytes from 139.59.209.225: seq=92 ttl=52 time=26.953 ms
64 bytes from 139.59.209.225: seq=93 ttl=52 time=26.717 ms
64 bytes from 139.59.209.225: seq=94 ttl=52 time=24.517 ms
64 bytes from 139.59.209.225: seq=95 ttl=52 time=27.311 ms
64 bytes from 139.59.209.225: seq=124 ttl=52 time=557.311 ms
64 bytes from 139.59.209.225: seq=125 ttl=52 time=32.252 ms
64 bytes from 139.59.209.225: seq=126 ttl=52 time=30.048 ms
64 bytes from 139.59.209.225: seq=127 ttl=52 time=23.867 ms
64 bytes from 139.59.209.225: seq=128 ttl=52 time=25.703 ms
64 bytes from 139.59.209.225: seq=129 ttl=52 time=25.522 ms
64 bytes from 139.59.209.225: seq=130 ttl=52 time=26.280 ms
64 bytes from 139.59.209.225: seq=131 ttl=52 time=23.083 ms
64 bytes from 139.59.209.225: seq=132 ttl=52 time=26.864 ms
64 bytes from 139.59.209.225: seq=160 ttl=52 time=1670.572 ms
64 bytes from 139.59.209.225: seq=161 ttl=52 time=670.477 ms
64 bytes from 139.59.209.225: seq=162 ttl=52 time=24.051 ms
64 bytes from 139.59.209.225: seq=163 ttl=52 time=25.869 ms
^C
--- lede-project.org ping statistics ---
164 packets transmitted, 42 packets received, 74% packet loss
round-trip min/avg/max = 23.083/201.499/2004.813 ms

any help in understanding what's going on here would be much appreciated, thanks!

How is the OpenWrt router connected to the Flybox?

the connection between the wrt box and the flybox is a wired connection from one the lan ports of the flybox to the wan port of wrt router.

1 Like

I;ve tried with a completely fresh bare-bone install on different HW... and the behaviour remains the same. So either it is something in the flybox or something too deep in wrt for me to figure out. Either way not a useable setup.

So (FWIW) I've bridged the flybox in the "LAN", and downgraded my router to an (intelligent) switch. Not ideal but at least no dropped packets.

Whoa...Are you saying the issue stopped in this configuration?

  • When you say downgrade, did you change firmware, or configure OpenWrt as such?
  • So, this means, you could have plugged the device into LAN of OpenWrt?
  • Did you edit anything like the firewall?
  • Do your device's instructions tell you this?

(Also you never showed the successful pings previously.)

BTW, welcome to the community!

Those pings raise concerns about RFI/physical interference for me... ( suspect would be closer to the flybox end.. any devices close to the cables etc...? tidy time? )

worth checking for interface errors... et. al.

Hi, thanks for picking the thread up again...
I'll try to answer your question one by one:

  • Are you saying the issue stopped in this configuration?
    YES

  • When you say downgrade, did you change firmware, or configure OpenWrt as such?
    configure OpenWrt as such. I meant functionally - as in downgrade from a router to a switch/bridge. The version(s) i tried were 18.06.1 and 18.06.4. (on a ramips and an ar71xx).

  • Did you edit anything like the firewall?
    No - tests were done with a stock install (except for setting the WAN interface to DHCP)

  • Do your device's instructions tell you this?
    No - the current setup can be blamed solely on me :wink:

  • (Also you never showed the successful pings previously.)
    Not sure what you mean... the log above does show some pings go through.

so to be really clear about the current working setup.
on the flybox

  • disable "ethernet" (i.e. WAN) - set to "LAN only"
  • disable all forms of updates (not strictly necessary just reducing the possible problems)
  • disable DHCP, I also set the "LAN IP" to 10.0.0.1

on the (ex-router) bridge

  • flashed clean (not strictly necessary just reducing the possible problems)
  • br-lan: static 10.0.0.2 , GW 10.0.0.1 (as the flybox is on 10.0.0.1)
  • dhcp enabled: 10.0.0.100-200, option: 3,10.0.0.1
  • delete WAN/WAN6 interfaces
  • add eth0.2 (my WAN port) to br-lan (this effectivelys put the "wan" port in the LAN)

current setup uses exactly same cables/physical configuration... no dropped packets.
so i'm ruling physical issues out for now - but thanks for thinking along.

1 Like