WRT1900ACv2 Multiple WAN packet loss

I've been trying to get a mwan3 setup going on my Linksys WRT1900ACv2 for the last couple days, but I keep running into the same issue: One of the two WAN interfaces always looses lots of packets. This isn't an issue with the underlying connection, as swapping the ethernet cables leads to packet loss on the opposite interface (they also work individually). Rebooting the router sometimes also switches the faulty interface around. Additionally, there is a weird regularity to the packet loss: ~8 ICMP packets go though, followed by ~16 dropped packets.

What I've tried so far:

  • Install the kmod-switch-mvsw61xx package (although it already appears to be installed by default: mvsw61xx 10.mvsw61xx: Found MV88E6176 at f1072004.mdio-mii:10)
  • Try every possible combination of tagged / untagged in the LuCI switch control panel
  • Randomly changing vlan and port settings in swconfig (port_based, qmode, ...)
  • Randomly changed every available interface setting in LuCI (metric, static IP instead of DHCP, ...)
  • Re-installed from scratch (without keeping settings) multiple times
  • Switched through all physical ports

I had the same problem on an older LEDE version (~17), so this probably isn't a regression.

ping output for eth0:

PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=57 time=22.320 ms
64 bytes from 1.1.1.1: seq=1 ttl=57 time=21.965 ms
64 bytes from 1.1.1.1: seq=2 ttl=57 time=21.789 ms
64 bytes from 1.1.1.1: seq=3 ttl=57 time=23.441 ms
--- 1.1.1.1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 21.789/22.378/23.441 ms

ping output for eth1:

PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: seq=3 ttl=58 time=2078.906 ms
64 bytes from 1.1.1.1: seq=4 ttl=58 time=1078.894 ms
64 bytes from 1.1.1.1: seq=5 ttl=58 time=78.852 ms
64 bytes from 1.1.1.1: seq=6 ttl=58 time=15.281 ms
64 bytes from 1.1.1.1: seq=7 ttl=58 time=17.204 ms
--- 1.1.1.1 ping statistics ---
8 packets transmitted, 5 packets received, 37% packet loss
round-trip min/avg/max = 15.281/653.827/2078.906 ms

ifconfig output:

eth0      Link encap:Ethernet  HWaddr C2:56:27:C2:5C:DA
          inet addr:192.168.0.101  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::c056:27ff:fec2:5cda/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:954519 errors:0 dropped:13 overruns:0 frame:0
          TX packets:934332 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:798859490 (761.8 MiB)  TX bytes:800221008 (763.1 MiB)
          Interrupt:37

eth1      Link encap:Ethernet  HWaddr C0:56:27:C2:5C:DA
          inet addr:192.168.1.104  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::c256:27ff:fec2:5cda/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3985 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3988 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:532
          RX bytes:2253676 (2.1 MiB)  TX bytes:990619 (967.4 KiB)
          Interrupt:36

dmesg output:

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.95 (buildbot@builds-03.infra.lede-project.org) (gcc version 7.3.0 (OpenWrt GCC 7.3.0 r7627-753531d)) #0 SMP Mon Jan 28 08:54:32 2019
[    0.000000] CPU: ARMv7 Processor [414fc091] revision 1 (ARMv7), cr=10c5387d
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt: Machine model: Linksys WRT1900ACv2
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] On node 0 totalpages: 131072
[    0.000000] free_area_init_node: node 0, pgdat c0931000, node_mem_map dfbf9000
[    0.000000]   Normal zone: 1024 pages used for memmap
[    0.000000]   Normal zone: 0 pages reserved
[    0.000000]   Normal zone: 131072 pages, LIFO batch:31
[    0.000000] random: get_random_bytes called from 0xc08008d0 with crng_init=0
[    0.000000] percpu: Embedded 12 pages/cpu @dfbd0000 s18956 r8192 d22004 u49152
[    0.000000] pcpu-alloc: s18956 r8192 d22004 u49152 alloc=12*4096
[    0.000000] pcpu-alloc: [0] 0 [0] 1
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 130048
[    0.000000] Kernel command line: console=ttyS0,115200 root=/dev/mtdblock5 ro rootdelay=1 rootfstype=jffs2 earlyprintk mtdparts=armada-nand:2048K(uboot)ro,256K(u_env),256K(s_env),1m@9m(devinfo),40m@10m(kernel),34m@16m(rootfs),40m@50m(alt_kernel),34m@56m(alt_rootfs),80m@10m(ubifs),-@90m(syscfg)
[    0.000000] Bootloader command line (ignored): console=ttyS0,115200 root=/dev/mtdblock5 ro rootdelay=1 rootfstype=jffs2 earlyprintk mtdparts=armada-nand:2048K(uboot)ro,256K(u_env),256K(s_env),1m@9m(devinfo),40m@10m(kernel),34m@16m(rootfs),40m@50m(alt_kernel),34m@56m(alt_rootfs),80m@10m(ubifs),-@90m(syscfg)
[    0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[    0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[    0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Memory: 510920K/524288K available (6144K kernel code, 198K rwdata, 812K rodata, 1024K init, 232K bss, 13368K reserved, 0K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xe0800000 - 0xff800000   ( 496 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xe0000000   ( 512 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0008000 - 0xc0700000   (7136 kB)
[    0.000000]       .init : 0xc0800000 - 0xc0900000   (1024 kB)
[    0.000000]       .data : 0xc0900000 - 0xc0931900   ( 199 kB)
[    0.000000]        .bss : 0xc0931900 - 0xc096b9d8   ( 233 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] L2C-310 enabling early BRESP for Cortex-A9
[    0.000000] L2C-310 full line of zeros enabled for Cortex-A9
[    0.000000] L2C-310 D prefetch enabled, offset 1 lines
[    0.000000] L2C-310 dynamic clock gating enabled, standby mode enabled
[    0.000000] L2C-310 Coherent cache controller enabled, 16 ways, 1024 kB
[    0.000000] L2C-310 Coherent: CACHE_ID 0x410054c9, AUX_CTRL 0x56070001
[    0.000006] sched_clock: 64 bits at 800MHz, resolution 1ns, wraps every 4398046511103ns
[    0.000016] clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles: 0xb881274fa3, max_idle_ns: 440795210636 ns
[    0.000028] Switching to timer-based delay loop, resolution 1ns
[    0.000142] Ignoring duplicate/late registration of read_current_timer delay
[    0.000148] clocksource: armada_370_xp_clocksource: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 76450417870 ns
[    0.000280] Calibrating delay loop (skipped), value calculated using timer frequency.. 1600.00 BogoMIPS (lpj=8000000)
[    0.000287] pid_max: default: 32768 minimum: 301
[    0.000339] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000344] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000555] CPU: Testing write buffer coherency: ok
[    0.000566] CPU0: Spectre v2: using BPIALL workaround
[    0.000662] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
[    0.000770] Setting up static identity map for 0x100000 - 0x100060
[    0.000838] mvebu-soc-id: MVEBU SoC ID=0x6820, Rev=0x4
[    0.000909] mvebu-pmsu: Initializing Power Management Service Unit
[    0.000951] Hierarchical SRCU implementation.
[    0.001123] smp: Bringing up secondary CPUs ...
[    0.001235] Booting CPU 1
[    0.001381] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
[    0.001383] CPU1: Spectre v2: using BPIALL workaround
[    0.001426] smp: Brought up 1 node, 2 CPUs
[    0.001430] SMP: Total of 2 processors activated (3200.00 BogoMIPS).
[    0.001434] CPU: All CPU(s) started in SVC mode.
[    0.002911] VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
[    0.002959] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.002966] futex hash table entries: 512 (order: 3, 32768 bytes)
[    0.003012] pinctrl core: initialized pinctrl subsystem
[    0.003322] NET: Registered protocol family 16
[    0.003829] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.004469] cpuidle: using governor ladder
[    0.004625] mvebu-pmsu: CPU hotplug support is currently broken on Armada 38x: disabling
[    0.004631] mvebu-pmsu: CPU idle is currently broken on Armada 38x: disabling
[    0.010198] reg-fixed-voltage usb3_1-vbus: could not find pctldev for node /soc/internal-regs/pinctrl@18000/usb3_1-vbus-pins, deferring probe
[    0.010342] SCSI subsystem initialized
[    0.010508] libata version 3.00 loaded.
[    0.010603] usbcore: registered new interface driver usbfs
[    0.010634] usbcore: registered new interface driver hub
[    0.010666] usbcore: registered new device driver usb
[    0.011321] clocksource: Switched to clocksource arm_global_timer
[    0.011659] NET: Registered protocol family 2
[    0.011901] TCP established hash table entries: 4096 (order: 2, 16384 bytes)
[    0.011924] TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
[    0.011956] TCP: Hash tables configured (established 4096 bind 4096)
[    0.011991] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    0.012005] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    0.012067] NET: Registered protocol family 1
[    0.012080] PCI: CLS 0 bytes, default 64
[    0.012989] Crashlog allocated RAM at address 0x3f00000
[    0.013171] workingset: timestamp_bits=30 max_order=17 bucket_order=0
[    0.014604] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.014609] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    0.016514] io scheduler noop registered
[    0.016521] io scheduler deadline registered (default)
[    0.016942] armada-38x-pinctrl f1018000.pinctrl: registered pinctrl driver
[    0.017864] mvebu-pcie soc:pcie: PCI host bridge to bus 0000:00
[    0.017873] pci_bus 0000:00: root bus resource [io  0x1000-0xfffff]
[    0.017879] pci_bus 0000:00: root bus resource [mem 0xe0000000-0xe7ffffff]
[    0.017884] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.017899] pci 0000:00:01.0: [11ab:6820] type 01 class 0x060400
[    0.017986] pci 0000:00:02.0: [11ab:6820] type 01 class 0x060400
[    0.018056] PCI: bus0: Fast back to back transfers disabled
[    0.018062] pci 0000:00:01.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.018069] pci 0000:00:02.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    0.018115] pci 0000:01:00.0: [11ab:2a55] type 00 class 0x020000
[    0.018139] pci 0000:01:00.0: reg 0x10: [mem 0x40000000-0x400fffff 64bit pref]
[    0.018149] pci 0000:01:00.0: reg 0x18: [mem 0x40100000-0x401fffff 64bit pref]
[    0.018207] pci 0000:01:00.0: supports D1 D2
[    0.018211] pci 0000:01:00.0: PME# supported from D0 D1 D3hot D3cold
[    0.018264] PCI: bus1: Fast back to back transfers disabled
[    0.018271] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    0.018319] pci 0000:02:00.0: [11ab:2a55] type 00 class 0x020000
[    0.018348] pci 0000:02:00.0: reg 0x10: [mem 0x42000000-0x420fffff 64bit pref]
[    0.018360] pci 0000:02:00.0: reg 0x18: [mem 0x42100000-0x421fffff 64bit pref]
[    0.018439] pci 0000:02:00.0: supports D1 D2
[    0.018442] pci 0000:02:00.0: PME# supported from D0 D1 D3hot D3cold
[    0.018498] PCI: bus2: Fast back to back transfers disabled
[    0.018505] pci_bus 0000:02: busn_res: [bus 02-ff] end is updated to 02
[    0.018521] pci 0000:00:01.0: BAR 8: assigned [mem 0xe0000000-0xe01fffff]
[    0.018526] pci 0000:00:02.0: BAR 8: assigned [mem 0xe0200000-0xe03fffff]
[    0.018534] pci 0000:01:00.0: BAR 0: assigned [mem 0xe0000000-0xe00fffff 64bit pref]
[    0.018544] pci 0000:01:00.0: BAR 2: assigned [mem 0xe0100000-0xe01fffff 64bit pref]
[    0.018553] pci 0000:00:01.0: PCI bridge to [bus 01]
[    0.018559] pci 0000:00:01.0:   bridge window [mem 0xe0000000-0xe01fffff]
[    0.018567] pci 0000:02:00.0: BAR 0: assigned [mem 0xe0200000-0xe02fffff 64bit pref]
[    0.018577] pci 0000:02:00.0: BAR 2: assigned [mem 0xe0300000-0xe03fffff 64bit pref]
[    0.018587] pci 0000:00:02.0: PCI bridge to [bus 02]
[    0.018592] pci 0000:00:02.0:   bridge window [mem 0xe0200000-0xe03fffff]
[    0.018802] mv_xor f1060800.xor: Marvell shared XOR driver
[    0.061769] mv_xor f1060800.xor: Marvell XOR (Descriptor Mode): ( xor cpy intr )
[    0.061874] mv_xor f1060900.xor: Marvell shared XOR driver
[    0.121730] mv_xor f1060900.xor: Marvell XOR (Descriptor Mode): ( xor cpy intr )
[    0.121867] Serial: 8250/16550 driver, 16 ports, IRQ sharing enabled
[    0.122484] console [ttyS0] disabled
[    0.142557] f1012000.serial: ttyS0 at MMIO 0xf1012000 (irq = 22, base_baud = 12500000) is a 16550A
[    0.917072] console [ttyS0] enabled
[    0.922132] loop: module loaded
[    0.925571] ahci-mvebu f10a8000.sata: AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl platform mode
[    0.934671] ahci-mvebu f10a8000.sata: flags: 64bit ncq sntf led only pmp fbs pio slum part sxs
[    0.943816] scsi host0: ahci-mvebu
[    0.947363] scsi host1: ahci-mvebu
[    0.950843] ata1: SATA max UDMA/133 mmio [mem 0xf10a8000-0xf10a9fff] port 0x100 irq 41
[    0.958808] ata2: SATA max UDMA/133 mmio [mem 0xf10a8000-0xf10a9fff] port 0x180 irq 41
[    0.967129] pxa3xx-nand f10d0000.flash: This platform can't do DMA on this device
[    0.974903] nand: device found, Manufacturer ID: 0x01, Chip ID: 0xf1
[    0.981283] nand: AMD/Spansion S34ML01G2
[    0.985231] nand: 128 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
[    0.992845] pxa3xx-nand f10d0000.flash: ECC strength 16, ECC step size 2048
[    1.000021] Bad block table found at page 65472, version 0x01
[    1.006129] Bad block table found at page 65408, version 0x01
[    1.012166] 10 fixed-partitions partitions found on MTD device pxa3xx_nand-0
[    1.019243] Creating 10 MTD partitions on "pxa3xx_nand-0":
[    1.024759] 0x000000000000-0x000000200000 : "u-boot"
[    1.029976] 0x000000200000-0x000000240000 : "u_env"
[    1.035067] 0x000000240000-0x000000280000 : "s_env"
[    1.040141] 0x000000900000-0x000000a00000 : "devinfo"
[    1.045396] 0x000000a00000-0x000003200000 : "kernel1"
[    1.050735] 0x000001000000-0x000003200000 : "ubi"
[    1.055709] 0x000003200000-0x000005a00000 : "kernel2"
[    1.061048] 0x000003800000-0x000005a00000 : "rootfs2"
[    1.066376] 0x000005a00000-0x000008000000 : "syscfg"
[    1.071631] 0x000000280000-0x000000900000 : "unused_area"
[    1.077538] libphy: Fixed MDIO Bus: probed
[    1.081963] libphy: orion_mdio_bus: probed
[    1.086234] mvneta_bm f10c8000.bm: Buffer Manager for network controller enabled
[    1.094966] mvneta f1070000.ethernet eth0: Using hardware mac address c0:56:27:c2:5c:da
[    1.104258] mvneta f1034000.ethernet eth1: Using random mac address 2e:ff:c1:5a:a7:81
[    1.112280] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.118834] ehci-pci: EHCI PCI platform driver
[    1.123328] ehci-platform: EHCI generic platform driver
[    1.128638] ehci-orion: EHCI orion driver
[    1.132752] orion-ehci f1058000.usb: EHCI Host Controller
[    1.138183] orion-ehci f1058000.usb: new USB bus registered, assigned bus number 1
[    1.145832] orion-ehci f1058000.usb: irq 38, io mem 0xf1058000
[    1.181303] orion-ehci f1058000.usb: USB 2.0 started, EHCI 1.00
[    1.187458] hub 1-0:1.0: USB hub found
[    1.191235] hub 1-0:1.0: 1 port detected
[    1.195603] usbcore: registered new interface driver usb-storage
[    1.201799] i2c /dev entries driver
[    1.216596] (NULL device *): hwmon_device_register() is deprecated. Please convert the driver to use hwmon_device_register_with_info().
[    1.229017] orion_wdt: Initial timeout 171 sec
[    1.233717] sdhci: Secure Digital Host Controller Interface driver
[    1.239922] sdhci: Copyright(c) Pierre Ossman
[    1.244392] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.253638] marvell-cesa f1090000.crypto: CESA device successfully registered
[    1.261206] NET: Registered protocol family 10
[    1.266108] Segment Routing with IPv6
[    1.269812] NET: Registered protocol family 17
[    1.274300] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    1.287419] 8021q: 802.1Q VLAN Support v1.8
[    1.291683] Registering SWP/SWPB emulation handler
[    1.292552] ata2: SATA link down (SStatus 0 SControl 300)
[    1.301944] ata1: SATA link down (SStatus 0 SControl 300)
[    1.308826] xhci-hcd f10f8000.usb3: xHCI Host Controller
[    1.314190] xhci-hcd f10f8000.usb3: new USB bus registered, assigned bus number 2
[    1.321768] xhci-hcd f10f8000.usb3: hcc params 0x0a000990 hci version 0x100 quirks 0x0000000000010010
[    1.331047] xhci-hcd f10f8000.usb3: irq 43, io mem 0xf10f8000
[    1.337045] hub 2-0:1.0: USB hub found
[    1.340823] hub 2-0:1.0: 1 port detected
[    1.344877] xhci-hcd f10f8000.usb3: xHCI Host Controller
[    1.350215] xhci-hcd f10f8000.usb3: new USB bus registered, assigned bus number 3
[    1.357746] xhci-hcd f10f8000.usb3: Host supports USB 3.0  SuperSpeed
[    1.364249] usb usb3: We don't know the algorithms for LPM for this host, disabling LPM.
[    1.372551] hub 3-0:1.0: USB hub found
[    1.376329] hub 3-0:1.0: 1 port detected
[    1.380615] UBI: auto-attach mtd5
[    1.383977] ubi0: attaching mtd5
[    1.394594] random: fast init done
[    1.481030] ubi0: scanning is finished
[    1.489041] ubi0: attached mtd5 (name "ubi", size 34 MiB)
[    1.494473] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    1.501380] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    1.508196] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    1.515188] ubi0: good PEBs: 272, bad PEBs: 0, corrupted PEBs: 0
[    1.521218] ubi0: user volume: 2, internal volumes: 1, max. volumes count: 128
[    1.528474] ubi0: max/mean erase counter: 4/2, WL threshold: 4096, image sequence number: 1681116493
[    1.537647] ubi0: available PEBs: 0, total reserved PEBs: 272, PEBs reserved for bad PEB handling: 20
[    1.546921] ubi0: background thread "ubi_bgt0d" started, PID 696
[    1.553255] block ubiblock0_0: created from ubi0:0(rootfs)
[    1.558763] ubiblock: device ubiblock0_0 (rootfs) set to be root filesystem
[    1.565945] mvsw61xx 10.mvsw61xx: Found MV88E6176 at f1072004.mdio-mii:10
[    1.572769] mvsw61xx 10.mvsw61xx: Using direct addressing
[    1.586496] hctosys: unable to open rtc device (rtc0)
[    1.591920] Waiting 1 sec before mounting root device...
[    2.633175] VFS: Mounted root (squashfs filesystem) readonly on device 254:0.
[    2.640882] Freeing unused kernel memory: 1024K
[    2.836603] init: Console is alive
[    2.840083] init: - watchdog -
[    3.013157] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    3.046801] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    3.054692] init: - preinit -
[    3.214055] mvneta f1070000.ethernet tmp0: renamed from eth0
[    3.219911] random: procd: uninitialized urandom read (4 bytes read)
[    3.272713] mvneta f1034000.ethernet eth0: renamed from eth1
[    3.322677] mvneta f1070000.ethernet eth1: renamed from tmp0
[    3.657180] random: jshn: uninitialized urandom read (4 bytes read)
[    3.674269] random: jshn: uninitialized urandom read (4 bytes read)
[    3.849768] mvneta f1034000.ethernet eth0: configuring for fixed/sgmii link mode
[    3.857604] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[    3.863659] mvneta f1034000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[    3.871584] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[    6.951773] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 805
[    6.979677] UBIFS (ubi0:1): recovery needed
[    7.051398] UBIFS (ubi0:1): recovery completed
[    7.055889] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
[    7.063760] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[    7.073721] UBIFS (ubi0:1): FS size: 27934720 bytes (26 MiB, 220 LEBs), journal size 1396736 bytes (1 MiB, 11 LEBs)
[    7.084202] UBIFS (ubi0:1): reserved for root: 1319425 bytes (1288 KiB)
[    7.090843] UBIFS (ubi0:1): media format: w4/r0 (latest is w5/r0), UUID 897A786A-B908-402F-A13C-B4FC396AB85A, small LPT model
[    7.102595] mount_root: overlay filesystem has not been fully initialized yet
[    7.111012] mount_root: switching to ubifs overlay
[    7.155386] ubi1: attaching mtd8
[    7.257346] ubi1: scanning is finished
[    7.265456] ubi1: attached mtd8 (name "syscfg", size 38 MiB)
[    7.271140] ubi1: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    7.278055] ubi1: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    7.284874] ubi1: VID header offset: 2048 (aligned 2048), data offset: 4096
[    7.291868] ubi1: good PEBs: 296, bad PEBs: 8, corrupted PEBs: 0
[    7.297898] ubi1: user volume: 1, internal volumes: 1, max. volumes count: 128
[    7.305153] ubi1: max/mean erase counter: 159/116, WL threshold: 4096, image sequence number: 732253408
[    7.314588] ubi1: available PEBs: 0, total reserved PEBs: 296, PEBs reserved for bad PEB handling: 12
[    7.323855] ubi1: background thread "ubi_bgt1d" started, PID 815
[    7.334123] UBIFS (ubi1:0): background thread "ubifs_bgt1_0" started, PID 819
[    7.355337] UBIFS (ubi1:0): recovery needed
[    7.407117] random: crng init done
[    7.410532] random: 3 urandom warning(s) missed due to ratelimiting
[    7.471136] UBIFS (ubi1:0): recovery completed
[    7.475635] UBIFS (ubi1:0): UBIFS: mounted UBI device 1, volume 0, name "syscfg"
[    7.483068] UBIFS (ubi1:0): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[    7.493028] UBIFS (ubi1:0): FS size: 34283520 bytes (32 MiB, 270 LEBs), journal size 1777664 bytes (1 MiB, 14 LEBs)
[    7.503510] UBIFS (ubi1:0): reserved for root: 1619295 bytes (1581 KiB)
[    7.510153] UBIFS (ubi1:0): media format: w4/r0 (latest is w5/r0), UUID 27C1F136-C0DC-4850-A3D1-5EC59A935C92, small LPT model
[    7.524033] urandom-seed: Seed file not found (/etc/urandom.seed)
[    7.556810] mvneta f1034000.ethernet eth0: Link is Down
[    7.623501] procd: - early -
[    7.626417] procd: - watchdog -
[    8.228085] procd: - watchdog -
[    8.231337] procd: - ubus -
[    8.284205] procd: - init -
[    8.368350] kmodloader: loading kernel modules from /etc/modules.d/*
[    8.375752] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    8.382589] Loading modules backported from Linux version wt-2017-11-01-0-gfe248fc2c180
[    8.390627] Backport generated by backports.git v4.14-rc2-1-31-g86cf0e5d
[    8.404459] <<Marvell Mac80211 Wireless PCIE Network Driver version 10.3.8.0-20181114>>
[    8.412536] pci 0000:00:01.0: enabling device (0140 -> 0142)
[    8.418310] ieee80211 phy0: priv->pcmd_buf = ddc34000  priv->pphys_cmd_buf = 1dc34000
[    8.424724] ieee80211 phy0: fw download start
[    8.541695] ieee80211 phy0: FwSize = 118776 downloaded Size = 118776 curr_iteration 65495
[    9.417300] ieee80211 phy0: fw download complete
[    9.417312] ieee80211 phy0: device node: /soc/pcie/pcie@1,0
[    9.423388] ieee80211 phy0: pcmd = ddc34000
[    9.440316] ieee80211 phy0: firmware version: 0x702091a
[    9.463590] ieee80211 phy0: firmware region code: 0
[    9.500393] ieee80211 phy0: regdomain: FCC
[    9.500398] ieee80211 phy0: Channel: 36: 0x0 0x0 0xf
[    9.500403] ieee80211 phy0: 12 12 12 12 12 12 12 12 10 10 10 10 f f f f
[    9.500407] ieee80211 phy0: Channel: 40: 0x0 0x0 0xf
[    9.500412] ieee80211 phy0: 16 16 16 16 16 16 16 16 16 16 16 16 f f f f
[    9.500415] ieee80211 phy0: Channel: 44: 0x0 0x0 0xf
[    9.500420] ieee80211 phy0: 16 16 16 16 16 16 16 16 16 16 16 16 f f f f
[    9.500423] ieee80211 phy0: Channel: 48: 0x0 0x0 0xf
[    9.500429] ieee80211 phy0: 16 16 16 16 16 16 16 16 16 16 16 16 f f f f
[    9.500431] ieee80211 phy0: Channel: 52: 0x0 0x0 0xf
[    9.500437] ieee80211 phy0: 11 11 11 11 10 10 10 10 12 12 12 12 10 10 10 10
[    9.500440] ieee80211 phy0: Channel: 56: 0x0 0x0 0xf
[    9.500445] ieee80211 phy0: 11 11 11 11 10 10 10 10 12 12 12 12 10 10 10 10
[    9.500448] ieee80211 phy0: Channel: 60: 0x0 0x0 0xf
[    9.500453] ieee80211 phy0: 11 11 11 11 10 10 10 10 12 12 12 12 10 10 10 10
[    9.500456] ieee80211 phy0: Channel: 64: 0x0 0x0 0xf
[    9.500462] ieee80211 phy0: 11 11 11 11 10 10 10 10 12 12 12 12 10 10 10 10
[    9.500465] ieee80211 phy0: Channel: 100: 0x0 0x0 0xf
[    9.500470] ieee80211 phy0: 10 10 10 10 11 11 11 11 12 12 12 12 10 10 10 10
[    9.500473] ieee80211 phy0: Channel: 104: 0x0 0x0 0xf
[    9.500479] ieee80211 phy0: 11 11 11 11 10 10 10 10 12 12 12 12 10 10 10 10
[    9.500482] ieee80211 phy0: Channel: 108: 0x0 0x0 0xf
[    9.500487] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500490] ieee80211 phy0: Channel: 112: 0x0 0x0 0xf
[    9.500495] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500498] ieee80211 phy0: Channel: 116: 0x0 0x0 0xf
[    9.500503] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500506] ieee80211 phy0: Channel: 120: 0x0 0x0 0xf
[    9.500512] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500514] ieee80211 phy0: Channel: 124: 0x0 0x0 0xf
[    9.500520] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500523] ieee80211 phy0: Channel: 128: 0x0 0x0 0xf
[    9.500528] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500531] ieee80211 phy0: Channel: 132: 0x0 0x0 0xf
[    9.500536] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500539] ieee80211 phy0: Channel: 136: 0x0 0x0 0xf
[    9.500544] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500547] ieee80211 phy0: Channel: 140: 0x0 0x0 0xf
[    9.500553] ieee80211 phy0: 11 11 11 11 10 10 10 10 11 11 11 11 10 10 10 10
[    9.500556] ieee80211 phy0: Channel: 149: 0x0 0x0 0xf
[    9.500561] ieee80211 phy0: 16 16 16 16 16 16 16 16 15 15 15 15 15 15 15 15
[    9.500564] ieee80211 phy0: Channel: 153: 0x0 0x0 0xf
[    9.500569] ieee80211 phy0: 16 16 16 16 15 15 15 15 15 15 15 15 15 15 15 15
[    9.500572] ieee80211 phy0: Channel: 157: 0x0 0x0 0xf
[    9.500578] ieee80211 phy0: 16 16 16 16 15 15 15 15 15 15 15 15 15 15 15 15
[    9.500581] ieee80211 phy0: Channel: 161: 0x0 0x0 0xf
[    9.500586] ieee80211 phy0: 16 16 16 16 15 15 15 15 15 15 15 15 15 15 15 15
[    9.500589] ieee80211 phy0: Channel: 165: 0x0 0x0 0xf
[    9.500594] ieee80211 phy0: 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15
[    9.501136] ieee80211 phy0: 2G disabled, 5G enabled
[    9.506049] ieee80211 phy0: 4 TX antennas, 4 RX antennas
[    9.511531] pci 0000:00:02.0: enabling device (0140 -> 0142)
...
[   10.590220] ieee80211 phy1: 2G enabled, 5G disabled
[   10.595146] ieee80211 phy1: 4 TX antennas, 4 RX antennas
[   10.601842] ip_tables: (C) 2000-2006 Netfilter Core Team
[   10.609052] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
[   10.625203] xt_time: kernel timezone is -0000
[   10.632052] PPP generic driver version 2.4.2
[   10.636748] NET: Registered protocol family 24
[   10.642190] kmodloader: done loading kernel modules from /etc/modules.d/*
[   12.490870] mvneta f1034000.ethernet eth0: configuring for fixed/sgmii link mode
[   12.498382] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[   12.504619] mvneta f1034000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[   12.516089] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   12.526411] br-lan: port 1(eth0.1) entered blocking state
[   12.531865] br-lan: port 1(eth0.1) entered disabled state
[   12.537409] device eth0.1 entered promiscuous mode
[   12.542234] device eth0 entered promiscuous mode
[   12.551273] br-lan: port 1(eth0.1) entered blocking state
[   12.556710] br-lan: port 1(eth0.1) entered forwarding state
[   12.562423] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[   12.576901] mvneta f1070000.ethernet eth1: configuring for fixed/rgmii-id link mode
[   12.585031] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
[   12.590961] mvneta f1070000.ethernet eth1: Link is Up - 1Gbps/Full - flow control off
[   12.607099] IPv6: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
[   13.511394] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready

swconfig output:

VLAN 2:
	port_based: 0
	vid: 2
	ports: 4 6
VLAN 3:
	port_based: 0
	vid: 3
	ports: 3 5

/etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd26:e27e:0737::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth0.1'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'wan'
	option proto 'dhcp'
	option ifname 'eth0'
	option metric '5'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option vid '1'
	option ports '0 1 2 5t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option vid '2'
	option ports '4 6'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option vid '3'
	option ports '3 5'

config interface 'wan2'
	option proto 'dhcp'
	option ifname 'eth1'
	option metric '10'

Version: 18.06.2 r7676-cddd7b4c77

This clearly implies that one of the ethernet cables is damaged, when you swap the cable, so also the packet loss swaps = the cable is the problem.

1 Like

Should have phrased that better: Not swapping the actual cables, swapping port on the router.
Still, thanks for the suggestion, I'm completely out of ideas at this point :neutral_face:

well if you move a bad cable to a new port, it will still be a bad cable... and the new port will experience the loss so again, replace both cables and see what happens.

Already did, the cables aren't the issue. As stated they both work fine individually, the packet drops only occur once the 3rd VLAN is enabled, and then the faulty interface sometimes switches between reboots.

Try using both cpu ports tagged for the vlans they participate in, and use eth0.1, eth0.3 eth1.2 for your interface definitions.

Have you noticed that WAN eth1 IP address conflicts with LAN IP address?

I don't see an IP conflict. But I do see:

  • That the CPU has some bad untagged VLANs configs (VLAN 2 is off and VLAN 3 is marked untagged)
  • The OP has defined a VLAN 3 on the Switch, but there's no VLAN 3 defined in Network Interfaces

WAN

and the LAN

both 192.168.1.X

1 Like

Oh wow...yea, the OP needs to fix that conflict too.

I applied the following changes:

  1. Resolved the IP conflict (thanks @trendy)
  2. Tagged the respective CPU for both WAN VLANs instead of leaving them untagged (thanks @dlakelan)

Sadly, the issue still persist, but I can at least reliably reproduce and provoke the issue now: https://www.dropbox.com/s/a8l168nurh04ocx/openwrt.mp4?dl=0

The CPU is still off and untagged on VLANs 1 and 2. You need to fix that.

3 Likes

Sorry about that.

Just to confirm that I understood your response correctly:

1 Like

That is correct.

1 Like

Oh my god it actually works now :open_mouth:
Thank you!

2 Likes

One of the first things I do on any network using IPv4 NAT is to immediately generate two random numbers between 2 and 255 and set the LAN as 10.x.y.0/24 :wink: I figure the chance of a conflict is pretty low. If I know that I'm going to be in a network using the 10. range I might use 192.168.x.0/24 with x generated randomly between about 13 and 241 (avoiding the low and high range where things tend to be set) or maybe that other range what is it 172.16. - 172.31.

solve a lot of problems before they even happen.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.