you using pppoe + ipv6?
pppoe no, ipv6(dhcpv6-pd) yes.
Hey @db260179 : it looks like you're making good progress here; BTW, switching my Mikrotik devices to PoE injectors improved uptime, but still seeing power cycling. Two questions...
- Is there a newer build for the patches I saw mentioned over the past 3 weeks, or would I need to compile that from the Gitlab stuff?
- I saw Upgrade to 21.02 without switching to dsa : openwrt (reddit.com) ; which had me wondering if this is part of why you had to make this patchset to begin with.
Hope all is well.
1st question, my initial patches for master are just for people to confirm ipv6 and the disabled flow control. ipv6 hwnat (routing) is now in the upstream code, still yet to push a patch for the flow control disable.
Its up to the individual to sync the latest code i have and build (docker build tools in the source code).
2nd question: using the old swconfig brings back some missing switch features that the dsa code doesn't have fully - yet, i believe felix is working on some backport code, but will have to wait and see.
If you desperately need the old switch code features, then my 21.02-ralink branch will satisfy most people needs.
Thanks, busy at work, so no free time now.
1 Like
had 3 days uptime with software upload and ipv6 enabled followed by another 3.5 days with hardware offload and the ipv6 interface disabled. enabled the ipv6 interface today and had a crash in less than an hour.
root@OpenWrt:/# [ 2759.734862] CPU 1 Unable to handle k[ 2er759.739977] neCl PpaUgin 2g reUquestna batle to hviartnuadle kelr andderel pa ,isns g00 00reques0t10 8,a et virtpuc a==l 8 addre2s91s37 60000001,0 r8a ,== epc = =82 91837291376704
ra == 82913774
[ 2759.739995] CPU 0 Unable to handle kernel paging request at virtual address 00000108, epc == 82913760, ra == 82913774
[ 2759.740013] Oops[#1]:
[ 2759.740033] CPU: 0 PID: 3767 Comm: kworker/u8:42 Not tainted 5.10.100 #0
[ 2759.740046] CPU 3 Unable to handle kernel paging request at virtual address 00000108, epc == 82913760, ra == 82913774
[ 2759.740077] Workqueue: nf_ft_offload_del nf_flow_rule_route_ipv6 [nf_flow_table]
[ 2759.740096] $ 0 : 00000000 00000001 00000100 00000002
[ 2759.740127] $ 4 : 81688bc8 00000001 00000004 00000004
[ 2759.740157] $ 8 : 00000000 00000000 00245d17 ffffffff
[ 2759.740187] $12 : 00000001 00000400 00000000 00000000
[ 2759.740217] $16 : 000000f8 00000001 817adbdc 00000000
[ 2759.740247] $20 : 817adbe4 00000001 807f0000 00000000
[ 2759.740277] $24 : 00000000 80010f48
[ 2759.740308] $28 : 824ae000 824afd38 807f0000 82913774
[ 2759.740340] Hi : 000ff631
[ 2759.740346] Lo : fe7ede27
[ 2759.740369] epc : 82913760 nf_flow_offload_ipv6_hook+0x83c/0x1774 [nf_flow_table]
[ 2759.740390] ra : 82913774 nf_flow_offload_ipv6_hook+0x850/0x1774 [nf_flow_table]
[ 2759.740396] Status: 11008403 KERNEL EXL IE
[ 2759.740422] Cause : 40800008 (ExcCode 02)
[ 2759.740429] BadVA : 00000108
[ 2759.740437] PrId : 0001992f (MIPS 1004Kc)
[ 2759.740442] Modules linked in: pppoe ppp_async nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet wireguard pppox ppp _generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_ limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntr ack libchacha20poly1305 libblake2s slhc poly1305_mips nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 libcurve25519_g eneric libcrc32c libblake2s_generic crc_ccitt chacha_mips ip6_udp_tunnel udp_tunnel leds_gpio gpio_button_hotplug crc32c_generic
[ 2759.740756] Process kworker/u8:42 (pid: 3767, threadinfo=fa7bcd3d, task=f21c23bd, tls=00000000)
[ 2759.740762] Stack : 00000000 807f0000 81fa6000 808a8b40 00000000 00000003 00000000 824afd98
[ 2759.740814] 00000001 825df104 00000000 00000000 00000000 00000000 00000000 00000000
[ 2759.740863] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 2759.740912] 00000000 0000000c 808a8b40 80010dc4 00000000 81000318 00000000 80066b90
[ 2759.740962] 00000000 4e8545e8 809b2574 8300cb98 00000001 8300cb80 81407c00 00000000
[ 2759.741015] ...
[ 2759.741033] Call Trace:
[ 2759.741055] [<82913760>] nf_flow_offload_ipv6_hook+0x83c/0x1774 [nf_flow_table]
[ 2759.741077]
[ 2759.741082] Code: 12420039 2450fff8 00008825 <8e020010> 8e060018 27a50010 0040f809 24040002 04400002
[ 2759.741150]
[ 2759.741158] Oops[#2]:
[ 2759.741165] ---[ end trace 88751dc76d44e04b ]---
[ 2759.741176] CPU: 3 PID: 3676 Comm: kworker/u8:50 Tainted: G D 5.10.100 #0
[ 2759.741181] Kernel panic - not syncing: Fatal exception
[ 2759.741197] Workqueue: nf_ft_offload_del nf_flow_rule_route_ipv6 [nf_flow_table]
[ 2759.741225] $ 0 : 00000000 00000001 00000100 00000002
[ 2759.741250] $ 4 : 81c4de08 00000001 00000000 00000000
[ 2759.741275] $ 8 : fffffffc 00000001 016a382d 00000000
[ 2759.741301] $12 : 81033b80 02df8385 00000000 00000001
[ 2759.741327] $16 : 000000f8 00000001 817adbdc 00000000
[ 2759.741353] $20 : 817adbe4 00000001 807f0000 00000001
[ 2759.741379] $24 : e5b906e6 0000b21d
[ 2759.741404] $28 : 8309c000 8309dd38 807f0000 82913774
[ 2759.741431] Hi : 00009fd4
[ 2759.741436] Lo : 00000000
[ 2759.741454] epc : 82913760 nf_flow_offload_ipv6_hook+0x83c/0x1774 [nf_flow_table]
[ 2759.741471] ra : 82913774 nf_flow_offload_ipv6_hook+0x850/0x1774 [nf_flow_table]
[ 2759.741477] Status: 1100fc03 KERNEL EXL IE
[ 2759.741497] Cause : 40800008 (ExcCode 02)
[ 2759.741502] BadVA : 00000108
[ 2759.741509] PrId : 0001992f (MIPS 1004Kc)
[ 2759.741513] Modules linked in: pppoe ppp_async nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet wireguard pppox ppp _generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_ limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntr ack libchacha20poly1305 libblake2s slhc poly1305_mips nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 libcurve25519_g eneric libcrc32c libblake2s_generic crc_ccitt chacha_mips ip6_udp_tunnel udp_tunnel leds_gpio gpio_button_hotplug crc32c_generic
[ 2759.741764] Process kworker/u8:50 (pid: 3676, threadinfo=d1d26f8e, task=570e0dbe, tls=00000000)
[ 2759.741769] Stack : 00000000 807f0000 808a8b40 808a8b40 00000000 00000003 00000000 8309dd98
[ 2759.741812] 00000001 82906054 00000000 00000000 00000000 00000000 00000000 00000000
[ 2759.741854] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[ 2759.741896] 00000009 81688c20 00000009 0078b000 807ec418 81c4de60 81688c20 81688bc0
[ 2759.741940] 81033b40 4e8545e8 00000000 8300c098 00000001 8300c080 81407c00 00000000
[ 2759.741984] ...
[ 2759.741997] Call Trace:
[ 2759.742015] [<82913760>] nf_flow_offload_ipv6_hook+0x83c/0x1774 [nf_flow_table]
[ 2759.742028]
[ 2759.742032] Code: 12420039 2450fff8 00008825 <8e020010> 8e060018 27a50010 0040f809 24040002 04400002
[ 2759.742088]
[ 2760.753186] Rebooting in 3 seconds..
Type of connection: IPoE or PPPoE?
If you test with latest snapshot version?
Please download the latest snapshot from openwrt (not my repo)
Recent patches have fixed some hwnat issues.
1 Like
sorry, my last post was an update to my previous one. I'm running OpenWrt SNAPSHOT r18974-8144f9c665 not your repo.
Ok, but you are still running on an old snapshot - latest is r19053.
1 Like
I didn't see any commits that would be relevant but i updated to OpenWrt SNAPSHOT r19061-0890fb2df4 it still crashed.
root@OpenWrt:/# [49715.831887] CPU 0 Unable to handle kernel paging request at virtual address ffffd500, ep c == 8086ac08, ra == 828eb774
[49715.853107] Oops[#1]:
[49715.857626] CPU: 0 PID: 27804 Comm: kworker/u8:5 Not tainted 5.10.103 #0
[49715.870987] Workqueue: nf_ft_offload_del nf_flow_rule_route_ipv6 [nf_flow_table]
[49715.885703] $ 0 : 00000000 00000001 8086ac04 8319c680
[49715.896106] $ 4 : 00000002 83037d48 00000000 82526e80
[49715.906506] $ 8 : 00000010 001cedad 000621a0 ffffffff
[49715.916907] $12 : 81006b80 000621a0 ffffc091 00000001
[49715.927308] $16 : 8333dd98 00000001 824570dc 00000000
[49715.937708] $20 : 00000000 00000001 807f0000 00000001
[49715.948107] $24 : 81800000 80317ec4
[49715.958509] $28 : 83036000 83037d38 807f0000 828eb774
[49715.968910] Hi : 00000266
[49715.974625] Lo : 66666700
[49715.980341] epc : 8086ac08 0x8086ac08
[49715.987965] ra : 828eb774 nf_flow_offload_ipv6_hook+0x850/0x1774 [nf_flow_table]
[49716.003185] Status: 11008403 KERNEL EXL IE
[49716.011509] Cause : 40800008 (ExcCode 02)
[49716.019468] BadVA : ffffd500
[49716.025183] PrId : 0001992f (MIPS 1004Kc)
[49716.033317] Modules linked in: pppoe ppp_async nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flo w_table_inet wireguard pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_red ir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv 6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack_netlink nf_conntrack libchacha20poly1305 libblake2s slhc poly1305_mips nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_ defrag_ipv6 nf_defrag_ipv4 libcurve25519_generic libcrc32c libblake2s_generic crc_ccitt chacha_mips ip6_udp _tunnel udp_tunnel leds_gpio gpio_button_hotplug crc32c_generic
[49716.155752] Process kworker/u8:5 (pid: 27804, threadinfo=0ec5d0a2, task=805b3392, tls=00000000)
[49716.173045] Stack : 00000001 807f0000 808a8b40 808a8b40 00000000 00000003 00000000 83037d98
[49716.189683] 00000001 82891b54 00000000 00000000 00000000 00000000 00000000 00000000
[49716.206321] 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
[49716.222958] 0076d000 80010dc4 00000009 0076d000 807ec410 80065718 81006b80 807f0000
[49716.239597] 809b2574 91440311 81015b40 8245c018 00000001 8245c000 81407800 00000000
[49716.256235] ...
[49716.261095] Call Trace:
[49716.261100]
[49716.268917] [<80010dc4>] mips_smp_send_ipi_mask+0x98/0x21c
[49716.279828] [<80065718>] enqueue_entity+0xb0/0x384
[49716.289360] [<828ecfac>] nf_flow_rule_route_ipv6+0x4fc/0x588 [nf_flow_table]
[49716.303394] [<8005bc88>] try_to_wake_up+0x240/0x5b0
[49716.313101] [<800607b0>] pick_next_task_idle+0x38/0x4c
[49716.323338] [<80049cd4>] process_one_work+0x214/0x478
[49716.333386] [<8004a0b8>] worker_thread+0x180/0x5e8
[49716.342908] [<80049f38>] worker_thread+0x0/0x5e8
[49716.352092] [<80049f38>] worker_thread+0x0/0x5e8
[49716.361277] [<80050be4>] kthread+0x13c/0x144
[49716.369759] [<80050aa8>] kthread+0x0/0x144
[49716.377900] [<80050aa8>] kthread+0x0/0x144
[49716.386041] [<80050aa8>] kthread+0x0/0x144
[49716.394192] [<80002ff8>] ret_from_kernel_thread+0x14/0x1c
[49716.404932]
[49716.407881] Code: ffffffff 00000000 829bd500 <829bd500> 00000000 00000000 8086ac14 8086ac14 ffffff ed
[49716.427293]
[49716.430877] ---[ end trace 268592e020164e23 ]---
[49716.440140] Kernel panic - not syncing: Fatal exception
[49716.450576] Rebooting in 3 seconds..
1 Like
Humm
List your packages installed?
ISP is PPPoE or IPoE?
Probably have regression. Or incompatiblite with tunnels ipv6 in ipv4 or vice versa.
Or it's on nftables. Remenber that on master is runnig migration of iptables to nftables.
I not detect this crash on my router with master.
like i said above i'm not using PPPoE. my ipv6 wan is DHCPv6-PD.
i just have a basic setup with 2 lans using vlan filtering.
for me, latest patches to mt7621 offload and switch changes on ubiquity erx brought instability with hw offloading
Putting my good experience in context, I've only briefly tested hw offload and then switched right back to sw offload only. I am not using PPPoE either.
That said, I've got 12 days of ER-X gateway runtime without issues on r18993-a1b8a4d7b3 using only sw offload.
I'm using HW offlload (IPv4 only) with a build from Feb 18th for 19 days now!
I have selected this build as my "stable" build (better than 21.02.2 for my needs).
1 Like
the issues seems to be with ipv6 combo, there was a previous patch for ipv6 but they seems to have added a different one, I think is not well tested for ipv4+ipv6 scenario in HW offload (previous I had uptime of 40+ days, until a new flash)
this new commit improve perfomance.
I not have time for test now. Only for report for team
https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=fdbb468c5998d9adb1819751ffaf38013768a8eb
2 Likes
Does it have any effect on mt7621 series?
2 Likes
"This can improve performance under load by ensuring that NAPI processing is not pinned on CPU 0."