Mesh_hwmp - Latest Snapshot Kernel dump

For Developers
1

Hi,

I tested a mesh setup on Tp-Link EAP255 v3 Outdoor (ath79) and Cell C router (ipq40xx) and get kernel dumps on both using the latest snapshot code.

22.03.x does not have the issue.

Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.057527] ------------[ cut here ]------------
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.057582] WARNING: CPU: 2 PID: 679 at backports-5.15.74-1/net/mac80211/mesh_hwmp.c:348 airtime_link_metric_get+0xe0/0x528 [mac80211]
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.061309] Modules linked in: pppoe ppp_async option nft_fib_inet nf_flow_table_ipv6 nf_flow_table_ipv4 nf_flow_table_inet batman_adv ath10k_pci ath10k_core ath usb_wwan qmi_wwan pppox ppp_generic nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_objref nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_counter nft_compat nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mac80211 iptable_mangle iptable_filter ipt_REJECT ip_tables cfg80211 xt_time xt_tcpudp xt_multiport xt_mark xt_mac xt_limit xt_comment xt_TCPMSS xt_LOG x_tables usbserial usbnet slhc nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc_ccitt compat cdc_wdm tun ghash_arm_ce cmac leds_gpio xhci_plat_hcd xhci_pci xhci_hcd dwc3 dwc3_qcom gpio_button_hotplug mii crc32c_generic
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.129673] CPU: 2 PID: 679 Comm: rpcd Tainted: G        W         5.15.76 #0
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.151864] Hardware name: Generic DT based system
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.158985] [<c030ce38>] (unwind_backtrace) from [<c0309838>] (show_stack+0x10/0x14)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.163673] [<c0309838>] (show_stack) from [<c0617d28>] (dump_stack_lvl+0x40/0x4c)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.171570] [<c0617d28>] (dump_stack_lvl) from [<c0321d5c>] (__warn+0xb8/0x114)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.178949] [<c0321d5c>] (__warn) from [<c0321e20>] (warn_slowpath_fmt+0x68/0x78)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.186153] [<c0321e20>] (warn_slowpath_fmt) from [<bf215060>] (airtime_link_metric_get+0xe0/0x528 [mac80211])
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.193805] [<bf215060>] (airtime_link_metric_get [mac80211]) from [<bf1c0044>] (sta_set_sinfo+0x6c0/0xe24 [mac80211])
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.203705] [<bf1c0044>] (sta_set_sinfo [mac80211]) from [<bf1d68cc>] (ieee80211_dump_station+0x58/0x70 [mac80211])
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.214381] [<bf1d68cc>] (ieee80211_dump_station [mac80211]) from [<bf168994>] (nl80211_dump_station+0xdc/0x110 [cfg80211])
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.224713] [<bf168994>] (nl80211_dump_station [cfg80211]) from [<c083cb94>] (netlink_dump+0x160/0x34c)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.235809] [<c083cb94>] (netlink_dump) from [<c083d7ec>] (__netlink_dump_start+0x1b0/0x278)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.245183] [<c083d7ec>] (__netlink_dump_start) from [<c08406b0>] (genl_rcv_msg+0x314/0x340)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.253863] [<c08406b0>] (genl_rcv_msg) from [<c083f630>] (netlink_rcv_skb+0xb8/0x11c)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.262283] [<c083f630>] (netlink_rcv_skb) from [<c083fc00>] (genl_rcv+0x28/0x34)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.270005] [<c083fc00>] (genl_rcv) from [<c083ed40>] (netlink_unicast+0x1ec/0x2d4)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.277558] [<c083ed40>] (netlink_unicast) from [<c083f004>] (netlink_sendmsg+0x1dc/0x444)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.285025] [<c083f004>] (netlink_sendmsg) from [<c07b886c>] (____sys_sendmsg+0x1c8/0x258)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.293361] [<c07b886c>] (____sys_sendmsg) from [<c07ba4a4>] (___sys_sendmsg+0xa4/0xdc)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.301605] [<c07ba4a4>] (___sys_sendmsg) from [<c07ba600>] (sys_sendmsg+0x40/0x6c)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.309546] [<c07ba600>] (sys_sendmsg) from [<c0300040>] (ret_fast_syscall+0x0/0x48)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.317142] Exception stack(0xc2aa3fa8 to 0xc2aa3ff0)
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.325131] 3fa0:                   00000000 00000000 00000007 bed65b90 00000000 00000000
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.330085] 3fc0: 00000000 00000000 0000001d 00000128 b6f5ec0b b6df9174 bed65c08 b6dfbaa2
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.338240] 3fe0: bed65b38 bed65b28 b6f33d94 b6f33270
Mon Nov  7 06:23:35 2022 kern.warn kernel: [  399.346480] ---[ end trace 0421ab7e008aa012 ]---

If you need any more info please let me know.

2

then why not use stable 22.03 ?

3

The TP Link EAP 225 outdoor v3 is not included in 22.03.

It used to have some serious network related issues (on the Ethernet side) which now seems fixed in the latest snapshot code. The only issue is if you use it in a mesh environment.

4

Ah, I see .... well, then I guess you need to keep applying the latest snapshots until it doesn't dump any more.
They are betas, after all.

2 Likes
5

I have exactly the same issue even with 22.03.2 and opened an issue for that. I thought this might be a MediaTek 7915E issue, but it is clearly not if you have it on other hardware as well. Please post your information into my ticket: https://github.com/openwrt/openwrt/issues/11149

6

Thanks for the heads-up that its also on 22.03.2.

I did not upgrade yet from 22.03.0 to 22.03.2 since my firmware makes use of Curl (devices are centrally managed and we use Curl since it has support for HTTPS) and I saw there was a report about Curl and ssl library issues.

7

It would be more interesting to see what happened just before the dump.
It is not possible to set many mesh parameters in the uci wireless config as the interface has to be up before setting the parameter but the uci config is used to start the process uf bringing up the interface. Hwmp settings are included.
This, depending on the driver,can cause a crash.
See mesh11sd:

It might be something else entirely but lets see what was in the logs before the dump.

9

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.