Openwrt crash, how to build debuginfo to mt7628.ko?

I have a mt7628 running openwrt running crashed:

[  228.888334] BUG: scheduling while atomic: RtmpMlmeTask/3536/0x00000200
[  228.894996] Modules linked in: qcserial option iptable_nat cdc_mbim usb_wwan sierra_net sierra rndis_host qmi_wwan pppoe pl2303 nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 l2tp_ppp ipt_REJECT ipt_MASQUERADE huawei_cdc_ncm ftdi_sio cp210x cdc_ncm cdc_ether xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG usbserial usbnet usb_serial_ch9344 ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda rtc_jxr160 pptp pppox ppp_mppe ppp_async nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables crc_ccitt ch341 cdc_wdm fuse mt7613 mt7628 gpio_pca953x i2c_mt7621 i2c_dev i2c_core ledtrig_heartbeat ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables pppoatm ppp_generic slhc ip_gre gre l2tp_netlink l2tp_core udp_tunnel ip6_udp_tunnel ip_tunnel tun vfat fat br2684 atm nls_utf8 nls_iso8859_1 nls_cp437 sha1_generic ecb usb_storage leds_gpio uhci_hcd ohci_platform ohci_hcd ehci_platform ehci_hcd sd_mod scsi_mod gpio_dev gpio_button_hotplug jfs ralink_eeprom_api usbcore nls_base usb_common mii cryptomgr aead crypto_null crypto_hash
[  229.014266] CPU: 0 PID: 3536 Comm: RtmpMlmeTask Not tainted 4.4.296 #0
[  229.020897] Stack : 80369870 00000000 00000001 803c0000 8626744c 803b5303 80349db4 00000dd0
          804034b4 00020000 00010000 00018001 00000001 80050a94 80350020 803b0000
          00000003 00020000 8034e28c 8627377c 00000001 8000a7ec 00000000 00000000
          00000001 00000000 00000000 00000000 00000000 00000000 00000000 00000000
          00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
          ...
[  229.057091] Call Trace:
[  229.059588] [<80019b9c>] show_stack+0x70/0x8c
[  229.064035] [<80044b5c>] __schedule_bug+0x44/0x60
[  229.068827] [<8000e0d8>] __schedule+0x64/0x668
[  229.073346] [<8000e740>] schedule+0x64/0x7c
[  229.078238] [<86733ba4>] RtmpusecDelay+0x28/0xa0 [mt7628]
[  229.084990] [<866af080>] MtAsicUpdateRxWCIDTable+0x9ec/0xadc [mt7628]
[  229.092794] [<866a010c>] MacTableInsertEntry+0x75c/0xb64 [mt7628]
[  229.100211] [<8660b190>] APPeerAuthSimpleRspGenAndSend+0x11b0/0x1528 [mt7628]
[  229.108081]
[  229.110104] BUG: scheduling while atomic: RtmpMlmeTask/3536/0xfffffe00
[  229.116771] Modules linked in: qcserial option iptable_nat cdc_mbim usb_wwan sierra_net sierra rndis_host qmi_wwan pppoe pl2303 nf_nat_ipv4 nf_conntrack_ipv6 nf_conntrack_ipv4 l2tp_ppp ipt_REJECT ipt_MASQUERADE huawei_cdc_ncm ftdi_sio cp210x cdc_ncm cdc_ether xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG usbserial usbnet usb_serial_ch9344 ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda rtc_jxr160 pptp pppox ppp_mppe ppp_async nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_nat nf_log_ipv4 nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ip_tables crc_ccitt ch341 cdc_wdm fuse mt7613 mt7628 gpio_pca953x i2c_mt7621 i2c_dev i2c_core ledtrig_heartbeat ip6t_REJECT nf_reject_ipv6 nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables x_tables pppoatm ppp_generic slhc ip_gre gre l2tp_netlink l2tp_core udp_tunnel ip6_udp_tunnel ip_tunnel tun vfat fat br2684 atm nls_utf8 nls_iso8859_1 nls_cp437 sha1_generic ecb usb_storage leds_gpio uhci_hcd ohci_platform ohci_hcd ehci_platform ehci_hcd sd_mod scsi_mod gpio_dev gpio_button_hotplug jfs ralink_eeprom_api usbcore nls_base usb_common mii cryptomgr aead crypto_null crypto_hash
[  229.236067] CPU: 0 PID: 3536 Comm: RtmpMlmeTask Tainted: G        W       4.4.296 #0
[  229.243920] Stack : 80369870 00000000 00000001 803c0000 8626744c 803b5303 80349db4 00000dd0
          804034b4 c0531608 00000000 00000000 00000000 80050a94 80350020 803b0000
          00000003 c0531608 8034e28c 86273da4 00000000 8000a7ec 00000000 00000000
          000001f2 00000000 00000000 00000000 00000000 00000000 00000000 00000000
          00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
          ...
[  229.280112] Call Trace:
[  229.282609] [<80019b9c>] show_stack+0x70/0x8c
[  229.287051] [<80044b5c>] __schedule_bug+0x44/0x60
[  229.291842] [<8000e0d8>] __schedule+0x64/0x668
[  229.296361] [<8000e740>] schedule+0x64/0x7c
[  229.301248] [<86734988>] __RtmpOSTaskWait+0xac/0x110 [mt7628]
[  229.308325] [<8665a634>] MlmeHandler+0x878/0x920 [mt7628]
[  229.314430]
[  229.317039] entry=0
[  229.582533] Rcv Wcid(1) AddBAReq
[  229.585906] Start Seq = 00000000
[  231.408443] da match,0x000c0a083118

Seeing that the crash point is inside the [mt7628] module, I found the source location in . /build_dir/target-mipsel_24kec+dsp_uClibc-1.0.x/linux-ramips_mt7628/mt7628-4.1.1.0/ipkg-mipsel_24kec_dsp/kmod-mt7628/lib/modules/ 4.4.296/mt7628.ko.

Next, I want to check the corresponding source file and code location, so I use gdb to check it, and gdb says no debug symbols, how can I compile mt7628.ko with debug symbols?

$ gdb ./build_dir/target-mipsel_24kec+dsp_uClibc-1.0.x/linux-ramips_mt7628/mt7628-4.1.1.0/ipkg-mipsel_24kec_dsp/kmod-mt7628/lib/modules/4.4.296/mt7628.ko
...

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./build_dir/target-mipsel_24kec+dsp_uClibc-1.0.x/linux-ramips_mt7628/mt7628-4.1.1.0/ipkg-mipsel_24kec_dsp/kmod-mt7628/lib/modules/4.4.296/mt7628.ko...
(No debugging symbols found in ./build_dir/target-mipsel_24kec+dsp_uClibc-1.0.x/linux-ramips_mt7628/mt7628-4.1.1.0/ipkg-mipsel_24kec_dsp/kmod-mt7628/lib/modules/4.4.296/mt7628.ko)

make makeconfig selects a lot of debuginfo related options, other .ko files have debug symbols, but only mt7628.ko does not.

$ make menuconfig
Advanced configuration options (for developers) 
 -> Target Options
  -> -O2 -pipe -mno-branch-likely -march=24kec -mdsp -g

 Global build settings
  -> Collect kernel debug information = y
  -> Compile the kernel with profiling enabled = y
  -> Compile the kernel with symbol table information = y
  -> Compile the kernel with debug information = y
  -> Compile packages with debugging info = y
  -> Binary stripping method (none) 
  -> Strip unnecessary exports from the kernel image = n
  -> Strip unnecessary functions from libraries = n
  -> Strip unnecessary functions from libraries = n

How can I compile mt7628.ko with debug symbols? Or can I compile mt7628.ko into the linux kernel so that I can gdb load it to see the linux kernel symbols?

Whatever you are trying to compile, this is not vanilla OpenWrt, but it looks like a vendor SDK based on an ancient version of OpenWrt. The 4.4 kernel was used in LEDE 17.01 which is long out of support.

AFAIK, mt7628.ko is the proprietary Mediatek driver, so you won't find source code for it anyway.

You have two options:

  1. Ask the supplier of the SDK
  2. Upgrade to a supported version of OpenWrt and switch to the mt76 driver.
1 Like

Hi @andyboeh , you're right. I searched the entire directory and couldn't find the source code for mt7621, only bin file mt7621.ko.

Thank you

Although it may not help here (with vendor SDK), but for future use with the actual OpenWrt:

  • build_dir contains the stripped versions without debug symbols
  • staging_dir contains the original intermediate versions with debug symbols, before stripping.

You should provide the staging_dir path for gdb for debugging.

Example discussion e.g. in

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.