WRT3200ACM wifi failure

This router worked perfectly until one day I realized 5Ghz wifi causing random crash, autoreboot without any error log. I tested by setting it up as an AP only, I tried flashing other firmwares but still in vain. The longest uptime record is 2 days on Linksys firmware. On DD-WRT, the router might restart within a minute before I could access admin page to disable wifi. Turn out OpenWRT disables wifi by default is really a good option :smiley:

In another test, I changed 5Ghz wifi password so that no client can connect to. After a while, 2.4Ghz wifi failed too, it was still visible on the scan but couldn't connect. Luci became unresponsive, switch still worked fine (my PC connected to a LAN port, had Internet access), then I ssh and noticed that load average reached above 3.0, still had normal free memory, syslog was spammed with these errors:

Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.570941] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.710989] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.770369] ieee80211 phy1: cmd 0x801d=MEMAddrAccess timed out
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.771010] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.776233] ieee80211 phy1: return code: 0x001d
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.786374] ieee80211 phy1: timeout: 0x001d
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.921080] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37262.981085] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37263.121145] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.debug kernel: [37263.180359] ieee80211 phy1: MACREG_REG_INT_CODE: 0x0000
Fri Jul 17 07:52:02 2020 kern.err kernel: [37263.185618] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37263.321202] ieee80211 phy1: buffer is NULL for tx done ring
Fri Jul 17 07:52:02 2020 kern.err kernel: [37263.381220] ieee80211 phy1: buffer is NULL for tx done ring

I reboot, disabled 5Ghz band, only used 2.4Ghz and didn't see crash or spam log anymore. But the story is not over yet. When a villain is taken down, another one emerges. The router halted after 1 or 2 days. I observed that 2.4Ghz causing memory leak. This doesn't occur immediately after a reboot, free memory is slightly decreased over time until depleted and hang. I didn't see any error or strange messages in syslog. While the router was running out of memory, I went to Luci wireless page, clicked on restart radio1 (802.11bgn) and all leaked memory (I think) was released so that router could keep working. Syslog showed these:

Wed Jul 15 23:34:39 2020 daemon.notice hostapd: wlan1: interface state ENABLED->DISABLED
Wed Jul 15 23:34:39 2020 daemon.notice hostapd: wlan1: AP-STA-DISCONNECTED xx:xx:xx:xx:xx:xx
Wed Jul 15 23:34:39 2020 daemon.notice hostapd: wlan1: AP-DISABLED
Wed Jul 15 23:34:39 2020 daemon.notice hostapd: wlan1: CTRL-EVENT-TERMINATING
Wed Jul 15 23:34:39 2020 daemon.notice hostapd: nl80211: deinit ifname=wlan1 disabled_11b_rates=0
Wed Jul 15 23:34:39 2020 kern.info kernel: [130626.232973] device wlan1 left promiscuous mode
Wed Jul 15 23:34:39 2020 kern.info kernel: [130626.237603] br-lan: port 2(wlan1) entered disabled state
Wed Jul 15 23:34:39 2020 kern.debug kernel: [130626.318054] ieee80211 phy1: staid 1 deleted
Wed Jul 15 23:34:39 2020 kern.debug kernel: [130626.337904] ieee80211 phy1: change: 0x60
Wed Jul 15 23:34:39 2020 kern.debug kernel: [130626.422939] ieee80211 phy1: change: 0x100
Wed Jul 15 23:34:39 2020 daemon.notice netifd: Network device 'wlan1' link is down
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.482086] ------------[ cut here ]------------
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.486815] WARNING: CPU: 1 PID: 3099 at kernel/softirq.c:165 0xc012d0d8
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.493632] Modules linked in: pppoe ppp_async pppox ppp_generic nf_conntrack_ipv6 iptable_nat ipt_REJECT ipt_MASQUERADE xt_time xt_tcpudp xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_FLOWOFFLOAD xt_CT slhc rfcomm nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack mwifiex_sdio mwifiex iptable_mangle iptable_filter ip_tables hidp hci_uart crc_ccitt btusb btmrvl_sdio btmrvl btintel bnep bluetooth hid evdev input_core mwlwifi mac80211 cfg80211 compat nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 tun ecdh_generic kpp ecb cmac gpio_button_hotplug
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.565509] CPU: 1 PID: 3099 Comm: hostapd Not tainted 4.14.180 #0
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.571802] Hardware name: Marvell Armada 380/385 (Device Tree)
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.577837] Function entered at [<c010ebf8>] from [<c010a8b8>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.583782] Function entered at [<c010a8b8>] from [<c063ed54>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.589726] Function entered at [<c063ed54>] from [<c0128620>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.595670] Function entered at [<c0128620>] from [<c01286ec>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.601613] Function entered at [<c01286ec>] from [<c012d0d8>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.607556] Function entered at [<c012d0d8>] from [<c052ed84>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.613500] Function entered at [<c052ed84>] from [<c056b0ec>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.619444] Function entered at [<c056b0ec>] from [<c056b2c0>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.625388] Function entered at [<c056b2c0>] from [<bf058b50>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.631338] Function entered at [<bf058b50>] from [<bf07941c>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.637285] Function entered at [<bf07941c>] from [<bf095d74>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.643231] Function entered at [<bf095d74>] from [<bf095e2c>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.649176] Function entered at [<bf095e2c>] from [<bf0a8a24>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.655121] Function entered at [<bf0a8a24>] from [<bf0a8be0>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.661066] Function entered at [<bf0a8be0>] from [<c05330a8>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.667010] Function entered at [<c05330a8>] from [<c053be00>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.672954] Function entered at [<c053be00>] from [<c053bec8>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.678898] Function entered at [<c053bec8>] from [<c05acf34>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.684843] Function entered at [<c05acf34>] from [<c051d308>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.690786] Function entered at [<c051d308>] from [<c02039a4>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.696730] Function entered at [<c02039a4>] from [<c0203a48>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.702674] Function entered at [<c0203a48>] from [<c01074e0>]
Wed Jul 15 23:34:40 2020 kern.warn kernel: [130626.708618] ---[ end trace ace704e767e969ae ]---
Wed Jul 15 23:34:40 2020 user.notice mac80211: Failed command: iw phy phy1 set antenna 0xffffffff 0xffffffff
Wed Jul 15 23:34:40 2020 daemon.notice netifd: radio1 (1918): command failed: Not supported (-95)
Wed Jul 15 23:34:40 2020 user.notice mac80211: Failed command: iw phy phy1 set distance 0
Wed Jul 15 23:34:40 2020 daemon.err hostapd: Configuration file: /var/run/hostapd-phy1.conf
Wed Jul 15 23:34:40 2020 kern.debug kernel: [130627.185882] ieee80211 phy1: change: 0xffffffff
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.277447] IPv6: ADDRCONF(NETDEV_UP): wlan1: link is not ready
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.284395] br-lan: port 2(wlan1) entered blocking state
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.289890] br-lan: port 2(wlan1) entered disabled state
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.295430] device wlan1 entered promiscuous mode
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.300333] br-lan: port 2(wlan1) entered blocking state
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.305765] br-lan: port 2(wlan1) entered forwarding state
Wed Jul 15 23:34:40 2020 daemon.err hostapd: Using interface wlan1 with hwaddr 60:38:e0:xx:xx:xx and ssid "OpenWRT"
Wed Jul 15 23:34:40 2020 kern.debug kernel: [130627.355822] ieee80211 phy1: change: 0x100
Wed Jul 15 23:34:40 2020 kern.debug kernel: [130627.368259] ieee80211 phy1: change: 0x42
Wed Jul 15 23:34:40 2020 kern.info kernel: [130627.520321] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
Wed Jul 15 23:34:41 2020 daemon.notice hostapd: wlan1: interface state UNINITIALIZED->ENABLED
Wed Jul 15 23:34:41 2020 daemon.notice hostapd: wlan1: AP-ENABLED
Wed Jul 15 23:34:41 2020 kern.debug kernel: [130627.604111] ieee80211 phy1: change: 0x20
Wed Jul 15 23:34:41 2020 daemon.notice netifd: Network device 'wlan1' link is up

I brought this router to my room where there are just a few devices and wow, no memory leak, it runs stable. At this time of writing the uptime is more than 7 days. So I guess the memory leak may be related to the number of connected devices, the signal/distance that makes clients keep associating to.

In summary, I think 5Ghz wifi has hardware defect. However I config it could not work. I read some people report that their 5Ghz works but 2.4Ghz does not, and vice versa. My question is WRT series is a famous, quite expensive router but their wifi chip is not much reliable. My WRT3200ACM lasted for more than a year before having these problems although running in a normal temperature room with good ventilation. On the box it says "Developed for use with OpenWRT and DD-WRT" but the wifi driver and firmware were abandoned for a long time.

Losing 5Ghz band is a big disadvantage over 2.4Ghz band. Now I have a plan to set this up as the main router at my home's basement because without wifi this router is still a beast and I rarely need wifi in there.

Has anybody experienced similar issue with me? Did you fix it successfully? Thanks.

What version of Open WRT is on the router?
(Just curious..)

I installed the stable version 19.07.3, and htop package. Just keep it as most simple as I can.

I thought some of the Pros would weigh in by now...