Users needed to test Wi-Fi stability on Linksys WRT3200ACM & WRT32X on OpenWrt 21.02

Model Linksys WRT3200ACM
Firmware Version OpenWrt 21.02.0 r16279-5cc0535800 / LuCI openwrt-21.02 branch git-21.231.26241-422c175
Kernel Version 5.4.143

I also suffer from the wifi random cutout issue. At a first glance, wifi seems to work, but after some time (seconds) wifi blocks for a few seconds up to a minute. I tried with v21.02.0-rc4 and now with v21.02.0.

Today I figured out, that wifi seems to work stable (I tested this for the duration of a few hours), as soon as I disable WMM (option wmm '0'). When I start WMM again, the cutouts appear again. I tired several times, always with the same result.

The only sideeffect I experienced so far is, that the maximum wifi transfer speed is reduced to approximately 16Mbit/s (which I could live with), when WMM is disabled. Wifi transfer speed is much higher (40Mbit/s, which is my maximum ADSL speed), when WMM is enabled, but wifi is practically unusable due to its instabilities.

Please accept my apologies, in case this has been reported before.

Could anyone please verify this behavior?

I am an Openwrt-novice, but I wonder if this behaviour is also related to the 88W8964 firmware issue, or if it could be unrelated.

Best
Alexander

This is interesting. If more users can reproduce this, it might help developers and more experienced OpenWrt users to determine where this issue might have been introduced and when. I will test this out later on my 21.02 partition as well.

WMM being an interoperability issue for mwlwifi is a known fact, but it's also a mandatory feature for 802.11n an up (without it, you're limited to 54 MBit/s)

1 Like

that is really sad. that still the best openWRT router, but wireless are so. and one day i hope we will change that.

Samsung ultra 21 we have in our house mtu was causing issues reduce it to 1480 see if that helps

Also downgrading the wifi driver on my wrt32x helped alot I wish they would just remove the upgrade stick with the old in future distros...any chance you could do this with the 21.02 official release ?

As described in my last post, disabling WMM fully re-established wifi's stability on my WRT3200ACM running Openwrt 21.02.0.

Nevertheless I encountered the following obstacle:

After disabling WMM, Pocket PC Clients (Windows CE) are still being displayed in the list of associated stations, but it seems, they cannot communicate with our server as they formerly used to, when WMM was enabled. I also tried on a WRT3200ACM with Openwrt 19.07.8, giving the same results with Pocket PCs. Setting up a second wifi with WMM enabled for these clients helped, but is of course a poor remedy.

If there is a way to get around this, I would appriciate any advice. Otherwise I might be forced to downgrading to 19.07.8 and wait until 21.x has been cured against these issues.

If your reference is to backup to the previous BLOB on 21.x

I also tried this BLOB, but the cutouts still remain on 21.2.0 until I disable WMM. The more I try, the less I understand.
I have no idea, what could be different in my environment. I already re-installed a factory-image to be confident, that there is no obscure configuration which coud cause these issues, but unfortunately nothing changed.

Is the mwlwifi issue regarding WMM specific to ESP-based devices only?
Or are there other WMM issues with mwlwifi as well?

I wish that it was easier to understand why 21.02 releases are triggering these wireless cutouts. I have no idea if WMM is the source of this current issue or not, but disabling WMM is not very feasible or worthwhile these days. Going back to 19.07 may be the last resort for WRT3200ACM/WRT32X devices unfortunately.

No.

Yes, interoperability with other chipsets has been a problem with this chipset and its firmware/ driver from day one. Some more pronounced than others, Espressif's chipsets just seem to be the most common and obvious/ severe ones.

2 Likes

@slh Thank you for confirming. It's unfortunate that the developer of the mwlwifi driver wasn't able to fix many of the remaining issues. Also unfortunate that we don't have any open specifications for the firmware blob either.

I have 2 WRT32X's - one as main router one as dumb AP - both running 21.02 Release - both have WMM enabled for my IOT SSID - I have over 30 Espressif (some 8266 and some 8285) based lights, switches and sensors - they all work 100%

So, I'm not sure where this idea that ESP devices don't work with WRT32X comes from

It comes from dozens of people (myself included) that have reported this issue... I'm glad to know that it's working for you, some ESP clients work and some don't (perhaps different firmware?), and looks like you were lucky.

Oh, right - yeah - tasmota updae some time back made it good

Don't give up quite yet. I still hold out hope that if it worked in 19.07, then someone will figure out how to get it to work with a patch to 21.02.

I was thinking about this recently.

Has anyone attempted a bisect between David's last build (which can be found here) and 21.02 to find the offending commit where things broke?

I'm not averse to doing this myself, but I wanted to ask in case anyone has narrowed it down to some range of commits?

@adworacz That is a great idea. I do not have the skills to do the bisecting, but I do recall the timeline of events quite clearly from the forum itself. Hopefully pinpointing these timeframes may be beneficial for anyone capable of bisecting.

May 14:

First comments on Divested-WRT thread regarding wifi cutouts; there were also reports in 21.02 RC1 thread as well at the same time frame. So that means it hit 5.4 and 5.10 kernels at the same time because Divested-WRT was 5.10 at the time if I recall correctly.

Divested-WRT: No-nonsense hardened builds for Linksys WRT series - Community Builds, Projects & Packages - OpenWrt Forum

Divested-WRT: No-nonsense hardened builds for Linksys WRT series - Community Builds, Projects & Packages - OpenWrt Forum

Those were just the first two, but of course many more were to follow.

Two days prior, 20210512-00 release of Divested came out.

According to the changelog (https://divested.dev/unofficial-openwrt-builds/mvebu-linksys/CHANGELOG.txt):

20210512-00
- update to 6713fe030fca32fc3d5ad9761f3b2f96501aedd6
- [upstream] mitigations for https://www.fragattacks.com

Or unless those users were quick to pickup 20210514-00:

20210514-00
- update to 7fea9d9f5dd282a7049d77cc6b75e0a703ead26c
- [upstream] update to kernel 5.10.37 (security and bug fixes)

Regarding 21.02.0-rc1, we know that was released on 26 April 2021 officially. I will dig into the 21.02.0-rc1 thread later for relevant comments.

I found this comment from April 11th interesting:
Divested-WRT: No-nonsense hardened builds for Linksys WRT series - Community Builds, Projects & Packages - OpenWrt Forum

A significant DSA roaming fix went into 21.02 branch yesterday for the MV88 switches in all mvebu routers (along with the new wireguard). It's in the new snapshots and will presumably be in this next build here. I wonder if that'll help some of these less common network setups people have here using external switches. Some really nice polishing is happening on 21.02 branch and it's starting to look like 21.02 will be a very solid release once its done.

My main purpose right now is narrowing down the timeline based simply on comments from this forum. With the timeline, hopefully it can help narrow down which commit may have caused these issues or assist in bisecting if necessary.

I will dig in some more later when I have more time.

@mmortal03 Good call on not giving up on this yet. Your post gave me determination and energy to do some more research into this issue.

1 Like

I don't know if this commit is the cause or not, but I'm putting it here so that I don't lose it. It's the commit for kernel: DSA roaming fix for Marvell mv88e6xxx. Link: kernel: DSA roaming fix for Marvell mv88e6xxx ยท openwrt/openwrt@f1158fb (github.com)

Also, I have 21.02-SNAPSHOT downloads from 2021-03-06 and 2021-03-17 that I never ended up testing. I ended up waiting until RC1. However, those builds may narrow things down further since they may have been built before the commit was introduced that caused the wifi cutout issues.

The only problem is that I don't know if I will have time to test those builds out before next weekend to determine if I can reproduce the issue.

Thanks for the deep dive @WildByDesign!

What little information I can dig up from David's builds, his last build included changes from May 17th, 2020. If the timestamps are to be believed, he posted his last build r13342 on May 23rd, 2020.

So that may be our start date that we can bisecting forward from. Ideally we identify when these issues first started cropping up, which WildByDesign has already started to do. That will give us a potential end date to bisect backwards from.

Once we have that range, it should be just good ol' binary search to nail that down.

I can provide test builds as necessary.

Edit: I believe this is David's last post with the announcement for r13342: Davidc502- wrt1200ac wrt1900acx wrt3200acm wrt32x builds - #4925 by davidc502

Edit2: Saving this for later, but this will simplify building specific hashes, and keeping feeds in sync: