Users needed to test Wi-Fi stability on Linksys WRT3200ACM & WRT32X on OpenWrt 21.02

Hey guys,

First of all, awesome thread, I also had the same issues.
Got the mac80211 5.7.5 running on 21.02.1 , and so far no issues, but still no 24h yet.
If this shows up to be stable i'll share the images
The Images are from my setup , and contain more than the stock packages, but I can remove them if you want to.

Cheers!

1 Like

@kevdel @adworacz Possibly @eduperez can give some suggestions on compiling to a specific known stable build and creating corresponding ipk package for this mac80211 5.7.5?

I recall once or twice he had those package hash issues when sharing ipk packages of mwlwifi drivers compiled toward specific stable build kernels. Slightly different situation, but the logic to solve this hash issue might be the same.

OpenWrt kernel packages' installation is tightly tied to the hash of all included kernel modules and options in the config at the kernel compialtion time, so it is really hard to add kmods to third-party builds.

The main option is to checkout the relevant OpenWrt source code, do the necessary modifications, and compile everything from scratch with the full toolchain at the same time, so that you get a full firmware image with the modified package.

Alternative is using the release SDK. I think that @eduperez has been using the release SDK for the "stable build driver" compilation, so that the kernel in the SDK used for the compilation is the actual release kernel. And those kmods can be installed to the official release (as the kernel has been the same).

1 Like

I have some tips for you in this matter.

I've had success running master on WRT3200 with WPA2-EAP. I noticed that the lock ups happened after a kernel warning:

Rekeying PTK for STA xx:xx:xx:xx:xx:xx but driver can't safely do that.

This warning was introduced in linux 4.20, so by itself, the rekeying change that came with the message does not cause trouble.

I have created a package in https://github.com/cotequeiroz/customfeed/tree/master/mwlwifi-thaw-clients that greps /dev/kmsg, and uses hostapd_cli to disassociate the client. I don't know why it works or why the message does not appear with WPA2-PSK, and haven't really looked into it.

Another patch is needed to add a newline to the message, so that it will be sent to kmsg right away without buffering. I have a branch in my fork of openwrt.git, rebased to 4b0f877, that has the three commits I'm mentioning here: https://github.com/cotequeiroz/openwrt/commits/mwlwifi

As for the kernel hash, it is tied to the kernel build options. It is generated in include/kernel-defaults#L121, and gets used to define LINUX_VERMAGIC. I haven't tested it, but I think you can set it from the make command line. The conventional way of matching this is to not have any CONFIG_KERNEL* differences between the official config.buildinfo and in your scripts/diffconfig.sh output.

If the kernel hash issue does not get resolved, or if you're going for a custom-build, I have two commits that will change the arch of the armada-385 series to use the neon extensions--always a hot topic. The openwrt_core packages (https://downloads.openwrt.org/snapshots/targets/, notice /targets/) , where the target-specific packages live, will have to be built locally. The rest of the standard repos in /etc/opkg/distfeeds.conf (from https://downloads.openwrt.org/snapshots/packages/, notice /packages/) can be used, as the packages for the arm_cortex-a9_neon arch are built by the bots.

I add -funsafe-math-optimizations to CONFIG_TARGET_OPTIMIZATION generate neon fp code, and keep my own repo of packages, but I understand it is not for everybody. Without it, though, the performance gain will not be too relevant. Some of the most relevant packages, such as openssl, will check for neon presence at run-time, using optimized code. Kernel crypto modules are already built to use neon, so IMMV.

1 Like

Thanks for the build, I've been running both builds r16325 && r16335 since you released them and it seems to do the trick.

I'm running WRT32x, didn't spot any issues with both builds.

Also thanks for the patch and blog post, very informative.
Will continue following.

Thanks again,
Del.

1 Like

Is there any benefit to the following?:

# Workaround for PTK rekey issues
#
# PTK0 rekeys (rekeying the PTK without "Extended Key ID for Individually
# Addressed Frames") can degrade the security and stability with some cards.
# To avoid such issues hostapd can replace those PTK rekeys (including EAP
# reauthentications) with disconnects.
#
# Available options:
# 0 = always rekey when configured/instructed (default)
# 1 = only rekey when the local driver is explicitly indicating it can perform
#	this operation without issues
# 2 = never allow PTK0 rekeys
#wpa_deny_ptk0_rekey=0
1 Like

Huhhhhhh. That makes zero sense, but the sizes match up so I'm not sure what's going on that would give different hashes between my router and yours. I'm going to try flashing the official 21.02.1 image this weekend to see if I can reproduce and fix.

Ideally, I'd like to avoid all of the extra work that the others post above, as it's a damn pain compared to just swapping the hashes manually.

If that doesn't work for some reason, I may need to do as hnyman says and create an exact image replica, albeit with the modified mac80211 package, based on the OpenWrt 21.02.1 source code tarball:

It certainly sounds like If you set it to 1 or 2 (it won't matter here), then it will do exactly what my script is trying to do in a much cleaner way. I'll give that a try. Thanks.

1 Like

Unfortunately, I couldn't use wpa_deny_ptk0_rekey=1 or 2. One of the notebooks can't connect at all unless it is set to 0. Windows shows an Intel Dual Band Wireless-AC 8265. There wasn't enough time to evaluate Apple devices. So I'm back to patch + script.

@adworacz I was feeling brave this morning and before the rest of my family woke up, I decided to install build r16335 (21.02.1 - kernel 5.4.158 - mac80211 to 5.7.5). By brave, I mean my kids had to do online school today and be constantly connected to Google Meet for 8+ hours on two Chromebooks. This had the potential to be a bad decision. Plus an iPhone with heavy usage and my work laptop. Also some streaming of YouTube videos on two iPads for certain periods of the day.

Zero issues to report. As a matter of fact, everything was faster than with Davidc502's last build. DHCP was handing out IP addresses lightning fast like I've never seen before on OpenWrt. LuCI is super snappy and always responsive. General browsing is great.

I am far more impressed than I had expected to be.

1 Like

Unfortunately I have some bad news about the wireless cutouts. I feel terrible because my last post was extremely optimistic after having success throughout the day. But I just had a chance to really look into things more.

It looks like we are no longer having wireless cutouts that make the wifi on the client device stop working entirely which required toggling the wifi off/on on the client device.

Well, it looks like we have traded in those bigger wireless cutouts for thousands of mini cutouts that are able to resolve themselves. One after another, after another...

You wont likely notice it on YouTube because of the buffering, but once you start to load a few different live streams with no buffering and from various different sources, that is where you see non-stop stuttering of videos. The loading of other web sites during those times are painfully slow.

I had no choice but to switch back to Davidc502's last build for now. I loaded all of the live streams from the same various (dozen or so) sources, and I have not experienced any stuttering of videos.

I am so frustrated right now. Sorry everyone.

Are there logs for this or could it be a client side behavior?

I moved to that build also and I've had rock solid performance all day on streaming sites, work vpn, file downloads and uploads.

Definitely something to do with the iPhone XR. Everything was good all day long while the XR was powered off. It wasn't until the evening when I decided to test it because that is the device that always triggered the cutouts throughout all of this testing.

Your experience is exactly how mine was. Up until the iPhone XR connected. I will have to sleep on this for now and have a fresh go at it again tomorrow.

I have been doing some more tests this morning with this r16335 and my wireless speed is far slower than usual. I am consistently getting between 22 Mbps and 35 Mbps across a variety of different online speed tests. Typically, my wireless speed is consistently around 110 Mbps and 125 Mbps.

The wireless settings that I use have remained the same throughout the weeks of testing to ensure the best consistency possible.

So I have to look into this further because the wireless settings on the router and client device have remained the same. I'll share more details as I do more testing throughout the day.

Has anyone else experienced slower overall wireless speeds on this build?

EDIT: That was with 2.4GHz band.

Testing the 5GHz band, I am getting 14 Mbps and throughout the testing it keeps dropping down to the Kbps dial-up type of range.

EDIT2: This speed testing this morning is actually on my laptop that has been immune to the wireless cutout issue throughout testing. The iPhone XR is powered down.

This all explains why I could not stream live video last night.

I've been following along with this thread because my WRT32X was experiencing the wifi cutouts with an old iPhone 7 Plus. I managed to compile the kmod-mac80211 package based on v21.02.1 and your patch and it installed over my current v21.02.1 using the --force-downgrade flag, but wifi just didn't work after i rebooted. I only compiled mac80211 since I have an old system available to compile images and it takes me a day and a half to compile the entire package. I'm currently compiling the whole v21.02.1 package with the modified mac80211 package in case it just doesn't work by compiling it standalone. Compiling this way has worked for me in the past when creating downgraded mwlwifi packages, so I'm thinking we need to replace more than just the kmod-mac80211 package.

Yeah that's possible. I think mac80211 officially depends on a cfg80211 package as well, so we may need to install both. I'm trying to make a build using the official 21.02.1 config.buildinfo file to see if that unlocks the ability to install mac80211/cfg80211 easily on existing 21.0.2.1 images.

@WildByDesign My wifi speeds have been rock solid on the latest builds. This is testing on an iPhone, desktop, and a laptop, all on 5Ghz.

1 Like

Hello,

Thanks for the builds! I'm trying r16325 but sadly

deauthenticated due to inactivity (timer DEAUTH/REMOVE)

Is still happening with iPhones... on WRT3200ACM

But are you getting wireless cutouts? Those logs haven't shown to be really material in replicating the wifi cutouts, they seem to be more of a redherring than anything.

I've gone ahead and created packages builds using the official OpenWrt 21.02.1 config.buildinfo for kmod-mac80211 and kmod-cfg80211 (a dependency of kmod-mac80211).

You can find the packages here: https://openwrt.austindw.com/linksys-wrt/working-wireless/testing/21.02.1-config.buildinfo/

There's also a checksums file for those who want to verify their downloads.

I manually checked the control file in each package and they both show a kernel dependency hash of 345456c5234da787004df250a0ea7bbd so they should install pretty cleanly.

I'm not certain if the kmod-cfg80211 package is also needed or not, but based on @trinidude4 's comments, it's possible that the kmod-mac80211 package wasn't the only thing that needed to be changed in order for wifi to work.

Please try installing these packages using an official OpenWrt 21.02.1 image. I'll try and get some time this weekend to flash an official image myself to do some testing.

1 Like

I installed the packages you created and I get the same problem with wifi not starting. I forgot to copy the system log before rebooting, but it said something about PHY not being defined for radio0 and radio1. I can't try it again right now to get the exact messages because I have to leave the router up and running for a while.