I’m saving the individual commits into patches, then we can bisect them.
1 Like
You would have to add a hacked up revert patch to the mac80211 package (locally), how feasible that is, can only be answered once anyone tries.
1 Like
Hi. I'm also affected by this wifi bug in WRT3200ACM. I'm happy to have a working wifi again.
I'm currently struggling with building a working image with kmod-rt2800-usb enabled, since i need this driver for my usb dongle with external wwan antenna.
I've tried to install it on working builds from this thread, even with --force-depends, no success.
The patch from @adworacz blog post is removing rt-2x00 / ralink module from mac80211 completely, so there is no kmod-rt2800-usb.
I'm currently building an image whith a modified patch (wich I hand rolled, basically re added the rt-2x00 parts to the makefile again). will see if it works out. (maybe this also can help to narrow down the issue of the bug, since I'm manually re-adding files from the 21.02 branch. Will see if this works out 
I'm happy to help out for further testing on Images for WRT3200ACM. (I've also got the WRT1900AC, with simmilar wifi cut outs). Also got iphone, ipad, macbook to test out. My Thinkpad is mostly unaffected from this bugs, as others stated allready.
That's one of the reasons why this workaround isn't a solution in any way, shape or form. rt2800usb hard-depends on the corresponding mac80211 package it has been built against, the same goes for anything else provided by the backports package.
I haven’t saved the modules in my drive, but the instructions I’ve provided do not disable any drivers, and builds what I believe should a compatible package.
Removing and installing the modules @cotequeiroz provides works like a charm!
I have an issue that maybe someone here might know about.. I cannot seem to get a guest network working for the life of me, and was wondering if anyone has successfully done so on these devices (3200acm/32x).
I have tried both the CLI and LuCI methods [https://openwrt.org/docs/guide-user/network/wifi/guestwifi/start], but can not get the network to associate.. is this a device specific issue or am I just doing something wrong? The CLI method is a one-shot, so it should theoretically work since it doesn't leave much room to misconfigure something.
I am trying to isolate untrusted devices from my network.. if anyone knows what's wrong, I would be very grateful.
Thanks, and apologies if this is the wrong thread to ask in. Just figured it might be the best place since we are using the same hardware/firmware.
Wi-Fi connecting users to an isolated network works fine on my WRT32X.
Remove all existing SSIDs and create them from scratch if it still won't work.
Wireless
config wifi-device 'radio1'
option type 'mac80211'
option hwmode '11g'
option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
option country 'US'
option cell_density '0'
option channel '11'
option htmode 'HT20'
config wifi-iface 'wifinet2'
option device 'radio1'
option mode 'ap'
option ssid 'arinc9'\''s People'
option encryption 'psk2'
option isolate '1'
option key 'password'
option network 'guest'
Network
config device
option type 'bridge'
option name 'br1'
config interface 'guest'
option proto 'static'
list ipaddr '10.13.37.1/24'
option device 'br1'
Firewall
config zone
option name 'guest'
option output 'ACCEPT'
option forward 'REJECT'
list network 'guest'
option input 'REJECT'
config forwarding
option src 'guest'
option dest 'wan'
config rule
option name 'guest - Allow DHCP requests'
list proto 'udp'
option src 'guest'
option src_port '67-68'
option dest_port '67-68'
option target 'ACCEPT'
config rule
option name 'guest - Allow DNS queries'
option src 'guest'
option dest_port '53'
option target 'ACCEPT'
UCI's way of including wireless interfaces on networks should change but that's really off-topic for me to speak of it here.
I've joined the party here and have flashed to 21.02.1. I have an iPhone in the house and another piece of equipment that has given OpenWRT issue in the past. But I have yet to see any cutouts, even after throwing iperf3 at it. Maybe I am lucky.
But if not, I am prepared to bisect mac80211 between kernel v5.8-rc2 and v5.7.5. I am taking the opposite approach from cotequeiroz: first, I got the backport program to work and it is producing the driver tarballs that I can point package/kernel/mac80211/Makefile at. I can then use git bisect to drive the search between v5.8-rc2 and v5.7.5. If it would be of any assistance to any of you to have these tarballs or the compiled OpenWRT mac80211 packages from them I can do so.
2 Likes
I've got a list of patches to bisect here: https://github.com/cotequeiroz/openwrt.git
The list is longer than I had anticipated (96 patches), because of support files from outside of net/mac80211 but it is not that bad. It should only need one or two extra bisect steps. There are some useless patches, but doing it like this ensures the end result matches vanilla backports-5.8-rc2-1 (except for drivers, which are disabled).
I won't be able to test this myself, so I'm counting on brave volunteers.
In theory, you should not need to rebuild the whole image & flash it. You will need to remove kmod-mwifiex, since it is not being built. You'll lose mlan0. Then you just need to copy the modules (cfg80211.ko, compat.ko from kmod-cfg80211; mac80211.ko from kmod-mac80211; mwlwifi.ko from kmod-mwlwifi) to do the bisect. To ensure that theory meets practice, start with 09700c1f01, and make sure that using just the modules will work. Running this from your source dir should do the job (adjust IP as needed):
scp build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/{backports-5.7.5-1/.pkgdir/kmod-{cfg,mac}80211,mwlwifi-2020-02-06-a2fd00bb/.pkgdir/kmod-mwlwifi}/lib/modules/5.4.154/* root@192.168.1.1:/lib/modules/5.4.154/
Then reboot the router. Be sure to run make package/{mac80211,mwlwifi}/clean in between runs.
Of course, force-installing the packages, or flashing the whole image should work. I can't produce the same magicversion as the official builds, since I need to disable building most wireless drivers to reduce the number of patches to bisect.
There are a number of limitations to building this. Since I have disabled most wireless drivers, you can't use the official config.buildinfo. It will fail because some of the routers built with the default config will need ath10k-ct drivers, and will fail to install. So, I've provided a default config.builldinfo at the topdir, that will build images just for the Linksys routers. Just run cp config.buildinfo .config && make defconfig to use it.
I have not tested the repo with the feeds. You should not need the feeds to build just the modules. If you want to build an image with luci, then you'll need at least the packages & luci feeds. I haven't event build-tested this with feeds enabled.
Cheers
Edit: corrected the scp line.
2 Likes
I've gone ahead and updated the git repo I created before with a custom build script and Dockerfile, that automatically builds and extracts the 5.7.5 kernel modules.
See here: https://github.com/adworacz/openwrt-linksys-wireless
You can see the kmod-* packages and corresponding checksum files listed here: https://github.com/adworacz/openwrt-linksys-wireless/tree/main/releases/21.02.1
I've also gone ahead and updated my post with new build instructions based on @cotequeiroz 's work, which removes the need for patches.
I'll be in and out of town for the holidays, but I might be able to build custom images for easy flashing based on @cotequeiroz 's post above, if people would appreciate that:
2 Likes
@cotequeiroz I just kicked of builds for 5 images, interspersed commits between 1-96.
One thing I noticed is that the mwlwifi/mac80211/cfg80211 packages are not showing up in the resulting *.manifest file. But, the corresponding packages do show up in the bin/target/.../packages directory. I haven't flashed this on my router to confirm if these packages are included in the image or not yet, but the missing entries in the *.manifest file has me suspicious.
This is building with your config.buildinfo file provided in the repo, with luci-ssl and advanced-reboot turned on.
Is this expected? I thought the *.manifest file recorded every package included in an image, so it seems suspicious that our main test packages aren't listed in that file.
1 Like
Yes, it is expected. Because of CONFIG_TARGET_MULTI_PROFILE=y. The .manifest file will only have the "common" packages. The device-specific packages, which are taken from target/linux/mvebu/image/cortexa9.mk will add kmod-btmrvl kmod-mwifiex-sdio mwlwifi-firmware-88w8964 to the list. The firmware package will add mwlwifi as a dependency. mwlwifi will add kmod-mac80211, which finally adds kmod-cfg80211.
You may check the installed files under build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/target-dir-*: In my build, there are two target-dirs: target-dir-0fc15bb1, for the wrt1* routers, and target-dir-70ea5744/, for wrt3*. The kmods are there:
$ (cd build_dir/target-arm_cortex-a9+vfpv3-d16_musl_eabi/linux-mvebu_cortexa9/target-dir-70ea5744/lib/ && ls -l modules/5.4.154/{mwlwifi,mac80211,cfg80211,compat}.ko firmware/mwlwifi/*.bin)
-rwxr-xr-x 1 equeiroz devel 451040 Oct 24 06:01 firmware/mwlwifi/88W8964.bin
-rw-r--r-- 1 equeiroz devel 321200 Oct 24 06:01 modules/5.4.154/cfg80211.ko
-rw-r--r-- 1 equeiroz devel 3228 Oct 24 06:01 modules/5.4.154/compat.ko
-rw-r--r-- 1 equeiroz devel 620260 Oct 24 06:01 modules/5.4.154/mac80211.ko
-rw-r--r-- 1 equeiroz devel 191160 Oct 24 06:01 modules/5.4.154/mwlwifi.ko
Edit: Just to be clear. I'm not building kmod-mwifiex-sdio, since it is part of the mac80211 drivers that have been disabled by a55d7aaa29 mac80211: disable driver packages. The third radio (mlan0) will not appear, and this is to be considered normal. Once we find the breaking commit, and are able to fix it, the drivers will be enabled again and it will return 
.
1 Like
I'm going to kicskstart a bisect. My original config.buildinfo was not including luci, which was not intended. Should I rebase my fork to correct the config.buildinfo? It may cause problems with people who already started their own bisect, if they run git pull.
1 Like
I've saved the config into a new 'bisect' branch.
I'm going to publish my images/packages/modules in my google drive, under the bisect folder. Here's a link to the directory: https://drive.google.com/drive/folders/1am6HgA2JGYaakxnRXjhk4vExUYkIRmzT?usp=sharing
I'm saving each build in a numbered folder, starting with 0000, meaning no patches were applied. The second number is the commit id, from the bisect branch.
Please test 0000 to ensure it is good, then the last one, 0096, to ensure it is bad. Post the results and I will mark them to continue the bisect. I'll be renaming the folders to indicate the build(s) that need to be tested, and the results of the ones already tested.
When reporting the results, please indicate the model tested (WRT3200ACM and WRT32X only!), and the method used: flashed the image/reinstalled the kmod-* packages/copied the *.ko modules. This is to sort out eventual conflicting results. If using the packages or modules methods, do not forget to restart the router!
1 Like
@cotequeiroz I really like the way that you have organized this testing and especially the layout on the Google Drive. Excellent work!
I will test a bunch of these later tonight and will follow your detailed instructions as well regarding device, method of install, etc.
I am creating a spreadsheet right now for my own testing purposes following the numerical order of your patches and modules. And I will do a lot of testing tonight and will share details at that time.
If the packages and modules fail, for whatever reason, I can also flash sysupgrade builds as a last resort. I am hopeful that the packages/modules will be successful though because it will make testing faster.
2 Likes
If you replicate my folder structure, you may use this to skip commits that don't change output:
$ for f in 00*; do [ -f $f/kmods/mwlwifi.ko ] || continue; echo $(basename $f): $(echo $(cd $f/kmods && sha256sum *) | sha256sum); done
0000-2dc174b50d: 6210798b8243a8ea99fd631456e9583c4e55f541a5462bbc58bb83c9ad2ad115 -
0001-c9902c6598: 6210798b8243a8ea99fd631456e9583c4e55f541a5462bbc58bb83c9ad2ad115 -
0002-fe348f2d70: 6210798b8243a8ea99fd631456e9583c4e55f541a5462bbc58bb83c9ad2ad115 -
0003-b22b8b3c09: 8f5fac566e04ff946a3bbfd48cada32072da9060d343a5f75757e122d5daf5e7 -
0004-c7cb3f22af: a1e8514bc16e3bbbd0d0235b40e4e2b8cd10a6323a2c911133770aab12a6b1da -
0005-8010c235e7: b72bae0e887b0cab472909bdd1f6ba8ee6dabf5b1fa895e23e288356c7337c5e -
0006-4d97da2d07: 1e470710ff5ea1d20d63e5e17cb26f9e1f707bf6aafe865a367464e0387da6dc -
0007-908caea8a4: 76588cb4cf43f851d0524429cd33f17214205f5053f2042b84e53201dc4e70c4 -
0008-036e7eb6e1: 842760a538ef10eb75b0a1cc2f909e27c1a1424d24b4874a09a5ddf32893c4ab -
0009-381cf99105: 97da1b98cd13365e16ee6164e3b8384e7e201583e878b98c3e336f563ffe9551 -
0096-3fae677e85: 4257c971c98fc944330ab90817c22394d60dd4b366f909cff35eeb778aac4b57 -
Edit: I'm marking the directory names with -nochange to indicate that there is no change in the modules compared to the previous commit/patch.
2 Likes
Haha I was about to start the same, but looks like you're already on it.
2 Likes
So far, so good. I downloaded all of the compiled builds/packages/modules that I need for testing hours earlier. Now it is time to get to it.
So far installation via ipk is successful. Since ipk install is successful, I would like to stick to this method for now until we can narrow down the offending commit. After that, I can spend some more time replacing kernel modules manually and whatever other testing we need.
I will continue to update this post:
Wireless Install
Cutouts Method
0000-2dc174b50d-test-should-be-good No ipk
0007-908caea8a4 (1) No ipk
0015-7a9802d4d5 No ipk
0016-41441e4e5c Yes ipk
0017-92e3408add Yes ipk
0020-f12f71bf9a Yes ipk
0025-b7e515dfa3 Yes ipk
0052-aa537df216 Yes ipk
0091-2a2d4362ee (2) Yes ipk
1. 0007-cfg80211-change-internal-management-frame-registration
2. 0091-cfg80211-fix-management-registrations-deadlock
EDIT: Removed my theories. Just testing.
1 Like
Offending commit: add 0016-mac80211-agg-tx-refactor-sending-addba.patch · cotequeiroz/openwrt@41441e4 (github.com)
Time for sleep. I will review my testing tomorrow if necessary, but as of right now, I am confident in these results. But also super tired right now.
4 Likes
For testing purposes, we might be able to disable AMPDU in the driver before compiling it:
I assume that this would have performance implications.