Unusable 5GHz channels due to misconfigured nearby networks for Macs and Intel wifi chipset clients

For anyone debugging 5GHz performance - here is a possible cause of much anguish:

Here is the text of the Apple's diagnostic output:

"A nearby wireless router has been detected which is identifying itself as originating from a country which conflicts with your current settings. This may prevent your Mac from automatically re-joining a previously joined Wi-Fi network.

Certain wireless routers have the ability to identify the country they are designated to work in, this is called the Country Code. Wireless routers should only be used in the country they were originally obtained from. Failure to do so can result in performance and reliability issues for nearby wireless clients.

If possible, contact the network owner to resolve the problem."

From - https://support.moonpoint.com/os/os-x/wireless/conflicting-country-code/

I think this also impacts Intel wifi card chipsets on client devices, where this is implemented in their firmware, although I've done less debugging on this.

In the network that I was trying to debug:

  • The United Kingdom has 6 distinct 80 MHz wide channels available in the 5GHz spectrum (although one will commonly be unavailable due to radar detection).
  • The site has 6 OpenWrt 802.11ax access points there, with country code correctly set to GB and frequencies allocated to minimise co-channel interference.
  • "Neighbor reports" (802.11k) are used to assist client roaming - the configuration is monitored and I'm satisfied that it's working correctly.
  • The site is located on the top floor of an isolated tall building in the middle of a town, and the building's windows lack any wifi-opaque coatings (e.g. solar reflecting, or low emissivity energy saving coatings), making them wifi-transparent. Beacons can be received from several hundred surrounding wifi networks (almost all on the 2.4 GHz band).
  • Macs show intermittent poor performance when connected to two different access points.
  • It appears that ~5 nearby wifi access points are misconfigured and report DE, NL, or US country codes instead of GB in their 802.11d beacons.

I have yet to fully debug this but I think the presence of DE and/or NL country codes are causing the Macs to reduce their transmit power levels to 25 mW on channels 138 to 165 from the GB limits of 1000 mW or 200 mW (channel dependent).

Extremely high packet loss is experiences except when the Mac are more than a few metres away from access points using the 132 or 149 (80 MHz) channels.

Because the access points still transmit at the full GB legal level, so the Macs see a strong receive signal and do not roam away to a different access point.

Possibly some mitigation would be possible using a daemon to use 802.11v to force Macs which are behaving like that to roam to a different AP.

Next week someone might start beaconing RU CN or ID and knock out another third of 5 GHz spectrum for all surrounding wifi clients, and someone acting maliciously could knock out nearly all of the 5GHz band for a big chunk of the town.

It appears that the stock firmware on many Asus routers sends out country code DE in their UK market wifi access points (not grey imports). They know about the issue and don't seem to care.

4 Likes

Should your client device not use the country code of the router it has connected to?

I am in full agreement with you, but it seems that neither Apple nor Intel share our opinion.

This seems unreasonable to me. I can only assume some national laws were interpreted pessimistically within both companies.

I suspect the motivation is to prevent the devices that they have sold to end users from transmitting in violation of national laws under any circumstances when it could have known otherwise.

The manufacturers appear to offer no override mechanism (well I can imagine a few but none of them are very practical).

If they wanted to retain the behaviour, then changing the lock-out to a super-majority vote (say go with the opinion of >80% of access points to eliminate rogues) would seem reasonably safe, but it's closed-source code, so with no influence over whoever maintains the code in question, this seems unlikely to get fixed.

If that is the case I do not know how OpenWRT can do anything about this, assuming OpenWRT is sending the right country code, but perhaps you need to double check that

I posted this thread mainly to help others diagnose and workaround similar problems.

Having said that, whilst OpenWrt can't directly fix the broken third party APs or fix the ultra conservative client behaviour, I can think of a few possible mitigations:

  • As I said previously it may be practical to use BSS Transition Management (commonly referred to as "802.11v") to detect and fix a subset of the problems which the situation can cause.
  • OpenWrt could flag any mismatching country codes when it carries out a channel scan.
  • In addition to the above, a more advanced implementation could calculate which channels will be unusable as a result of nearby misconfigured access points, and offer advice on reconfiguring the local access point to avoid problems.
  • An option could be added to carry out evasive manoeuvrers automatically (i.e. reconfigure the 5 GHz channel selected by the access point).
  • The above would only work if the access point is able to receive the same bogus beacons that the client is receiving.

Unfortunately the above would require development work on the order of days to weeks.

4 Likes

Considered a complaint to the spectrum regulator (OFCOM?) on the basis that the non-GB CC APs are a) interfering with the compliant operation of other equipment, and b) seem likely to be at least technically unauthorised transmissions when not using the GB country code?

1 Like

The US one is in legal violation, the DE/ NL ones not so much (not exceeding legal limits in the UK, just allowing less than possible, which is legal - not good, but legal).

1 Like

How could a wrong country configuration produce a high packet loss? Have you found the connection?

If the country code advertised is wrong, but the channel used and transmit power are in the legal limits you will have a hard time doing something from a legal point of view.

the defense will even invoke the fact that you do not have to advertise the country code and wrong country code or no country code is the same as long as the channel used and transmit power comply with the law.

if the client ignores local country (in whatever way the device is setting this up) and the country advertised by the ap it is connected to (assuming a country is even advertised) and is not picking the most restrictive from the 2 and it just decide to random check nearby aps that it will never connect to and random pick the most restrictive from all the country codes advertised in the band then the client is kinda malfunctioning if u ask me, because someone can just advertise a not existing country code and in this case the client should plain and simple refuse to work on that band because not existing country code can be translated into not allowed to operate in that band.

the main question here is how would a client should react when it connects to an AP that is advertising a country code that does not exists or are not letters (i saw this last case, equipement gave by the ISP…). it casually ignores it? checks in the list with country codes, not there, default to not allow to operate in that band? because it is invalid it decided to use the local set country? it defaults to the most restrictive that still allow it to operate in that band?

and in the case of ISP equipment doing funny things regarding the wifi who is at fault? the user that does not even own the device and can not even update the firmware? the ISP because he is the owner of the equipment and also the one doing the mantenance? u will have legal problem here to, u can say the user, but the user does not own the equipment and can not do firmware updates even if he wants.

and we can go on with the legal aspects if u want. lets say someone is buying from UK a product that is not advertising the right country code and is operating outside the UK laws. no firmware can get it to work in the UK laws. who is legally responsible? the user? the user might not even know that the transmit power is outside the limits because there is no indication about the transmit power. the shop that sold it? the UK supplier that brought and sold in UK an equipment that clearly does not comply with the laws? the manufacturer of the equipment that might not even be registered in UK?
u go after the user he will just show u that he bought the device from UK and u will have a serious legal battle regarding if he/she is actually responsible that a device sold in UK by an UK shop is not respecting the UK laws when the user had no way to check that it is not meeting the UK law (main problem here is if this device should had even be allowed to be sold in UK, because if it supposed to not be sold then u will have serious problems doing something to the user, u will have a long legal battle that in the end the user might even win it).

1 Like

It is three with radar detection out of FIVE (last one of six misses last 20MHz)
Recommend to switch off KV, most clients (depends on i-device age how quickly) roam to better signal in same ESSID domain without any pushing.

"I think the presence of DE and/or NL country codes are causing the Macs to reduce their transmit power levels to 25 mW on channels 138 to 165 from the GB limits of 1000 mW or 200 mW (channel dependent)."

This creates an asymmetric network where the client to AP traffic is 9 dB below the AP to client traffic, so the AP cannot reliably receive packets (including 802.11 ACKs) and so network performance drops to unacceptable levels.

The Mac still sees a strong signal from the AP, so it does not choose an alternative AP on a different band or a different 5GHz channel (i.e. roam to a different 5 GHz AP within the ESS).

I'm not currently doing any pushing - I only have neighbor reports enabled. This seems to reduce disconnected time during a roam event from a few seconds to about half a second (Apple's documentation supports this).

I've been using prometheus to collect client signal strength data across the site for a few months, but I don't have enough data as-yet to give a definitive answer on whether average signal strength is improved with NRs enabled (I only enabled them a few days ago). n.b. this Mac packet loss issue appeared before NRs were enable and persisted after it was enabled too.

Because of this "country code conflict" issue, I plan to make other changes to the channel plan across the APs, so I suspect this will muddy the waters a bit with the collected data.

In case it's of interest I'm using Prometheus with Grafana to track client signal strength over time. The below is aggregated data across 6 APs. Here is the last 14 days - the sparse areas are weekends. NRs were enabled on Monday. The traces along the bottom are some smart speakers which cannot be persuaded to make reasonable AP selection choices it seems, and I should probably create another plot with permanently present devices excluded.

Edit: for clarity this is a heat map of all wifi clients on the local network aggregated across 6 separate physical wifi access points. Clients associated with other networks are not included.

30dBm 1000mW not in GB ....
But possible, for some it helps to disable idevices wifi location.
https://support.apple.com/en-us/102766 - last paragraph but do in reverse. There is no problem per se with asymmetric powers, 15dbm is typical for older mobile phones or laptops.
There is a possibility far enough wifi signal acts as noise. Eg block of flats 200m away coming home and watching netflix. Quite credible with your graph.

According to OFCOM IR-2030, in the UK the regulations for 5470-5730 MHz specify Maximum mean e.i.r.p. of 1W and Maximum mean e.i.r.p. density of 50mW/MHz in any 1 MHz band (with both Dynamic Frequency Selection and Transmit Power Control).

The problem I'm mainly hitting is within 5725-5850 MHz (which is Maximum mean e.i.r.p. of 200mW Maximum mean e.i.r.p. density of 10 mW/MHz in any 1 MHz band).

Source for the above 2 statement - - page 41 of https://www.ofcom.org.uk/siteassets/resources/documents/spectrum/interface-requirements/ir-2030.pdf?v=335258 pages 41 and 42.

I think that because the noise floor is relatively high in this location, if a client suddenly backs off its tx power by 9 db, link quality goes down the toilet. As I said previously this appears to happen when a Mac which receives a distant beacon with set to DE or NL will reduce it's eirp to 25 mW in 5725MHz - 5875MHz (source https://efis.cept.org/adhoc_grabber.jsp?annex=4).

I'll do some more debugging tomorrow to confirm this.

Yeah, just that particular regulation does NO MENTION WHATSOEVER backing your claim.

You get this.
/ https://git.kernel.org/pub/scm/linux/kernel/git/wens/wireless-regdb.git/tree/db.txt#n748

Yes, I know regdb is out of date. This doesn't impact the main cause of the problem tho -
because the regdb GB entry says:

(5725 - 5850 @ 80), (200 mW), NO-OUTDOOR

but the DE entry says:

(5725 - 5875 @ 80), (25 mW)

and with the problem at hand this would only impact the AP transmit power, the problem is with the Mac client transmit power - it is presumably not using the Linux regdb but instead whatever Apple include with MacOS.

You have to ask apple store to explain. Wrong tree to bark here.

As I said above "[the Mac behaviour] seems unreasonable to me. I can only assume some national laws were interpreted pessimistically" and also "I posted this thread mainly to help others diagnose and workaround similar problems." i.e. I wished to bring to wider attention a pitfall for the unwary. See also: Unusable 5GHz channels due to misconfigured nearby networks for Macs and Intel wifi chipset clients - #5 by TimSmall

3 Likes

What do you want from OpenWrt? Since you do not disable wifi location i assume you intend to go to apple store with premiumest care package for an "expert" to flip the checkbox.
We can not supply you with faraday mesh to isolateyou from those pesky europeans.

"I posted this thread mainly to help others diagnose and workaround similar problems."

Tim small is documenting their findings publicly to try and help others if they find themselves in a similar situation.

1 Like