WDS instability - connectivity issue (atheros)

zoulou · January 15, 2019, 10:14pm

Hi,
I am reaching for help from the community as it's been years now that I can not fix a problem with I think a point to point link with WDS.
My setup :

in zone A :
A main router connected to internet
wired to a router configured as WDS AP
in a distant zone B :
router as WDS client wired to a network with other equipments

To all of those routers are connected other routers, AP, equipments
I would prefer keeping everything on the same L2 layer for convenience.
All routers are atheros based with openwrt latest stable version.

This setup works pretty well, although from time to times every other day, or sometimes several times per day, the network is lagging and impossible to ping from A to B. This can last a couple of minutes or hours or never recover.
Machines in zone A can ping each other, Machine in zone B can ping each other.
With iwinfo i still have a link with expected throughput: 45.7 MBit/s
Load is low on all routers. No errors but dropped packets on ifconfig but can not tell why or if this normal or not.

Weird thing is that I have cam from B that I can still see streams from A (bandwitdh is about 10Mb/s less than wifi bandwidth, problem occurs also with cams off )
with tcpdump I can see packets going out from AP WDS but I do not see them coming on Client WDS.
If I restart wifi on AP WDS, everything works fine.

Any idea, suggestion or way to troubleshoot this ?
I could not find what could trigger this problem, maybe with the use of spotify and sonos like device ( multicast packets ? )
I used to have wifi hanging with wpa group key renewal, I disabled group key and this problem seems to have disappeared.
Thanks for your help !

trendy · January 15, 2019, 10:45pm

What is the distance between the two zones?
What are the obstacles between them?
Are you in a residential zone, or some remote location?
Have you taken in consideration interference from other WLANs?
Are you using a fixed channel or auto?
What are the signal and noise levels on both sides?

zoulou · January 16, 2019, 12:17pm

There is 125m between wireless endpoints. no permanent obstacles.
I am in a remote location with really no other wifi nearby.
The point to point link is on channel 6 HT, my other AP are on channel 1 or 11 to prevent overlap.
I tried the CF-E110N which has a directive antenna of claimed 11dbi but roughly got the same signal level as with a classical TP-LINK WR1040 or WR841N.
output of iwinfo wlan0 assoclist that do not really change if network is lagging or not :

-68 dBm / -86 dBm (SNR 18)  0 ms ago
        RX: 81.0 MBit/s, MCS 4, 40MHz                1502664 Pkts.
        TX: 120.0 MBit/s, MCS 5, 40MHz                 22300 Pkts.
        expected throughput: 43.4 MBit/s

ont the client side :

-70 dBm / -95 dBm (SNR 25)  20 ms ago
        RX: 150.0 MBit/s, MCS 7, 40MHz                 51358 Pkts.
        TX: 90.0 MBit/s, MCS 4, 40MHz                1896299 Pkts.
        expected throughput: 38.9 MBit/s

As I am writting those lines, the network started lagging and I have to wait 30s for an ssh connections.

trendy · January 16, 2019, 1:46pm

From my experience the signal is quite weak for just 125m distance. But this shouldn't be such a problem. And the directive antenna should give way better signal than the stock omnidirectional antenna.
What I suggest is to check the connections and the cables (if any) from the router to the antenna. Also make sure the antenna aiming is optimal.
Then keep an traffic monitor program running, like iptraf to continuously monitor the traffic on the links to see if it is connected to the outages that you experience.
One more thing, on which interfaces do you see the dropped packets that you mentioned earlier?

zoulou · January 16, 2019, 6:26pm

This is an integrated antenna so this is factory made. In the past I tried to use an external yagi (cheap one) but with no improvement. I have to tell that one router is in a building cover with wood so this is not totally free space connection.
Dropped packets are on the bridge br-lan and all interfaces are on this bridge (eth0,eth1,wlan0).
There is no dropped packets on the individual interfaces.
I used to look at traffic with iftop but did not see any difference with or without the lagging

lleachii · January 16, 2019, 7:22pm

You didn't change the Distance Optimization to a number <= 125, did you?

trendy · January 16, 2019, 7:55pm

I usually configure that +10% to be on the safe side.

@zoulou
The 11dBi antennas for p2p links are not worth spending money on. Always go for the best you can get, say 20something dBi directive antenna, minus connectors and cable loss, and card transmit at minimum (usually 1mW). Still legal transmission power AND high gain on the receiver.
By the way, post the whole wireless config to have a look.

zoulou · January 17, 2019, 8:14am

Ok, How can I tell if this is a wireless reception issue, wouldn't the rate and signal should drop in iwinfo.
I do not see any difference with and without the problem, and when its working (most of the times) I am happy with the bandwith.
The wireless config is pretty simple:

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11g'
	option path 'platform/qca953x_wmac'
	option legacy_rates '1'
	option channel '6'
	option htmode 'HT40'
	option country 'FR'

config wifi-iface
	option device 'radio0'
	option mode 'ap'
	option network 'lan'
	option ssid 'xx'
	option hidden '1'
	option encryption 'psk2'
	option key 'xx'
	option wds '1'
	option disassoc_low_ack '0'
	option wpa_group_rekey 0
	option log_level 0

And on the client side :

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11g'
	option path 'platform/qca953x_wmac'
	option htmode 'HT40'
	option disabled '0'
	option channel 'auto'
	option country 'FR'
	option legacy_rates '1'

config wifi-iface
	option device 'radio0'
	option network 'lan'
	option mode 'sta'
	option ssid 'xx'
	option encryption 'psk2'
	option key 'xx'
	option wds '1'

trendy · January 17, 2019, 9:15am

The noise might as well raise. In server side you already have noise at -86, which means that there is already something near. On the contrary in client side the band is more clear, noise at -95 is practically no noise. Also the omnidirectional antennas collect noise from all around. In contrast a good directional antenna would isolate or attenuate the noise from the side and the rear.
Regarding the configs, fix the channel on client, and add the distance option (140m)
Fingers crossed!

zoulou · January 17, 2019, 3:08pm

I think the noise comes from the router itself(CF-E110n) , I have anoher unit with same noise reading and if I put any other router at the same place I get -95dBm. Someone stated the same problem with that device. CPE COMFAST CF-E110N won't work after downgrade
I don't know if it is true reading or not, if so that kind of defeat the directionnal antenna benefit
I changed the distance option to 150m on both ends and fix channel on client side, I will check if that made any improvement.

zoulou · January 18, 2019, 7:47pm

Well same thing with the new settings.
Do you know of any means to check what can be the cause of the lagging. I have the actual bandwith 3 times less than the expected throughput stated by iwinfo.
Can a queue get full and how to check ? thx

trendy · January 18, 2019, 9:52pm

I wouldn't trust the throughput estimation from iwinfo, as it is based only on the SNR ratio. Better run an iperf speed test between the wireless endpoints with different packetsizes and protocols. Maybe the speed is not as much as you thought.

zoulou · January 18, 2019, 10:48pm

I've ran an betterspeedtest.sh benchmark between devices on both sides of the wireless link and got 37.5Mbps donwn and 49.6Mbs up which is not so far that what iwinfo states

dlakelan · January 19, 2019, 2:05am

Set distance optimization to +50% if not +100%

Also I have had similar issues with WDS and eventually gave up on it. Try a mesh.

zoulou · January 19, 2019, 4:26pm

After a lot of tests I think also WDS has still some bugs altough it is working most of the time.
I set up the link as a mesh but it turned back to HT20 and not HT40. Is there any known limitation for that because the bandwith went down to 30Mbps.
I'll report if it is more stable
I was also planning to try relayd or GRE-tap, any advice on that.

hisham2630 · January 19, 2019, 5:09pm

can you add those settings to /etc/config/wireless :

#config wifi-device 'radio0'
        option frag '2346'
	option rts '2346'
	list ht_capab 'TX-STBC'
	list ht_capab 'RX-STBC1'
	list ht_capab 'DSSS_CCK-40'
	list ht_capab 'SHORT-GI-20'
	list ht_capab 'SHORT-GI-40'
	option turbo '1'
	option beacon_int '1000'
	option bursting '1'
	option ff '1'
	option compression '1'
	option disabled '0'
	option distance '100'

*note: use iw phy1 info to get a list of the supported ht_cabpab.

and add this under:

config wifi-iface 'default_radio0'
        option disassoc_low_ack '0'
	option dtim_period '255'
	option skip_inactivity_poll '1'

Don't forget to make a backup before make those changes, those settings may work and may not!

zoulou · January 19, 2019, 5:13pm

@hisham2630 : thanks for your recommendations, is this for the mesh or wds setup ?

hisham2630 · January 19, 2019, 6:01pm

you're welcome
Not sure, but i think it will work for all cases!