Optimized build for IPQ40xx devices

Well, I don't know about the CT kernel module and firmware. I do not have such an use case and thus haven't tested it deeply.

I would recommend trying the current configuration and changing things only if you're facing issues or the default configuration does not fit your needs.

Again, as a general advice in software, keep the things default unless you are having issues, you know exactly what you're doing or you are willing to fix something if you break it :blush:

Yea, so true. I'm also compiling your work at the moment, I got an interactive question all the sudden on a missing config settings: (cpu scheduler related). I answered 'N' as it was the default.

UCLAMP_TASK_GROUP
(probably had to do a make oldconfig which I didn't)

on the EA8300 , the current loaded ath modules are:

root@Dallas:~# lsmod | grep -i ath
ath                    24576  1 ath10k_core
ath10k_core           430080  1 ath10k_pci
ath10k_pci             49152  0 
cfg80211              286720  4 batman_adv,ath10k_core,ath,mac80211
hwmon                  16384  1 ath10k_core
mac80211              516096  1 ath10k_core

I have a gut feeling that this author replaced those drivers because he also experienced some issues, I'm going to keep it as it is for the moment, from all I'm seeing here they seem to fly like an eagle. Since I don't really know the intrinsic differences in those drivers , I think I belong in the category: Don't know exactly what I'm doing so I'm going to set everything up and keep those drivers as provided in your build.

Let me know if you want something tested on these routers @NoTengoBattery . I read you had some testers who lack sending you feedback, I'll do better :wink:

Cheers again

Hi,

Little update for those among us trying to get Mesh to work on tri-band devices like EA8300 and MR8300 using a dedicated backhaul, Before using @NoTengoBattery build, I was unable to ping from node02 to node01 on the lan addreses. So 192.168.1.2 and 192.168.1.1 , Unless I attached a cable between the lan ports. (duh) . But after using those builds and setting it up like below I could unplug the cable and everything kept working, having node02 only connected via bat0 backhaul bridged it worked.

But I was always able to use batctl p and batctl n to show that the mesh seemed to work. This is a really huge step forward . while I was messing about in the settings with the bridges and so on, there seemed to be nothing wrong with the configuration of network/wifi at all.

Some noteworthy need-to-knows for this to work:

  • set gw_mode to the proper modes, server and client
  • using @NoTengoBattery 's build, it actually shows the interfaces with their vlan id. So before I used the openwrt generic build, those did not show up, instead I got eth0 , eth1 and not their vlan counterparts
  • use the full wpad-wolfssl and not just wpad-mesh-wolfssl . It made a big difference
  • at some point, when uninstalling wpad-* packages, I had it happen on both routers that unload dns resolver seems to crash/stop working and then opgk install will fail to work (and confuse you). On the EA8300 it restarted itself and I didn't at first understand what happened, I thought it was the ssl package I just de-installed preventing opgk update to work. on the MR8300 it also stopped at exactly the same stage, but did't restart , also a restart of the router didn't seem to work, I guess it crashed because the wolfssl wasn't present and it probably needs it. But restarting later , and/or manually assigning a dns server like 8.8.8.8 in resolv.conf did the trick, then you install wpad-wolfssl immediately and the problem is gone.
  • I never manually assigned a MAC address, best way to avoid duplicate macs, I was able to export a backup tar.gz , extracted it locally, modded the settings, ip's and then upload it to the other node and it worked right of the bat(man). changed the ip's/dns
  • I had some issues trying to sysupgrade between both partitions, when they still had their stock firmware on 1. It failed a few times. I now have openwrt generic on the 'other' partition and @NoTengoBattery 's build on the other. using -n on sysupgrade from generic openwrt made me able to login properly using a cable in the lan port when it booted the alternative partition. when I didn't use -n the switch layout gets all screwed up and I got a few pings through while they booted but then they got all closed off, but booted fine so they didn't fall back to the other boot partition
  • for those 2 routers to manually make them switch boot partition, just boot 3 times, unplug the cable the second you see the WAN port go live with repeated flashing (after the lan ports do their little progress-bar-like flashing), and it will work. I had a little catch-22 there but having openwrt on both partitions was the solution, so I got rid of the stock firmware entirely.

Feel free to comment on these, in case I missed something or use bad practices etc. It's the first time I messing with mesh-capable devices. I only use the 2.4Ghz here to get proof-of-concept to work and the 3rd 5GHz band is disabled here.

You need these packages:

opkg install batctl-full kmod-batman-adv wpad-wolfssl

But de-install the conflicting ones first.

Finally, a few remarks on the @NoTengoBattery build, there are little gems in there, like the (b)ash history that is kept even when disconnecting, there are little things in there that are so handy to have around. I did notice that the memory use is a lot higher due to some additional setup differences, like the added swap, that isn't in the stock openwrt build. I doesn't seem to be a problem, but the EA8300 does have only half the memory of the MR8300. I never used those memory things, but in luci they can be modified.

I tried building those packages from the source and was able to successfully install them on my router, but it seems the kernel version is not the same as the @NoTengoBattery images that he so kindly made, which have a slighly higher kernel version. I haven't dug into that yet to see why as I'm using a clone of the github repo, but in any case.. the build ran fine , after installing it I did have to do a hard reset of the modem and launch the firstboot script to kinda clean it up and after a reboot it loaded all the modules fine and set the cli prompt to dallas like the prebuilt images. I did not use the -ct drivers as mentioned in many blogs, the patched drivers here perform excellent.

the configs

MR8300 - router 1 - WAN gateway

root@router1:~# cat /etc/config/network


config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdb2:74b3:833d::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'bat0'
	list ports 'eth0.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option ipaddr '192.168.1.1'
	list dns '192.168.1.1'
	list dns '192.168.1.2'

config interface 'wan'
	option device 'eth0.2'
	option proto 'dhcp'
	option dns '127.0.0.1'
	option peerdns '0'

config interface 'bat0'
	option proto 'batadv'
	option routing_algo 'BATMAN_IV'
	option aggregated_ogms '1'
	option ap_isolation '0'
	option bonding '0'
	option bridge_loop_avoidance '1'
	option distributed_arp_table '1'
	option fragmentation '1'
	option gw_mode 'server'
	option hop_penalty '30'
	option isolation_mark '0x00000000/0x00000000'
	option log_level '0'
	option multicast_mode '1'
	option multicast_fanout '16'
	option network_coding '0'
	option orig_interval '1000'

config interface 'mesh'
	option proto 'batadv_hardif'
	option master 'bat0'
	option mtu '1536'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '1 2 3 4 0t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '5 0t'

root@router1:~# cat /etc/config/wireless


config wifi-device 'radio0'
	option type 'mac80211'
	option channel '153'
	option hwmode '11a'
	option path 'soc/40000000.pci/pci0000:00/0000:00:00.0/0000:01:00.0'
	option htmode 'VHT80'
	option disabled '0'
	option country 'BE'
	option cell_density '0'

config wifi-device 'radio1'
	option type 'mac80211'
	option channel '6'
	option hwmode '11g'
	option path 'platform/soc/a000000.wifi'
	option country 'BE'
	option cell_density '0'

config wifi-iface 'default_radio1'
	option device 'radio1'
	option network 'lan'
	option mode 'ap'
	option encryption 'psk2'
	option key '2Hot4u3hardy'
	option ssid 'dd-wrtg'

config wifi-device 'radio2'
	option type 'mac80211'
	option channel '36'
	option hwmode '11a'
	option path 'platform/soc/a800000.wifi'
	option htmode 'VHT80'
	option disabled '1'
	option country 'BE'
	option cell_density '0'

config wifi-iface 'wmesh'
	option device 'radio0'
	option network 'mesh'
	option mode 'mesh'
	option mesh_id 'MeshCloud'
	option encryption 'sae'
	option key 'MeshPassword123'
	option mesh_fwding '0'
	option mesh_ttl '1'
	option mcast_rate '24000'
	option disabled '0'

EA8300 - router2 - mesh client

root@router2:~# cat /etc/config/network


config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdb2:74b3:833d::/48'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'bat0'
	list ports 'eth0.1'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option ipaddr '192.168.1.2'
	option gateway '192.168.1.1'
	list dns '192.168.1.1'
	list dns '192.168.1.2'

config interface 'wan'
	option device 'eth0.2'
	option proto 'dhcp'
	option dns '127.0.0.1'
	option peerdns '0'

config interface 'bat0'
	option proto 'batadv'
	option routing_algo 'BATMAN_IV'
	option aggregated_ogms '1'
	option ap_isolation '0'
	option bonding '0'
	option bridge_loop_avoidance '1'
	option distributed_arp_table '1'
	option fragmentation '1'
	option gw_mode 'client'
	option hop_penalty '30'
	option isolation_mark '0x00000000/0x00000000'
	option log_level '0'
	option multicast_mode '1'
	option multicast_fanout '16'
	option network_coding '0'
	option orig_interval '1000'

config interface 'mesh'
	option proto 'batadv_hardif'
	option master 'bat0'
	option mtu '1536'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '1 2 3 4 0t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '5 0t'

root@router2:~# cat /etc/config/wireless

config wifi-device 'radio0'
	option type 'mac80211'
	option channel '153'
	option hwmode '11a'
	option path 'soc/40000000.pci/pci0000:00/0000:00:00.0/0000:01:00.0'
	option htmode 'VHT80'
	option disabled '0'
	option country 'BE'
	option cell_density '0'

config wifi-device 'radio1'
	option type 'mac80211'
	option channel '6'
	option hwmode '11g'
	option path 'platform/soc/a000000.wifi'
	option country 'BE'
	option cell_density '0'

config wifi-iface 'default_radio1'
	option device 'radio1'
	option network 'lan'
	option mode 'ap'
	option encryption 'psk2'
	option key '2Hot4u3hardy'
	option ssid 'dd-wrtg'

config wifi-device 'radio2'
	option type 'mac80211'
	option channel '36'
	option hwmode '11a'
	option path 'platform/soc/a800000.wifi'
	option htmode 'VHT80'
	option disabled '1'
	option country 'BE'
	option cell_density '0'

config wifi-iface 'wmesh'
	option device 'radio0'
	option network 'mesh'
	option mode 'mesh'
	option mesh_id 'MeshCloud'
	option encryption 'sae'
	option key 'MeshPassword123'
	option mesh_fwding '0'
	option mesh_ttl '1'
	option mcast_rate '24000'
	option disabled '0'

Hope this helps someone trying to do this on a router

2 Likes

Well, the fact that you had to use a manual DNS is because in the default configuration, Unbound uses DNS over TLS that, you may guess, needs a SSL provider library.

On the other hand, "upgrading" from vanilla OpenWrt to this build is know to soft brick the device. This is the only circumstance where the -n may be required.

The build can be upgraded with a newer version of itself without trouble, but switching between vanilla and this build in any direction with keeping settings will make the device inaccesible.

I would say that everything's working as expected. Shall you face a bug, let us know.

1 Like

I agree, I didn't know unbound before I used this and lookin at the config I saw the TLS dependancies so that all made sense that in between package removal and install of the full ssl package there was a little situation where I was in no-man's land :slight_smile: . Using an external DNS was the solution, and after reboot the resolv.conf file was already reset, so great stuff

I was able to upgrade (using sysupgrade image) fine though, but even not using -n and doing a reset of the router to defaults on your build made me able to login again, but I now start to think that that is exactly what you mean with soft-bricking :wink:

the -n is still good if you want to start from zero, because of the bad switch functionality of openwrt generic build for this router (due to the driver problems I assume) , starting with a clean setup has proven to be the correct approach, in the end, it's only 2 files that need some love for this all to work and an install of a few packages. So I just manually merged the devices/interfaces into a clean install and that only takes 15minutes for 2 routers.

I do beg to differ : It's not working as expected. It's working BETTER than expected , it's just awesome I have them working perfectly. I do notice that DNS in unbound seems a bit slower with uncached requests than dnsmasq would be, but I think you made a few remarks on that already, which I read here and remembered, nothing to write home about, once the dns cache is filled up, I didn't see any stability or speed problems. Unbound looks interesting to play with so I'm keeping it for now. I do see that dnsmasq is running too on my install, but it doesn't seem to interfere with unbound for now, I still have lots of research to do in each component, so many options is going to take time to understand how they interact.

The only small thing I worry about it the memory footprint and the use of swap, but need to read up on those as well before I understand the implications.

So I would say : These things works like a true champ now , it's quite awesome man. You really know your stuff. I published my findings because I couldn't find someone who uses a dedicated backhaul with these 2 devices in the forums like I do, so just hope this feedback can be useful for others.

Thank you

Refferal to my previous test on link, and... after doing a little optimization my results is (in 2,4G wifi):

  • 2.4G with 2 metres (smartphone)- 27-32Mbps internet/ 75-80Mbps with iperf
  • 2.4G with 10 metres (smartphone; two monolithic reinforced concrete interstorey floor and one load-bearing wall)- 18-21Mbps internet/ 20-22Mbps with iperf
  • 2.4G with 40 metres (smartphone; one load-bearing wall and outdoor)- 12-15Mbps internet/ 12-15Mbps with iperf

Mayble it will be useful to someone in solving the problem of poor bandwidth in 2.4G :wink:
New code in sysctl.d (file v1)

net.ipv4.tcp_timestamps=0
net.ipv4.tcp_frto=0
net.ipv4.tcp_sack=1
net.ipv4.tcp_dsack=1
net.ipv4.tcp_fastopen=3
net.ipv4.tcp_pacing_ca_ratio=160
net.ipv4.tcp_pacing_ss_ratio=200
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_congestion_control=hybla

or file v2

net.ipv4.tcp_timestamps=0
net.ipv4.tcp_frto=1
net.ipv4.tcp_sack=0
net.ipv4.tcp_dsack=0
net.ipv4.tcp_fastopen=3
net.ipv4.tcp_pacing_ca_ratio=160
net.ipv4.tcp_pacing_ss_ratio=200
net.ipv4.tcp_no_metrics_save = 1
net.ipv4.tcp_congestion_control=bbr

and new script in /etc/hotplug.d/iface/ about the content:

#!/bin/sh

echo 1 > /sys/kernel/debug/ieee80211/phy0/ath10k/nf_cal_period
echo "7 64" > /sys/kernel/debug/ieee80211/phy0/ath10k/htt_max_amsdu_ampdu
LIMIT=6000 LIMIT1=23999; for ac in 0 1 2 3; do echo $ac $LIMIT $LIMIT1 > /sys/kernel/debug/ieee80211/phy0/aql_txq_limit; done

echo 1 > /sys/kernel/debug/ieee80211/phy1/ath10k/nf_cal_period
echo "7 64" > /sys/kernel/debug/ieee80211/phy1/ath10k/htt_max_amsdu_ampdu
LIMIT=6000 LIMIT1=23999; for ac in 0 1 2 3; do echo $ac $LIMIT $LIMIT1 > /sys/kernel/debug/ieee80211/phy1/aql_txq_limit; done

Pings is smaller than on default settings, bradwidth is better in 2.4G (for 4-5 clients on wireless and 1 ethernet), but mayble be even better probably....

Question :wink:
How change e.g initcwnd or initrwnd (for tests and optymize)?

root@Civic:~# ip route
default via 10.15.84.181 dev wwan0  src 10.15.84.182
10.15.84.180/30 dev wwan0 scope link  src 10.15.84.182
192.168.52.0/24 dev br-lan scope link  src 192.168.52.1
root@Civic:~# ip route change default via 10.15.84.181 dev wwan0  src 10.15.84.182 initcwnd 10                    
ip: either "to" is duplicate, or "initcwnd" is garbage
root@Civic:~# ip help
BusyBox v1.33.1 (2021-08-31 22:20:08 UTC) multi-call binary.

Usage: ip [OPTIONS] address|route|link|neigh|rule [ARGS]

OPTIONS := -f[amily] inet|inet6|link | -o[neline]

ip addr add|del IFADDR dev IFACE | show|flush [dev IFACE] [to PREFIX]
ip route list|flush|add|del|change|append|replace|test ROUTE
ip link set IFACE [up|down] [arp on|off] [multicast on|off]
        [promisc on|off] [mtu NUM] [name NAME] [qlen NUM] [address MAC]
        [master IFACE | nomaster]
ip neigh show|flush [to PREFIX] [dev DEV] [nud STATE]
ip rule [list] | add|del SELECTOR ACTION
root@Civic:~#

Yes, I noticed that too. @NoTengoBattery Seems like your Github repo is still on v3.0.0?

It is possible that I forgot to publish my changes :sweat:

The solution you provided works well for EA6500v3 as expected, but does it also work for linksys 1900ACS and if not, why?

The Linksys 1900 ACS is a Marvell Armada SoC that is a completely different target than the IPQ40xx devices.

I think you need to install ip-full.

@pingec please check the downloads server. I uploaded a build for your device. Let me know how it goes.

and no ip-tiny or ip-full? :wink: (edit: ok no iw-full)
I tested it... and best settings for me are (new scripts /etc/hotplug.d/iface/98-tuneroute)

#!/bin/sh

defrta=`ip route | grep "default" | head -1`
ip route change $defrta ssthresh 25 initcwnd 32 cwnd 25 initrwnd 39 quickack 1 fastopen_no_cookie 1
# defrtb=`ip route | grep "wwan0" | grep -v "default" | head -1`
# ip route change $defrtb ssthresh 25 initcwnd 32 cwnd 25 initrwnd 39 quickack 1 fastopen_no_cookie 1
# defrtc=`ip route | grep "br-lan" | head -1`
# ip route change $defrtc ssthresh 25 initcwnd 32 cwnd 25 initrwnd 39 quickack 1 fastopen_no_cookie 1

initcwnd and initrwnd are different on purpose, other options by tests are inconclusive

Question:
For further tests. Is it possible to experiment with the ring buffer size for increase speed of data transfer (on wireless)? Can I change it other somehow to ea6350?

root@Civic:~# ethtool -g wlan0                                                  
Ring parameters for wlan0:
Pre-set maximums:
RX:             0
RX Mini:        0
RX Jumbo:       0
TX:             0
Current hardware settings:
RX:             0
RX Mini:        0
RX Jumbo:       0
TX:             0
root@Civic:~# ethtool -g eth0
Ring parameters for eth0:
Pre-set maximums:
RX:             128
RX Mini:        0
RX Jumbo:       0
TX:             128
Current hardware settings:
RX:             0
RX Mini:        0
RX Jumbo:       0
TX:             0
root@Civic:~# ethtool -G eth0 rx 256
Cannot set device ring parameters: Not supported
root@Civic:~# ethtool -G wlan0 rx 256
Cannot set device ring parameters: Invalid argument

EDIT:
Bug or feature (sources of opkg)? :slight_smile:

Downloading https://downloads.notengobattery.com/projects/openwrt-v3.0.1/targets/ipq40xx/generic/packages/Packages.sig
Signature check failed.
Remove wrong Signature file.

Yep, totally a bug. Fixed!

Hi,

I'm using an Aruba AP303H (https://openwrt.org/toh/aruba/ap-303h)

This is also using the IPQ40XX , and the default image the vlans don't work. Could you also create an image for the aruba-303H?

Ring parameters (change in ethtool) not work, but not work it in NoTengoBattery images (2.10, 3, 3.0.1), snapshots of 20.02.2022, Openwrt by Cezary (on 19.07 and 21.02 based), on Linksys EA6350v3 and TPlink wr1043ndv3...
Either the right packages are missing, or can't do it in OpenWrt.

But by playing with snapshot image and option of fragmentation, treshold rts and ondemand of scaling governor settings, I gained even higher and more stable wireless connection (on 10 meters in test environment), i.e. not about 18-20Mbps but already 20-24Mbps connections to LTE Internet.

@NoTengoBattery @kastellsc @diego123

Sorry for the delay. Today I flashed Zyxel NBG6617 with the NoTengoBattery image that he kindly prepared for this device. Everything seems to simply just work at first glance... magical. Wifi works, internet works, I will see how it does in the following days (sqm, adblock, vlans). Is there anything specific that I can test / provide from my device?

Yes one thing, can you get stable 5Ghz wifi on the DFS channels? It's weird I can only use ch 36 and one of the higher up limited to 10dBm... All others are working only when the weather is right :slight_smile:

You are right. At 20 mhz channel width only channels 36 - 48 seem to work, for all other channels I get "Wireless not associated" and I don't see much in the log.

Actually often it helped forcing wifi up on the comsole or a reboot but it is erratic and unpredictable. so in my regdom only 2 usable 80MHz channels: 36 and up @23dBm, as well as something around 140 or so @10dBm max. allowed power.

All other is unusable unfortunately.

With the stock FW no such issues, so for now im running vanilla openwrt on one on ch13 and 36 and a stock nbg6617 as dumb AP on dome other channels, as 13 doesn't work with intel wifi adapters and 36 is very popular over here...

Hi,
try setting the option as in the link it supposedly helps a little on disconnection.

As for my tests I'm struggling to increase speed in 2.4Ghz - I don't want to modify Minstrel_HT algorithm (rate control module for wifi), and it may help to squeeze out even 10-15% more (Minstrel-Blues modification- although this modification is quite old). I'd rather let go of such experiments- currently it's about 80% of what I can get over cable (i.e. 30-35 with 40-45Mbps of LTE Internet).