Optimized build for IPQ40xx devices

I've been using this build (rebuilt to include 6rd and a few other packages but no other changes of any significance) as my default router for over a week now. This is with gigabit fiber internet and two routed local subnets. It's rock solid and the performance is excellent:

LAN, between two machines on separate routed subnets (no NAT, very few firewall rules):
Iperf/iperf3: 930-940 Mbps and stable.

WAN, NAT, nearly stock firewall rules:
Speed test, provided by my ISP CO: typically 800-850Mbps down, 600-700Mbps up.
Speed test, DSL reports: avg 85-90% of that.

(Previously when using an x86_64 build on a Sandy Bridge i5 with i340 NICs I was getting a solid 940Mpbs up/down on a fairly regular basis. But we may be comparing apples with oranges: Covid-19 stay-at-home behavior may be affecting area broadband performance generally, there are a lot of tech workers around here who'll be telecommuting now, and a lot of people staying at home watching Netflix)

(Edit: Finally dawned on me that I'm using a single LAN port from the router to a Gigabit switch for simultaneous tests, resulting in a bottleneck that would explain the ceiling observed; so maybe ignore the whole paragraph that used to be here. To do this right all the testing should be done on a bench with no links in common between simultaneous bandwidth tests.)

Anyway, tremendous work. I really like this router now.

1 Like

Thanks for the custom build, I have few questions,

  1. if I install your openwrt custom build / or generic openwrt, and after sometimes there's an update to this, do I have to reboot to original linksys firmware first then install custom rom if I want to retain at least 1 original linksys firmware intact?

  2. I'm interested in building your custom openwrt but with different modules built-in, currently I don't need 6in4 but I want to install basic build with usb 3.0 storage +extroot extension and wpad full (replacing wpad-mini), is there any tutorial on how to do it? thank you.

  3. Is this build based on 19.07.2?

Hi, tnaks for your questions.

  1. Yes, that is right. However, my custom build does have a way to reverting back to the original firmware at any time, given that you have the firmware from the Linksys website.
  2. OpenWrt does have a nice "wiki" on how to do that, the main difference is that you will be using my copy of their repository instead of the original. You probably will need to perform several changes, because this custom build already uses the OEM partition as an extroot (yes, you can access the full 43 MB partition used by the Linksys firmware) and it's designed to defend itself against that kind of changes in the runtime. I can help you to understand my modifications if you really want your custom build.
  3. Yes and no. The current in my repository (not updated since a long time due to the virus thing, will update it soon) is based on 19.07 plus all the modifications made in master. This is based on the snapshot brach, to say it another way. So, by this date it will be based on the 20.xx. If you want a stable release, the downloadable file contains the patches that can be used on the top of any 19.xx and 20.xx branch.

It would be nice to have your results, anyway and if you can, it is valuable feedback to me; since I live in a country where 10MiB/s is a corporate luxury and can not test this kind of thing.
And thanks for using my build and your feedback, always welcome to know more about use cases!


I've done some significant changes and OpenWrt have done other more (like upgrading the kernel) so I would like to know if it still performing good now, I was a bit distracted from OpenWrt due to the virus thing, that's why I didn't updated my repo but if you can test it agan when I upload the new version, it will be a nice feedback to have!

The results above are accurate: the thing I disclaim in parentheses at the bottom of the post refers to something in an earlier version of the comment, which I edited out.

I've just reproduced them:

Iperf3 between routed LAN segments (e.g. between hosts 192.168.10.80 <-> 192.168.20.41, where the 10 and 20 nets pass through separate interfaces on the ea6350, without NAT and with only a few simple firewall rules:

[  4] local 192.168.10.80 port 38268 connected to 192.168.20.41 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   119 MBytes  1.00 Gbits/sec    1   1.74 MBytes       
[  4]   1.00-2.00   sec   111 MBytes   933 Mbits/sec    0   1.82 MBytes       
[  4]   2.00-3.00   sec   110 MBytes   923 Mbits/sec    1   1.38 MBytes       
[  4]   3.00-4.00   sec   111 MBytes   933 Mbits/sec    0   1.46 MBytes       
[  4]   4.00-5.00   sec   110 MBytes   923 Mbits/sec    0   1.52 MBytes       
[  4]   5.00-6.00   sec   111 MBytes   933 Mbits/sec    0   1.56 MBytes       
[  4]   6.00-7.00   sec   111 MBytes   933 Mbits/sec    0   1.59 MBytes       
[  4]   7.00-8.00   sec   111 MBytes   933 Mbits/sec    0   1.62 MBytes       
[  4]   8.00-9.00   sec   111 MBytes   933 Mbits/sec    0   1.67 MBytes       
[  4]   9.00-10.00  sec   110 MBytes   923 Mbits/sec    0   1.72 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   937 Mbits/sec    2             sender
[  4]   0.00-10.00  sec  1.08 GBytes   929 Mbits/sec                  receiver

That's typical in both directions: sometimes more retransmits.

Centurylink speedtest: provided by the ISP and only 1 or 2 ms ping away, so only contending with nearest neighbors for FTTH bandwidth. Includes NAT, firewall rules mostly OpenWrt stock.


(Edit: earlier screenshot showed an unrepresentative 23ms jitter. Just did this one, a bit more typical. Note this is without disabling various other internet using devices such as chromecast, etc.)

Built from your v0.32 release, by git cloning the fork point from the mainline OpenWrt repo and then applying your .oc patches. My only .config modifications involved the addition of packages. Runtime changes, nothing relevant, though for now I've reverted to using dnsmasq (+stubby) rather than unbound, mostly because I need more comprehensive dhcp option support.

Looking forward to your next release, but I'm pretty happy now. The responsiveness is so much better than mainline!

1 Like

Hmm, I read that you compiled your own version. I have some notes to add:

  • Instead of modifying the .config directly, modify the configuarion seeds. Since the a2cf87-v1.00 version (and for people that want to further customize my work), I added some "marks" in the configuration seeds that indicate that these settings should not be modified in any way. You can generate a .config file by concatenating the seeds: cat global.seed modules.seed image.seed > .config
  • Same is true for the kernel seeds. You should concatenate your modified kernel seeds like this: cat kernel.seed >> target/linux/ipq40xx/config-4.19 and cat kernel.seed >> target/linux/ipq40xx/config-5.4.
  • The initial configuration for the device upon the first boot of a different version is in files/bin/bootz, it's a shell script with UCI commands that run when a canary file is not found. You can customize there the IP, your users and many other things during the first boot.
  • There are several files that are not safe to modify or delete, like the preboot scripts which, on mistake, will soft-brick the device, rendering even the failsafe mode useless.
  • There is an script in files/etc/extrafiles.txt that have to be run in the root of the source code to set the correct file permissions, since they are "inherited" to the image and some files with corrupted permissions can lead to security flaws; plus, some programs like ssh will refuse to use them.
  • The notes contains a line that says "Based on OpenWrt commit xxx". You probably want to checkout that commit before applying the patches.

I'm in the proccess of uploading the package to GitHub, so expect it to be there soon :slight_smile: and thanks for using this custom build and will be nice to have your test to compare against the 11737-v0.32 version, as changing the kernel can have significant performace differences.

2 Likes

For v0.32 I got good results by simply checking out OpenWrt commit 1173719, applying the .oc patches and then using "make menuconfig" to add all the additional packages (not just build-dependent ones like 6rd but pretty much everything I'm ever likely to want, since it's not wise to rely on the snapshot repository.) But it does have to be repeated for new builds if you do it that way, so I'll look at your recommendation for the next one.

As I mentioned over in the switch development thread, I'm becoming convinced that this device cannot route more than 1 Gpbs total (i.e. 500mbps symmetric, for example), and I'm wondering if that's just how Linksys wired up the board or if there's additional internal bandwidth that the firmware doesn't exploit. There's every indication that there's a physical 2gbps link between the SoC and the QCA8075 but it seems like we only get the use of half of it for whatever reason.

If true (and my bench testing suggests it is) then this is in effect a one-armed router plugged into a 5 port managed switch. Not terrible for something you can get for $30 on eBay, but still below its potential.

Normally it wouldn't matter that much; in real life I'm never trying to do a full gig bidirectionally over the lan (and it's unlikely any peer in the real world will give me that), but it does mean that this isn't suitable for simultaneous use as an edge and core router between internal subnets if you move significant amounts data between them.

Hi @NoTengoBattery, thank you for all your hard work on build; have been running it for few weeks without issues. Few items:

  • Any chance in adding vpnbypass and luci-app-vpnbypass packages as part of build/release cycle? These two compliment openvpn capabilities of the releases nicely.
  • I am running NoTengoBattery v0.32.sc ff331 (started at this version), looking at system messages i am seeing spamming related to Wed Apr 29 23:05:03 2020 daemon.err unbound: [4703:1] error: outgoing tcp: connect: Permission denied for 2606:4700:4700::1111 port 853. I see there is github issue #6 referencing this, but outlines its only relevant in .20 or below. Any idea?
  • The default build comes with two running SSH servers dropbear running on 2022 and openssh on 22. Any though to standardizing to a single server? My vote would be disabling openssh and configuring dropbear only on 22 (its lighter/saves a cpu cycle).

If I understand correctly the issues with the device, it is as you think: only one gigabit port goes to the CPU, which mean that the gigabit bandwidth is shared between LAN and WAN in the switch.

1 Like

Hi, thanks for your interest, let me answer your points:

  1. Yeah, sounds like a nice addition. However, vpnbypass can be installed from the Software section in LuCI. Try to install it and if it does not work, I think it's worth to add it to the firmware image.
  2. Yes, the issue is relevant always. The v0.20 part is that when using the default settings it will not happen. The thing is that you are using Cloudfare instead of the default Quad9. The issue is that your network is not capable of IPv6. You can solve the problem by deleting the IPv6 addresses in the Zones tab of the Recursive DNS service. I updated the issue to reflect your situation.
  3. Yes, I understand the SSH thing.
    3.1. OpenSSH is used because Dropbear does not work with FileZilla. FileZilla is important because it's part of the official method for reverting to the original firmware (which is in the OpenWrt page for the device, so it's really, really important). And as the build is intended to be used as a NAS, it is important to allow the users to use the SFTP service.
    3.2. Dropbear is needed because it is the only SSH server available during the Failsafe mode. Without it, you will not be able to use the Failsafe Shell over SSH, you will need to use a physical terminal (i.e. a UART cable).
    3.3. Don't worry about the CPU cycles, the dropbear process does not consume CPU time, and because of that I intentionally let it running. Also, it have a higher priority and some magic resource handling to try to keep it running, even if the device (that means, the other services like OpenSSH or LuCI) is not responding.

Hi,

Nice to see this wonderful device got an extraOpenWRT option.

The official build works quite good here, the only problem i have is poor 2.4ghz signal strength (comparing to other old 2.4ghz routers on same location with same client).
5ghz signal strength is better than the 2.4ghz one (and good comparing to old router).

Does this build have any improvements on this subject ? (since i dont see its mentioned here)
Or do you know any ways to tweak the 2.4ghz radio ?

Hi!

Let me tell you a bit of what happened time ago: at the beginning of the development, the 5 GHz radio was totally useless, bad signal strength and poor connectivity.

With the time, my investigation on the subject gave some good news to the device: 5GHz was useful, comparable to the OEM performance and sometimes better.

What did I do? Well, I tested what's called "calibration board files". The driver load those files to the wireless chip in order to overcome the physical differences that may degrade the wireless performance or make the device irradiate more power than allowed by law.

Linksys provide several of those files because it looks like there are some hardware variants for the design.


Answering your question: my build does contain the same calibration as the OpenWrt original (as I sent the files myself to the Linux Wireless developers), however, my build also contains a script that allows users to change the files in order to test all of the ones provided by Linksys. You should have to test them manually, one by one, as I have no way to know what should be correct for a particular device (I sent the file that best fit the majority).

You can install this build, run the script to test a particular calibration, perform some benchmarks, and proceed to do the same with the rest of the calibrations till you find the one that performs the best.

More details about the calibration testing script, of course, in my GitHub repository. The script is pretty straightforward to use, you call it once to list the options and then you call it with the file you want to test as the only required argument to the program. The device will reboot upon successful change operation and will load the new file to the chip during boot.


P.S.: I just checked my OP and it does say about the script in question.

Thanks for you thorough reply.

I tried changing the calibration board files and didn't notice any major changes, maybe i did it wrong ? or there should not be any big difference between them ?

My unit 5ghz reception is the best i have seen , but 2.4ghz reception is quite bad (about 10-15 less dbm than my old archer c2) , and worse than 5ghz on same spot from same unit (about 5 dbm less).

I see the 2.4ghz uses 20dbm (lowering it does lower reception) transmit power and 5ghz uses 23dbm , maybe there is a way to set 2.4ghz tx power to higher value (although i know it generally not recommended).

Is my reception experience matches yours or any one else ?

Well, I'm aware of the problem. I've tested some more calibration files (not present in the firmware, I have the only copy) and I want you to test one in particular.

Yes, there should be a noticeable difference between some files, other files perform almost the same. If you tested all files right, then when the device booted after performing the command testcalibration art, your 5.8 GHz wireless radio will stop working.

Please, download the attached file and copy it to the device, replace the existing file in the /lib/firmware/ath10k/QCA4019/hw1.0 path with the providen here, reboot the device and tell me about your performance.

Also: try to go to LuCI and in the Wireless page try to set up (manually) the wireless power to 30dBm.


I think that the problem here is that the calibration files does contain wrong data about the antenna gain. However, I have no idea of what the format of the files is, how the data is stored and it looks like there are no tools to inspect or modify the files. The only thing we can do is test the files that Linksys included with their firmware.


Firefox Send files do expire after a number of downloads or when the time is due. This link is, therefore, short-lived.


With that file (modified to improve the 5GHz, your 5 GHz will not perform well), I'm getting those results: -33 for 2.4 GHz and -56 for 5 GHz.

1 Like

Well dude , you did it.

Replaced the original file with the file you supplied and 2.4ghz reception is up to the roof...

Got about 25dbm boost on all my devices and 2.4ghz reception is now much better than 5ghz (as it should).

Also i dont experince worse 5ghz signal like you described , from try to set 40mhz on 5ghz and not 80mhz and you should get better signal.

I will countinue to check for a few days that there is no side effects , but it really seems promising.

Just to update.

The wifi is working just fine for the past 3 days , 2.4ghz is great here and also 5ghz doesn't seems to suffer.

Did a test with official image and changed the file , same results so far, so if any one got a bad 2.4ghz reception with official image, they can give it a try...

I have also tried it. It brings new life to 2.4ghz, but i did notice the laptop wifi signal drops a few bars and then back up again on 5ghz

So if I understand correctly this file does improve the wireless performance, am I right?

As a side note: during the process of finding the file, I did tested all the remaining calibrations and this is the best of all (for 2.4 GHz). If this file does not improve the 2.4 GHz quality, there is nothing more that can be tested now.

Thank you for your work on porting wrt to this device. If i have to choose which calibration board file, then I would use your latest. The one in 'firefox send'

I see. So in the sake of the reviews for the file, I'll be including it with my next prebuilt. If I don't see any problem with the time, I will send the files to the Linux kernel developers so they can update the OpenWrt's version.

Note that OpenWrt does not manage the ath10k calibration files, is the Linux people that does that directly. Therefore, changes will take some time to reach the official build.