Full-featured custom build for Dynalink DL-WRX36 (AX3600)

jkool702 · 2024-05-24T14:49:46.544Z

greem:

Imgur's AI has tagged your photo

Imgur likes to do that to me for some reason..I rebuilt our deck a year and a half ago and posted pictures of it and imgur did the same to that album. (I'll be honest I was a bit flattered that imgur thought the deck I built was too sexy for underage eyes). Heres one of our dog and puppy playing that I dont think imgur has tagged. But were getting off topic....I swear Its just a picture of our new chocolate lab puppy. lol.

greem:

I've been having some difficulty building it myself

Getting it to build is a bit of a process. A few packages I had to use the "source override" feature and build from a cloned git repo. Some packages I had to unpack and modify CMakeLists.txt and change -Werror to -Wno-error so it wouldnt error out on stupid stuff (annoyingly, cmake puts its own builtin flags after the ones you manually specify, so even if you explicitly give -Wno-error as a build flag in menuconfig it will be ignored). Some things required being a non-root user (building one of the build tools IIRC) but generating the final image failed unless you were root, so I had to start as a non-root user then at some point do a

sudo find ./ -user $USER -exec chown -R root:root {} +

and then finish building as root. There were one or 2 package-specific fixes too as well (zstd required adding a -fPIC build flag, and one package needed some custom fix that I'm drawing a blank on currently).

I dont include these in the build instructions script since the specifics change all the time and depend on what snapshot you are building and what versions of packages are in the openwrt repos. In the rebuild Im working on right now Im troubleshooting a few packages that dont seem to want to build and they arent the same packages that were being difficult a month ago.

directnupe · 2024-05-24T17:35:42.705Z

Dear jkool702,
Hello and thank you so very much for getting back to me with the
good news about you being kind enough to ( at some point ) make available
to us ( The Greater OpenWRT Community ) your custom build with dnsmasq-full.
I certainly can understand empathize and relate to taking care of children and
our canine family members. Both though labor intensive are truly rewarding and
bring joy to all involved.
I have learned to be patient as I know that good people ( such as yourself ) have
" real everyday lives " - and your generosity of spirit time and expertise is
well worth the wait. No need for apologies on your behalf - indeed -
Thank You - and may
You and Yours Always Be Blessed With Peace and Happiness

BTW - although my daughter is graduated from university and on her own - I still am tapped with sitting for her Corgi when she is as busy as young folks are apt to get
Your pup is good looking for certain - I am sure the family will enjoy the companionship for years to come

odhiambo · 2024-05-25T11:34:29.021Z

Can you please publish an image with dnsmasq-full? I think it will be easier for some of us to just install the supplemental packages we require after the fact.

jkool702 · 2024-05-25T23:45:19.187Z

snk0911:

I wanted to ask if you could assist me to make a custom firmware for an outdoor 5g modem router. I am working together with the guys from the ROOTer Project and we made a general firmware for it. However I want to tweak it to the MT7621AT SoC which is built in.

I want to add the optimizations and NSS like you did. I just need someone who can help me a little bit out to understand things.

You customized the kernel for example. The timer is set to 100Hz now. What do I need to do to make it 1000 Hz? Can we get in contact together so I can try out things? It would be perfect!

So, now that I finally have a free few minutes Im trying to catch up on missed messages.

As far as creating a custom build: when you build openwrt from source, you have the opportunity to specify what exactly to include via a terminal-based customization menu. make menuconfig lets you customize userspace stuff (e.g., what packages to install from the openwrt repos). make kernel_menuconfig lets you customize kernel parameters (but can be tricky to use).

My custom firmware repo on github has the build script I used to make the firmware. Its fairly well-commented and works as a good, robust way to generate a custom firmware image with custom kernel. It also provides a fairly robust way of incorporating existing configs / kernel configs so that you dont have to reconfigure everything from scratch every time.

Note there are a lot of options you can configure...between standard config and kernel config there are probably 10,000 individual configuration "on/off" switches that you can select from. Be prepared to spend many many hours looking through these menus. SOOO many hours. Also be prepared to do a lot of "trail and error recompiling"...you all see the final polished result, But it probably took 20+ re-builds that iteratively improved each time (well, most times) to get there.

Note: the 100 hz --> 1000 hz change for the interrupt timer is one of the many options that you will be presented in the make kernel_menuconfig menu.

Regarding NSS - you are correct that (some) mediatek chips have a "network accelerator" on chip that performs a similar function to the NSS processor on ipq807x chips. BUT, @slh is correct that it is not actually the proprietary qualcomm NSS...it serves the same purpose, but is its own distinct chip. Unfortunately, this means that the ipq807x NSS work that has been done by people like @qosmio wont help you any more than your car keys help you open the door to your house. To continue the analogy: this situation is a bit like how even though your car keys and house keys are both keys that unlock a lock, that similarity is really not useful in practice if you are locked out of your house.

this thread has archived much of the process of getting NSS figured out. Its....not a trivial process to say the least, and that with open source code. I cant imagine it happening at all if mediatek is closed source...

directnupe · 2024-05-26T00:52:00.795Z

Any news on the dnsmasq-full REMIX - just asking ?

Peace

BTW - enjoy the new puppy

jkool702 · 2024-05-26T02:40:54.220Z

directnupe:

Any news on the dnsmasq-full REMIX - just asking ?

Im troubleshooting an issue on the new build where WAN isnt connecting. As soon as I get that worked out Ill push 2 versions of it (one using unbound, one using dnsmasq full) to a new branch in the github repo.

Good news is that everything else appears at 1st glance to be running as it should, and I have a couple of leads to follow for figuring out the WAN issue. But I really need to have WAN working before I officially share the build (for obvious reasons).

Its hard to give an exact timeline since some issues are more of a pain to straighten out and require much more time than others, but...probably somewhere in the range of "a couple of hours to a week". I'm actively working on it though.

directnupe · 2024-05-26T18:33:50.944Z

Thanks a ton @jkool702 for the update. I for one - am not trying to apply any undue pressure to you regarding the upcoming dnsmasq-full REMIX of your NSS Build. My opinion from my research leads me to believe that your release will be outstanding. So - as I mentioned - it is best to be patient - as I know that you are most thorough at your craftwork. Once again - I truly appreciate all that you do - have done - and are in the midst of undertaking for us all.

Peace

snk0911 · 2024-05-28T14:21:53.928Z

Thanks for pinning all the information down. There is another problem with the router I wanted to mod... The modem runs only half speed because of the usb3 driver. I saw in other threads already, it is a big topic. So I will go there to discuss the problem with the usb...

However, I am interested if NSS is actually actively working in the stock firmware of the Dynalink router. Do you have any infos about that? I am curious if they actually cared about the full utilization of the hardware.

odhiambo · 2024-05-28T16:34:01.369Z

Awesome news!

jkool702 · 2024-05-28T21:16:17.393Z

snk0911:

However, I am interested if NSS is actually actively working in the stock firmware of the Dynalink router. Do you have any infos about that?

I dont know, sorry.

Literally the 1st thing I did with mine was flash openwrt, so I really cant say anything about how performance was on stock firmware.

jkool702 · 2024-05-30T14:40:54.715Z

@directnupe @odhiambo

Im happy to say I finally figured out the wan connectivity issue and I have pushed an update to the github repo with the custum openwrt firmware for the dynalink dl-wrx36. The main-NSS branch (default branch) uses unbound + odchpd for DNS/DHCP and the main-NSS-dnsmasq branch uses dnsmasq-full for DNS/DHCP.

Real life has thrown me a few curve balls this week which has/will put a limit on how much time I have to thoroughly test this update....and as such this update isnt as well-tested as Id like. To be honest I havent actually even flashed the dnsmasq version...But the unbound version is working seemingly as it should and has been working well since I flashed it ~5 hours ago. And I know some of you have been waiting a good while for this to drop, so....

In the dnsmasq variant, I removed the custom configs for dhcp and network and firewall that I use in the unbound variant and let openwrt automatically generate default configurations for these (which it should do correctly for dnsmasq). As such you'll probably need to do some configuration to get this setup running optimally, but it should be working and functional. If not let me know and Ill flash mine temporarily with it and try and get it straightened out (just trying to avoid needless wear on the flash, since my routers flash gets overwritten a good deal more than most).

Let me know any feedback and if there are any problems. If anyone has a good dnsmasq configuration that would be good for "general usage" (better than the auto-generated one at least) feel free to post it and I can include it as the default dnsmasq config in future buillds.

directnupe · 2024-05-30T15:47:27.338Z

jkool702:

Let me know any feedback and if there are any problems

Dear @jkool702,
Thanks for the good news about the new dnsmasq-full edition of your custom build. I will flash the firmware and get back to you.
RE:
Real life has thrown me a few curve balls this week
I hope that you and yours are all doing well in the Big Scheme of Things.
Peace My Friend and Thanks One More Again

jkool702 · 2024-05-30T23:01:54.798Z

FYI - I experienced a few minor issues throughout the day with the build, so I rebuilt both the unbound and dnsmasq builds against the most recent openwrt snapshot and have updated both branches on github with the new versions. The config is basically the same, but the kernel got bumped to 6.6.32 in the most recent snapshot.

Initial testing of this one seems to indicate the issues I saw have been fixed, and Im pretty happy with the performance im seeing.

A few notes:

sysupdate works well if you do the following from a ssh shell:

# download + register build key 
cd /tmp
wget 'https://raw.githubusercontent.com/jkool702/openwrt-custom-builds/main-NSS-dnsmasq/WRX36/bin/extra/keys/key-build.pub'
opkg-key add /tmp/key-build.pub

# download sysupdate image
cd /tmp

# dnsmasq variant
wget https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS-dnsmasq/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin

# unbound variant
wget https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin

# test image. Result should be 2 lines that both say "OK"
sysupgrade -T /tmp/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin

# do sysupdate
sysupgrade /tmp/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin

this firmware can use the NSS to do fq-codel-based SQM on gigabit (and probably even faster) connections. It is mostly pre-setup (and tuned for a fiber internet connection), you just need to add your upload + download speed to the config at /etc/config/sqm. Id start by setting it at 95% of your rated download speed. If you have a symmetric connection (upload and download speed are the same) then use 95% of upload seed too, otherwise use the full upload speed.

On a 1000/500 fiber connection (that was also being used by others in the house), Im getting results like:

# speedtest-netperf.sh
2024-05-30 18:42:42 Starting speedtest for 30 seconds per transfer session.
Measure speed to netperf-west.bufferbloat.net (IPv4) while pinging gstatic.com.
Download and upload sessions are sequential, each with 8 simultaneous streams.
...............................
 Download: 873.75 Mbps
  Latency: [in msec, 31 pings, 0.00% packet loss]
      Min:  15.200
    10pct:  15.400
   Median:  16.000
      Avg:  16.600
    90pct:  18.400
      Max:  20.200
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 28 samples]
     cpu0:  11.3 +/-  4.1  @ 2208 MHz
     cpu1:   7.2 +/-  3.0  @ 2208 MHz
     cpu2:   7.6 +/-  2.6  @ 2208 MHz
     cpu3:  11.6 +/-  3.7  @ 2208 MHz
 Overhead: [in % used of total CPU available]
  netperf:  24.2
................................
   Upload: 456.16 Mbps
  Latency: [in msec, 32 pings, 0.00% packet loss]
      Min:  15.200
    10pct:  15.300
   Median:  15.800
      Avg:  15.791
    90pct:  16.000
      Max:  16.300
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 29 samples]
     cpu0:   4.5 +/-  1.6  @ 2208 MHz
     cpu1:   3.2 +/-  1.2  @ 2208 MHz
     cpu2:   1.8 +/-  1.0  @ 2208 MHz
     cpu3:   4.7 +/-  2.2  @ 2208 MHz
 Overhead: [in % used of total CPU available]
  netperf:   2.0

CPU load while doing this shouldn't be too significant, since NSS has taken over most of the networking stack, including the SQM part.

directnupe · 2024-05-31T01:44:12.997Z

Dear @jkool702,
Yes - I was going to get back to you with the issues I encountered which precluded me from being able to sucessfully flash your Build.
Truth to be told - I held off on that feedback because I did not want to seem ungrateful, pushy or inconsiderate of your time ( and Real life )
I am a bit tired - and will give your custom build another go tomorrow.
Thanks for being so diligent and fastidious at what you do do to bring NSS on this device to the masses.
Peace

jkool702 · 2024-05-31T16:55:21.946Z

Yeah sorry about that...the build I previously posted was a bit...rough around the edges (though I suppose that is what I get for finishing it up at 5 AM in the morning).

The unbound variant of the new build version seems to have been working as it should for the last day, so the core build for this version should be OK. That said, Im letting openwrt auto-configure all the network stuff on the dnsmasq version (since I havent used dnsmasq in forever), and if that doesnt work it might take a few tries to get the configuration right.

Also, dont feel bad about reporting issues. Ive sunk a ton of time into getting this build to where it is...id much rather spend another hour or two and have other people be able to use it than to have all my previous work go to waste because of a couple of fixable issues.

snk0911 · 2024-05-31T17:44:37.110Z

I updated, for some reason I get oddly slow speeds (5G connection):

image855×527 28.9 KB

My actual speeds are 500-550/150. Any ideas @jkool702?

PS: Yes. It is because of the firmware. I double-checked it

EDIT:
I just disabled SQM and I have almost full speed now. Mhm I also can't edit the SQM DL/UL values, permission denied...

EDIT2:
Still seems like I get some problems. I can't open certain websites some DNS errors, no access to OpenWRT forums. Dunno what happened. Maybe it does not like my router combo. The modem router in front of the Dynalink router has no bridge mode. It runs either static or dhcp.

sampson · 2024-06-01T00:57:45.080Z

Thank you very much for your new builds. Previous I flashed your unbound build, which I am not knowledgeable enough to use unbound. I would like to try your dnsmasq build.

I got SSL error when doing wget from a ssh shell into router.

jkool702:

BusyBox v1.36.1 (2024-05-15 15:26:20 UTC) built-in shell (ash)

| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -| || | | || || |
|_____|| |||||___||| |____|
|| W I R E L E S S F R E E D O M

OpenWrt SNAPSHOT, r0-9e60b5e6

#wget https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS-dnsmasq/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin

SSL error: SSL - Bad input parameters to function
Connection error: Connection failed

Below is doing same command in a Linux shell OK:

#wget https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS-dnsmasq/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin
--2024-06-01 08:53:04--  https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS-dnsmasq/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin
Resolving github.com (github.com)... 20.205.243.166
Connecting to github.com (github.com)|20.205.243.166|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/jkool702/openwrt-custom-builds/main-NSS-dnsmasq/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin [following]
--2024-06-01 08:53:04--  https://raw.githubusercontent.com/jkool702/openwrt-custom-builds/main-NSS-dnsmasq/WRX36/bin/targets/qualcommax/ipq807x/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 2606:50c0:8003::154, 2606:50c0:8000::154, 2606:50c0:8001::154, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|2606:50c0:8003::154|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 61266729 (58M) [application/octet-stream]
Saving to: ‘openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin’

openwrt-qualcommax-ipq807x-dy 100%[===============================================>]  58.43M   109MB/s    in 0.5s    

2024-06-01 08:53:09 (109 MB/s) - ‘openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin’ saved [61266729/61266729]

jkool702 · 2024-06-01T01:24:10.895Z

sampson:

I got SSL error when doing wget from a ssh shell into router.

On my router (running the new unbound variant on my firmware) it works. There is an option for whether you want to compile in SSL support to wget or not. I added this on my firmware, but the one in the default openwrt repos may not have ssl support (meaning no downloading from https:// addresses).

You can try just changing the https:// to http:// and it should probably work. Or, download it on you linux pc and then transfer it over to the router via

scp openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.binroot@X.X.X.X:/tmp

(where X.X.X.X is the router ip address). If that command doesnt work then this one should

cat openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin | ssh root@X.X.X.X 'cat > /tmp/openwrt-qualcommax-ipq807x-dynalink_dl-wrx36-squashfs-sysupgrade.bin'

egc · 2024-06-01T07:05:01.618Z

Use curl instead of wget if that is available

jkool702 · 2024-06-01T07:40:00.316Z

snk0911:

I just disabled SQM and I have almost full speed now. Mhm I also can't edit the SQM DL/UL values, permission denied...

Hmm....thats weird. Can you manually edit the config file (e.g., via vim /etc/config/sqm?)

SQM works as it should on the unbound variant. On my 1000/500 fiber connection with SQM enabled (download set to 950000,upload set to 500000) im getting results like this from the waveform test (and similiar results from fast.com):

920-950 mbps with +0-3ms latency for download,
470-490 mbps with no latency increase for upload

snk0911:

Still seems like I get some problems. I can't open certain websites some DNS errors, no access to OpenWRT forums.

This might be related to how dnsmasq is being automatically configured by openwrt. It might need some manual configuration embedded in the build (much like what I do with the unbound variant).

snk0911:

The modem router in front of the Dynalink router has no bridge mode. It runs either static or dhcp.

This might (or might not) play a role.

It might be worth quickly (and temporairly) flashing the unbound variant just to see if these problems persist or not, strictly for diagnostic purposes. If the issues go away then it is a good chance they are related to the dnsmasq configuration. If they are still there then it is probably either related to your specific setup or it is a bigger underlying issue with the build that I just havent run into yet.