Full-featured custom build for Dynalink DL-WRX36 (AX3600)

I will try the dnsmasq version later but now back to the unbound version as this is more interesting now.

It is not the adblocker in the unbound version which causes the problems. Maybe it is a problem with the DNS loopback causing a glitch. You refered to my ISP router, I would not call it ISP router, it is a chinese Suncomm Model O3 Outdoor 5G router which runs a custom version of LEDE.

The Suncomm Router has following DNS configured while the DL-WRX36 behind that router has everything set out of box:
image

My question is:
Do I need to actually use the IP of my WRX36 router as my DNS IP?

Because of the Static or DHCP connection the DNS seems to glitch out. My modem router in front does not have a bridge mode and I am 100% when it would be bridged, your unbound variant would run perfectly.

So, the builtin setup (that works seemingly well for me) is to use the router's IP for the LAN-side DNS (10.0.0.1 for LAN on lan 1-3 + wifi, 192.168.0.1 for IoT on lan4) and loopback (127.0.0.1) for WAN-side DNS. I used loopback for WAN-side since my WAN address is DHCP assigned and as such can change. However, unbound is setup to only serve DNS queries on LAN/IoT, and since any WAN-side DNS queries get rejected anyways Im not sure if it really matters what that is set to.

That said, since (id assume) in your setup your WAN side ip is static you could try setting it to the WAN ip address and see if it changes anything.


I also have another test that is probably worth trying. Run the following

cat<<'EOF' >>/etc/config/unbound
config zone 'fwd_cloudflare'
        option enabled '1'
        option fallback '1'
        option tls_index 'cloudflare-dns.com'
        option tls_upstream '1'
        option zone_type 'forward_zone'
        list server '1.1.1.1'
        list server '1.0.0.1'
        list server '2606:4700:4700::1111'
        list server '2606:4700:4700::1001'
        list zone_name '.'
EOF

uci set unbound.auth_icann.enabled='0'
uci commit unbound

uci set firewall.adblock_lan53.enabled='0'
uci set firewall.adblock_IoT53.enabled='0'
uci commit firewall

uci set network.wan.dns='1.1.1.1'
uci commit network

service firewall restart

service unbound restart

service network restart

Running the above code should switch unbound over to "forwarding mode", where it resolves DNS queries by forwarding them on to another public DNS resolver (much like how dnsmasq works), and disables the DNS firewall rules that redirect all DNS traffic back to the router.

It will be interesting to see if unbound works properly as a forwarding resolver.

The reason why I suggest this test is because it is my understanding that, while 5G has speeds comparable to cable or even fiber internet, latency can often be significantly worse and is prone to huge spikes. How bad this issue is depends on what type of 5G internet you have, but from looking into it quickly it seems like unless you have mmWave with a great signal strength it is going to be an issue. This issue might also be "worse than usual" for traffic querying DNF authoratitive servers recursively, as Id guess your 5G ISP is prioritizing traffic and this sort of traffic probably gets prioritized pretty low (since 99.9% of people use a public DNS resolver instead).

So, DNS querier are basically a question asking

what is the ip address of the computer at ___________ (e.g., at www.google.com)

Unbound in recursion mode (which is the default for my firmware) recursively queries different authoritative dns servers to answer this question. It sends a request to a server in the root zone, the response it gets tells it the next server it should ask, it sends that server the request, that servers response tells it the next server to ask, etc. This repeats until it finds a server that actually knows the answer to your DNS query, and then that name-to-ip-address mapping is saved in unbound's cache so that the next time you make the same DNS query it doesnt have to go through that whole process.

Point is, IF all that back-and-forth communication is subject to a terrible latency penalty for every individual message to/from any of the authoritative DNS servers you are talking to, I could see this causing "weird DNS issues and overall bad performance". By using a forwarding resolver you only have to pay that latency penalty once, and then your chosen DNS forwarder (cloudflare/1.1.1.1) will either tell you the answer from its cache or it will go through that whole recursive process to find the answer for you and then tell you what the answer is.

At any rate, if switching unbound to forwarding mode fixes everything, Id guess that your 5G latency is too high for unbound to work effectively as a recursive resolver.

This is fantastic to hear!

1 Like

I ended up trying out your latest release (unbound) and was really impressed with the performance! I read up on your explanation of using recursive dns and it sure improved my dns query performance overall. I previously used to upload dns blocklists to the cloudflare zero trust and used their custom dns gateway.

The adblock+unbound setup works flawlessly out of the box.

I did have a problem trying to install some of your kmods from the repo while I was trying to install mwan3 but got kernel incompatibility errors. Were the kmods rebuilt for the kernel in the latest build? I tried installing through the .ipk files directly on the router but same error.

Glad to hear it is working well!

I really like unbound...not only is performance stellar (even with an adblock-generated blacklist with 600k+ entries,) but compared to using a public resolver it is both more private (because you dont give anyone a list of all your DNS queries) and more secure (because no one is going to "randomly" try and poison a DNS server that only serves your local network).

On devices like the WRX36 that have sufficient memory and CPU power to run unbound with optimal settings (and with the extra goodies like libevent2 and pthreads compiled in) Id always reccomend unbound over dnsmasq unless you were running something (like pbr) that has dnsmasq as a required package.

its worth noting that if a package like mwan3 has dependent kmod packages, youll probably need to install those manually before you install the package (otherwise opkg will try to get them from the main repos). Which means youll need to first install kmod-ipt-ipset, kmod-ipt-conncount, kmod-ipt-conntrack, and kmod-ipt-conntrack-extra from here. So, if you havent already tried doing that, try that and see if it works.


If not...

kmods were for sure generated during that build...my system has enough ram (128 gb) that I build on a tmpfs, enough CPU power (i9-7940x...14c/28t @ 4.2 ghz) that rebuilding everything doesnt take that long, and fast enough internet (1000/500 fiber) that cloning the openwrt git repo is only a few seconds, so every time I rebuild I start from scratch. So the kmods had to have been built in that same build run.

That said, thinking back I may have had to go change something minor in the configuration after almost everything was built because something wasnt building right. This might have triggered a minor change to the kernel and changed its .vermagic.

Unfortunately my system has been restarted since that firmware was built, so that build root is gone. I'll try and rebuild it within the next day or two.

While I bought this router specifically to be able to run adblocking and VPN without a hit on performance, I didn't put the effort to set it up since it seemed like a lot of hassle, especially unbound. So, I stuck with stubby+dnsmasq with a custom cloudflare gateway. Bonus was I could use the same gateway on mobile devices while I was away from home.

In my short testing of the adblock lists to use, the oisd big list seemed sufficient. While I was limited to 300k domains on CF, I can go crazy with 600k here as you say haha. I was wondering what combination of lists work well for you?

I used PBR before but since I want to keep using unbound it does not seem like an option anymore. mwan3 provides the same fuctionality and should work fine with unbound so I was hoping to set it up. I can also leverage it's load balancing feature which in my testing allowed me to substantially increase my multi-stream downloads/uploads over my 2 fiber connections.

Yes, I did try to install those kmods manually via gui and also directly though cli but got a kernel hash mismatch error which is probably due to that. I would have offered to build it myself but I do not unfortuanately have a beefy enough cpu/ram combo. I will wait for you to build and push the update.

Another thing I noticed was the pre-configuration of subnets that you mentioned on the README was't present on a fresh install and everything was openwrt default. Even ubound+adblock was not enabled.

Also, the custom opkg update configuration tries to check on the official openwrt sites (leading to errors since nss pkgs are not available) but should instead hit the git repo, right?

This is entirely fair. Ive been using this unbound+adblock setup for years now, but figuring out how to initially set this all up was a major time-consuming pain in the ass that took more than a few tries to get right.

I use what is baked into the firmware, though I'll be honest I havent really thoroughly gone through and tested all the various sources....I basically took the defaults, added in all the extra (non-foreign) sources that specifically were for phishing, tracking, spam, malware, ads, or telemetry (not the large general/compilation ones), then disabled one or two that ended up causing problems at some point.

Right now, LUCI tells me Im using

Blocked Domains: 615,473
Active Sources: adaway, adguard, adguard_tracking, android_tracking, anti_ad, anudeep, bitcoin, disconnect, easylist, easyprivacy, firetv_tracking, notracking, openphish, phishing_army, smarttv_tracking, spam404, wally3k, whocares, winspy, yoyo

Ive never used it, but the wiki indicates there is a "PBR with netifd" option that "requires no extra software installation". This sounds like it might be a PBR+unbound solution, though mwan3+unbound sounds like it might suit your needs better.

The update has been pushed (for the unbound variant at least....the dnsmasq variant will have to wait till tomorrow).

I also figured out how to add the git repo into the feeds, though its not included in this update (i'll add it to the next one).

So, as far as mwan3 goes, it should be possible to install it, but due to some quirks with how opkg works (and its seeming lack of support for prioritizing one repo over another one), the process is sort of ass backwards (since you need the kmods from my repo but an iptables package that I didnt build from the openwrt repos). The following script should (i think) install mwan3 successfully

# add github repo to feeds
cat<<'EOF' >> /etc/opkg/customfeeds.conf
src/gz custom-firmware_core https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/targets/qualcommax/ipq807x/packages
src/gz custom-firmware_base https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/base
src/gz custom-firmware_luci https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/luci
src/gz custom-firmware_nss_packages https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/nss_packages
src/gz custom-firmware_packages https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/packages
src/gz custom-firmware_routing https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/routing
src/gz custom-firmware_sqm_scripts_nss https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/sqm_scripts_nss
src/gz custom-firmware_telephony https://github.com/jkool702/openwrt-custom-builds/raw/main-NSS/WRX36/bin/packages/aarch64_cortex-a53/telephony
EOF

# temporairly remove "official" openwrt_core repo (this is the repo with the kmods)
echo '#'"$(cat /etc/opkg/distfeeds.conf)" > /etc/opkg/distfeeds.conf
 
# update opkg lists
opkg update

# download iptables-mod-conntrack-extra package (that I didnt build) with wget, 
# then install it and its required kmods (that I did build) using kmod packages from the github repo
cd /tmp
wget https://downloads.openwrt.org/snapshots/targets/qualcommax/ipq807x/packages/iptables-mod-conntrack-extra_1.8.8-r2_aarch64_cortex-a53.ipk
opkg install --noaction /tmp/iptables-mod-conntrack-extra_1.8.8-r2_aarch64_cortex-a53.ipk

# install mwan3
opkg install mwan3

# re-enable openwrt_core repo
sed -i s/'#'// /etc/opkg/distfeeds.conf

Hmmmm. That isnt supposed to happen.

A few builds ago I switched from directly including the files in /etc/config/<...> to setting them up using uci commands in /etc/uci-defaults. The intent was to

  1. better adapt to the "base" config being updated (since you can just modify the specific config items you need to instead of replacing the entire file with all the config in it), and
  2. You could see that the standard built-in config is since that will be in /rom/etc/config/<...>, which is useful for troubleshooting.

To avoid overwriting configs from your previous firmware during sysupgrade, I added a check where my configs wont get applied if both a) the config file exists under /etc/config and b) it differs from the version in /rom/etc/config (meaning it was already changed after the firmware was flashed). This part seems to not be working like how it is supposed to....Ill have to look into this.

I can see why, considering that I'm facing some DNS issues since I do not have your pre-configs. And you're doing us all a favour, thank you!

I see, thanks for that. I'll definitely explore more.

I'd have to figure that on my own if not for your script. It worked wonderfully well!

That's interesting. I installed your latest version too where you tried to fix the configs but unfortunately that too didn't fix the issue (fresh install without keeping previous config).

sorry about that...

I finally had a few minutes to more thoroughly investigate this. I believe that, with any luck, the update I just pushed (for the unbound variant) a few minutes ago will work for a fresh install without any previous config.

Turns out I had a number of small issues related to the uci commands I was using to add the configuration via uci-defaults. This ultimately resulted in most of the configuration either not being applied or getting mangled before being applied.

I think all these issues have been fixed. To "recreate" applying these uci-defaults scripts: I set up a tmpfs overlay of /rom on the router at an alternate mountpoint, chroot-ed into it, applied the uci-defaults scripts and diff-ed the resulting configs against the working ones. As far as I could tell, all the configs were being applied as intended, at least in this test.

I installed your latest custom build and saw much improved performance, thank you!

I'm having a small issue using nlbwmon with the luci GUI though. Has anyone else tried it with this build? Not sure if it's specific to the build or a user error on my part.

Hi, can someone tell me how to install wireguard ? do i do it from software tab or from repository ? i keep getting kernel errors, I'm out of my comfort zone here. If it's from the repo can someone give me the steps to do it.

Thanks

Ryan

So i have installed the unbound version and your config has been installed correctly.

@jkool702, thanks for your great work on this full-featured fork.

However, I would like some help, as for some reason I don't understand, I've faced countless resets and reconfigurations, which are very disheartening.

EDIT #2: I fixed everything I was complaining about by learning odhcpd and to properly create bridge devices. Everything was easier than I thought. Go figure!

The problem started when I had to configure a UPnP daemon, as some of the software and hardware pieces I have on my network use it to automatically open ports (mainly my Xbox and a couple of small servers I have around). I know, insecure insecure, but I still need it.

I tried the miniupnp installation and got this:

Multiple packages (libgcc1 and libgcc1) providing same name marked HOLD or PREFER. Using latest.
Multiple packages (libgcc1 and libgcc1) providing same name marked HOLD or PREFER. Using latest.
Installing luci-app-upnp (24.148.86114~d174bb8) to root...
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/luci/luci-app-upnp_24.148.86114~d174bb8_all.ipk
Installing libcap-ng (0.8.4-r1) to root...
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/packages/libcap-ng_0.8.4-r1_aarch64_cortex-a53.ipk
Installing miniupnpd-nftables (2.3.6-r1) to root...
Downloading https://downloads.openwrt.org/snapshots/packages/aarch64_cortex-a53/packages/miniupnpd-nftables_2.3.6-r1_aarch64_cortex-a53.ipk
Configuring libcap-ng.
Configuring miniupnpd-nftables.
Configuring luci-app-upnp.

Errors:

Collected errors:
 * pkg_hash_check_unresolved: cannot find dependency kernel (= 6.6.35~f37c2fdbdb753b455ef8cac06b859604-r1) for kmod-nfnetlink
 * pkg_hash_check_unresolved: cannot find dependency kernel (= 6.6.35~f37c2fdbdb753b455ef8cac06b859604-r1) for kmod-nf-conntrack
 * pkg_hash_check_unresolved: cannot find dependency kernel (= 6.6.35~f37c2fdbdb753b455ef8cac06b859604-r1) for kmod-crypto-crc32c
 * pkg_hash_check_unresolved: cannot find dependency kernel (= 6.6.35~f37c2fdbdb753b455ef8cac06b859604-r1) for kmod-nft-core
 * pkg_hash_check_unresolved: cannot find dependency kernel (= 6.6.35~f37c2fdbdb753b455ef8cac06b859604-r1) for kmod-nft-compat
 * pkg_hash_check_unresolved: cannot find dependency kernel (= 6.6.35~f37c2fdbdb753b455ef8cac06b859604-r1) for kmod-nf-conntrack-netlink

EDIT #3: Oh, I'm so freaking blind. Basically what this is complaining is that the version of the kernel in this image doesn't match the one required by the module. Again, is there any way to either update the kernel, or install a previous version of these modules so that they are compatible?

Is your nlbwmon error the same/similar or different to this error.

Sort of similar, but I'm not seeing any data collected at any point, unlike that post. I had it working on a snapshot before, but I installed on this build and it never seems to update and nlbw -c show returns just the headers of the usual data output.

Thank you for this build. I installed this a few days ago, moving from the upstream snapshot build, but I had some big issues with the unbound version.

I installed it through luci and, when asked, chose to not keep my existing configuration to avoid any conflicts with your setup. But I found that the big majority of sites don't load with the unbound variant, I immediately get that they don't exist, the only exception I could find was google sites, there might be others, but that's the one I found, even openwrt.org couldn't be pinged. I'm guessing it needs some changes on top of the configuration it has on by default.

On the other hand, the dnsmasq build worked for me, so that's what I've been using.

General question, is it me or does the router blink red for too long when rebooting? Even after the router is reachable and its serving devices, it keeps blinking for a while.

Is NSS enabled by default, or do we have to toggle it ourselves? Can it be used alongside SQM or does one preclude the other?

1 Like

I have been trying to find why my connection gets so terrible when there are 2 devices doing anything meaningful. I have a gigabit connection, and if I'm playing online games on my computer (wired) and my phone is downloading something at 2.5MB/s, the online game gets unplayable over the increased lag and dropped packages.

So I decided to look up what testing I could do and found out about flent. I tested with 2 computers connected to it, one test with one wired (the other one was 1 feet away and support wifi 6), and the other test with both connected through ethernet cables(the one below).

These results look pretty bad to me, could anyone else run flent as well? I'm worried that my hardware could be faulty.

1 Like

I'm experiencing something similar right now. I've had to add a crontab to restart the router everyday to avoid having this issue.

In my case, after about 48 hours, my TVs keep getting in and out the WiFi network, causing all sorts of issues when casting to them, all while some other devices don't seem to have any noticeable issues.

Just wanted to check on the status of this build/current issues? I actually have a need for the iot subnet set up of this build so i was considering updating to it but I just saw the most recent replies about devices disconnecting and wanted to see if that is still an issue?

If so, I guess I'll be looking into just using the other nss enabled build and figuring out the iot subnet set up later. Thanks!

Hey, unfortunately, we haven't heard from the dev of this build for quite a while now. Disconnections are still happening, i.e., my TV keeps getting in and out of the WLAN every 10 to 12 minutes. Nothing is printed to the logs.

What other NSS enabled build did you mention?