Build for Netgear R7800


#1447

Both dnsmaq AND adblock consume a growing amount of memory?

Specifically with this my build?
Or your own / snapshot?


#1448

I don't know exactly what happens. Seems if I go with about 773000 domains I get some lines in the kernel log that dnsmasq was killed because OOM.
To really verfiy this I need to revert to an older snapshot but the issues started around here I think.

It's my build using your scripts, nothing much extra added anymore so I can basically go back to your build now.


#1449

Hmm. 773k domains in adblock will consume quite a lot memory at dnsmasq. Simply thinking 20 char per domain and you have 15 MB right there. Likely much more.

Not quite sure where the practical limits for Adblock go, but 773k lines is a lot.
Cc @dibdot

Ps. you might discuss this at the adblock thread, as it likely has nothing to do with R7800 by itself.


#1450

dnsmasq uses 120Mb, nothing more than usual. I turned off backups in adblock and all seems good now.
But like I said, never had an issue in the past with that many domains. It doesnt seem like a memleak tho' beecause dnsmasq's memory is stable.
Could be that backups changed or something else uses a little more memory.


#1451

Then other devices with large adblock domain lists should suffer from OoM issues as well.

Might it have to do something with the (size of the) partitions?


#1452

Take a peek at the adblock init script

It's kind of uniq in the way it tries to asqynchronise it's downloading....

Got a feeling, something to do with it's logic.... relating to when and how it does things..... maybe a combination of @Videopac 's thoughts above about space interfering with clean setup completions....


#1453

IMHO much too much ... but I am not aware of such limit in adblock/dnsmasq.

Then you are using tmpfs for backups? Makes no sense - please prefer an usb stick or hard disk for that.

As @hnyman already mentioned probably unrelated to this build and the wrong thread for this.


#1454

I don't remember what it said but I don't think it only was backups that caused the increased memory usage, I found something else that makes more sense.
I just swiched to @hnyman's own build now he doesn't suppy tcpdump-mini so no domain statistics are generated. maybe these grew bigger and linux decided to kill the process that used most memory, which is dnsmasq.

Just for the sake of it I'm going back to a build that I know worked fine and try.

EDIT: Noticed the older build uses Adblock 3.6.4, the newer snapshots are 3.6.5.
@didot did you change something between the versions that could cause dns reporting to use more memory?


#1455

I find that hard to believe. DD-WRT is still worse than OpenWRT in its current state. It hasn't been truly stable for a very long time. I've gone thru multiple so-called "stable builds" over the years, and yet the old decrepit 14929 is still the most stable of them all.

When I got this router, I tried to install a recent "stable" KONG build on it, only to have my ISP turn off my Internet immediately! Somehow DD-WRT managed to overflow their NAT tables and they had to reset them manually!

Meanwhile, OpenWRT, for the most part, just works. The 17.01 build is always there in case things go really bad, but despite its speed and stability, it had major bugs. In DSLReports speed test, I got scores like C and B for bufferbloat and quality. With 18.06 I get all A's.

Of course I stick with the "stable" R7800-owrt1806-r7144 build, which was shortly after they fixed a slowdown, so it performs decently with "software acceleration" turned off... because that option is buggy.


#1456

I have very simple needs. I don’t really need to use a custom firmware (just like 99% of the users here) I just like to use CFW because I’m a nerd. (Also just like about 100% of the users here). Basically I just need some sort of QOS and maybe an adblocker.

Latest Kong for R7800 (as of now october 28, 2018, build 37495M) has been absolutely stable on my R7800. It's QOS may not be as up-to-date as Openwrt, and the configuration of Privoxy is not as point n click as Adblock in Openwrt, and I cant build my own custom build as easily, but it doesn't crash several times each day anymore and that's f****** beautiful. Has been rock solid for 4 days since I got fed up by OpenWrt. Will for sure go back OpenWrt if new builds become stable.


#1458

No, for reference see here: https://github.com/openwrt/packages/commit/be848a000ef6b09da3385849c79fc7c200aeef0c#diff-cff4d2df69efe7bc5b0f9ececd37f5d2


#1459

Yes, I've also seen this on one of my 64MB/32MB routers with ~40K domains. The problem I found is not with average memory usage, but transient usage. Using many adblock domains bloats dnsmasq and, since dnsmasq forks to handle each TCP request, this seems to consume more than usual memory (even with sharing) sometimes. If already low on memory, it's then easy to trigger the OOM reaper e.g. try opkg update. One option is to switch to ipset based blocking e.g. banIP.


#1460

Would you say the problem is new? For me it seems like it is but maybe I was just on the limit of what it could handle.

@hnyman Sorry for offtopic. :frowning:


#1461

Just trying to bring some closure to the tangent... And no, none of the OOM / dnsmasq / opkg / etc behaviour is new, just that it's a question of a safe and reliable operating point. But compared to my borderline 40K domains, your 800K seems far beyond :space_invader: .


#1462

Thank you. The conclusion I make is that something in OpenWRT might use a little bit more memory now so I hit the limit. I do have about 220mb mem free, I thought that was enough...
I only have one dnsmasq instance up tho', I see no forks, unless they close after a while.


#1463

I realize this is the OpenWrt forum but still:

Firmware:[ DD-WRT v3.0-r37495M kongat (10/28/18)]
Time: 02:02:05 up 13 days, 18:24, load average: 0.84, 0.64, 0.37
Wireless Packet Info
Received (RX) 136342904 OK, no error
Transmitted (TX) 192206497 OK, no error
(I do realize a small amount of TX/RX errors is normal, but I was kinda surprised there were none at all).

For reference I use a R7000 and EA6300v2 as wireless media bridges (with Advanced Tomato and DD-WRT), plus various iphones, ipads, androids, usb wireless clients, laptops and desktop machines with internal WLAN cards and friends over as well with their mobile phones.
I'll stfu now :slight_smile:


#1464

That's pretty cool. I am keeping that Kong build just in case I start using this router for WiFi. Wonder if DD-WRT is stabler than OWRT when you enable their version of NAT acceleration (SFE?). My OWRT build can hang after 6 days of that enabled. Sometimes.


#1465

What ath10k driver does DDWRT use that is stable compared to the one in OpenWRT?


#1466

Idk about DDWRT but afaik there are no other drivers than the ones that exists in linux by default, you can toy with the official firmwares in openwrt and see if you find one that's works for you.

The non-CT driver/fw should be quite stable in openwrt now tho'. I'm currently not experiencing issues with CT.


#1467

Even with the non-ct driver I still see crashes in dmesg :frowning: