Squid.. WARNING.. filedescriptors.. musl

I had some equipment fail b/c of storm damage.. so I'm trying to bandaid a few things..

I put squid on an openwrt box, and it works but getting the filedescriptors error..

and b/c the system is musl there is no /etc/security/limits.conf

I see unbound has a higher limit 4128 but cannot figure out how it's set..

any suggestions on increasing the file limits for a process?

ss -np | grep -c 3128
2578

squid

# cat /proc/`pidof squid`/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             63550                63550                processes
Max open files            1024                 4096                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       63550                63550                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

unbound

cat /proc/`pidof unbound`/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             63550                63550                processes
Max open files            4128                 4128                 files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       63550                63550                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us
 sysctl fs.file-max
fs.file-max = 1626010

I would think the defaults would be higher..

unbound uses something called PROCD.. ?

https://openwrt.org/docs/guide-developer/procd-init-scripts

still reading..

If anyone can offer a suggestion..

Thank you in advance.

https://openwrt.org/docs/guide-developer/procd-init-script-example

procd_set_param limits core="unlimited"

seems to get the 4096 available :slight_smile:

got something..

        procd_open_instance
        procd_set_param command $PROG -s -f $CONFIGFILE -N
        procd_set_param file $CONFIGFILE
        procd_set_param limits nofile="32768 32768"
        procd_set_param respawn
        procd_close_instance

and..

 cat /proc/`pidof squid`/limits
Limit                     Soft Limit           Hard Limit           Units
Max cpu time              unlimited            unlimited            seconds
Max file size             unlimited            unlimited            bytes
Max data size             unlimited            unlimited            bytes
Max stack size            8388608              unlimited            bytes
Max core file size        0                    unlimited            bytes
Max resident set          unlimited            unlimited            bytes
Max processes             63550                63550                processes
Max open files            16384                32768                files
Max locked memory         65536                65536                bytes
Max address space         unlimited            unlimited            bytes
Max file locks            unlimited            unlimited            locks
Max pending signals       63550                63550                signals
Max msgqueue size         819200               819200               bytes
Max nice priority         0                    0
Max realtime priority     0                    0
Max realtime timeout      unlimited            unlimited            us

https://openwrt.org/docs/guide-developer/procd-init-scripts#service_parameters

no way to make that easier?

but I guess running processes on a router isn't a normal thing..

Squid is such a anachronism when 99% of web traffic is encrypted...

Depends on the purpose. squid first of all is a caching proxy, but because of its many functions, caching not always the main purpose.

Using squid myself, to bypass all this IMHO bloated environment, I simply start squid from rc.local . And do the config for squid using squid.conf, only.

Squid can be an excellent policy management tool. For example I restrict my kids access to certain websites through squid, and these restrictions can be based on day and time and name of website (not IP address) and I tag packets from YouTube videos and use the tags to calculate quotas in nftables.

Caching isn't much use for encrypted stuff but policy management is still best done in a proxy.

Good idea. Could be mine :slight_smile: However, restrictions to access certain web sites I implement by means of DNS, in a customized DNS forwarder. Much more flexible, and faster, I guess.

YouTube Filter

request_header_add YouTube-Restrict moderate

I tended to have various schedules... for example perhaps saturday between 11am and 3pm you can access a game site, but sunday it's between 4pm and 6pm, and weekdays only 7pm to 8pm. I'm not sure how you'd do that with a DNS thing without writing what's essentially a DNS proxy with a policy engine like squid, and then once a DNS lookup has happened, nothing prevents the client from caching that response for hours and hours... so anyway there's still a role for squid in policy.

If you saved timevrising kids you can trust now it is time you figure out how to set up file descriptors.

Time based restrictions I implemented in customized DNS forwarder, too. Pars kept in DB, to be easily modifyable. In principle, you are correct, works like a very smart DNS-proxy. Wanted to do it as a commercial product, even, but did not find enough interest. My lack of marketing expertise/interest did not help, either :slight_smile:
... DNS lookup has happened, nothing prevents the client from caching that response for hours and hours...
I modified TTL, too, of course, to very small value. Or to ZERO, practically disabling DNS caching.
I used squid in this environment only for very limited policy functions. First of all, to block all IP-based URLs, trying to bypass DNS resolution.

I absolutely love Squid I have used it for years. It does an amazing job at url blocking. DNS url blockers have issues also. DoH and QBIC are harder to mitigate as its DNS over HTTPS and QBIC is https strictly over UDP. A get request is just that a get request again encrypted SNI and hello messages with TLS1.3 cause issues with Squid too. Again Squid can be configured to ignore stuff it can’t handle. I have no issues with Squid. I even use it with certificates and use ClamAV before items hit my secure PC. They all have there issues. I like the YouTube options also. Check out cachemgr.cgi with squid and filtering by MAC addresses on the host side too.