All well then, just keep in mind that if you now do a transfer to/from say 100ms away you likely will get lower throughput (especially if there is competition with a bulk transfer to/from say 20ms away).
Mind you, these might well be the trade-offs you are willing to make, I just want to be sure you are aware of the (potential) consequences...
Fully understand and willing to accept the trade-offs. Further distant TCP transfers are impacted by the higher latency anyway. The two connections I am mostly concerned with are lower latency connections. My work VPN connection peers closely with my ISP through one intermediary and when I play first-person shooters, thanks to Valve, my ISP directly peers with their transit network.
Thank you for taking the time to make me aware of these in case I didn't know. 
two questions P-States and C-States, do you deactivate it from the bios or within openwrt, is there something to do?
and this script "cake-autorate (config.primary.sh):" where you would put it
1 - BIOS settings, if available, machine dependent
2 - cake-autorate thread:
cake-autorate instructions:
Intel P-states, if supported, can be edited in the grub config as shown below. Add the variables at the end of the applicable kernel menu entry, save, and reboot to see the changes
root@OpenWrt:~# cat /boot/grub/grub.cfg
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 --rtscts=off
terminal_input console serial; terminal_output console serial
set default="0"
set timeout="5"
set root='(hd0,msdos1)'
menuentry "OpenWrt" {
linux /boot/vmlinuz root=PARTUUID=80cdd189-02 rootwait console=tty0 console=ttyS0,115200n8 noinitrd intel_pstate=passive
}
menuentry "OpenWrt (failsafe)" {
linux /boot/vmlinuz failsafe=true root=PARTUUID=80cdd189-02 rootwait console=tty0 console=ttyS0,115200n8 noinitrd
}
Check the status:
root@OpenWrt:~# dmesg | grep intel_pstate:
[ 2.994505] intel_pstate: Intel P-state driver initializing
For what it's worth, this is what I have set up in my grub.cfg:
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1 --rtscts=off
terminal_input console serial; terminal_output console serial
set default="0"
set timeout="5"
search -l kernel -s root
menuentry "OpenWrt" {
linux /boot/vmlinuz root=PARTUUID=80cdd189-4ce8-125d-9e5c-32a6bb5c6502 rootwait console=tty0 console=ttyS0,115200n8 noinitrd intel_pstate=passive ipv6.disable=1 processor.max_cstate=1 idle=poll intel_idle.max_cstate=0
}
menuentry "OpenWrt (failsafe)" {
linux /boot/vmlinuz failsafe=true root=PARTUUID=80cdd189-4ce8-125d-9e5c-32a6bb5c6502 rootwait console=tty0 console=ttyS0,115200n8 noinitrd
}
It works well. I also found that my ISP dropped my speed from 950-1000Mbps without SQM enabled to 900Mbps, which is why I had bufferbloat to begin with (plus some misconfigurations and lacking optimizations that everyone here in this thread helped me with, which I am grateful for). I lowered my SQM setting to 700000 and 30000 for upload (because that was also lowered) and now I have solid connectivity, even without the cake-autorate.
My sqm config:
config queue 'wan'
option qdisc 'cake'
option script 'layer_cake.qos'
option interface 'eth1'
option linklayer 'none'
option download '700000'
option upload '30000'
option enabled '1'
option debug_logging '1'
option verbosity '5'
option squash_ingress '0'
option ingress_ecn 'ECN'
option egress_ecn 'ECN'
option squash_dscp '0'
option qdisc_advanced '1'
option qdisc_really_really_advanced '1'
option iqdisc_opts 'docsis besteffort ingress nat dual-dsthost rtt 20ms'
option eqdisc_opts 'docsis nat ack-filter dual-srchost rtt 20ms'
Hope this helps someone.
Reposted because I accidentally replied to mindwolf and didn't mean to.
I see you have layer_cake.qos selected. You'll need tcpdump installed first, but you can run this these two lines seperately in an ssh session to verify if your ISP respects DSCP markings. The majority of US based ISPs only respect COS or DSCP markings e2e within their domains.
# Show only live TOS Hex bits:
tcpdump -v -i eth1 | grep -o 'tos 0x[0-9a-f]\+'
Next convert those to DSCP Hex, Decimal, or Class
https://www.tucny.com/networking/dscp-tos
When I run the command you suggested, I get a majority of 0x0 with a sprinkle of 0x28 (translated to "priority" based on the guide you shared). Would this then confirm that the ISP respects the markings?
So I use the following tcpdump invocation:
tcpdump -i pppoe-wan -v -n '(ip and (ip[1] & 0xfc) >> 2 != 0)' or '(ip6 and (ip6[0:2] & 0xfc0) >> 4 != 0)'
so tcpdump goves me reports for any packet (IPv4 or IPv6) that carries a different DSCP than zero...
No, to test that you need to measure whether outside of your router packets with specific DSCPs get differential treatment (lower/higher latency, or lower/higher loss, or lower/higher throughput). But realistically an ISP is likely going to neuter your egress DSCPs anyway, that is they either ignore these (treat these exactly like BE/CS0) or even re-map these to the actual CS0 bitpattern...
Similar for ingress quite unlikely that your ISP is going to send you something really useful...
DSCPs make most sense inside a controlled domain like your home network, but that means you need to make sure incoming DSCPs match your own rules and desires (see the cake-qos-simple and DSCP classify project for clever ways of assuring that, if you are so inclined...)
I should have been clearer for the use on device eth1, which is exiting your domain to the ISPs' subnet. You can toggle these to off (0) inside /etc/config/sqm and run either commands me and Moeller posted. There are many variations to display different results.
option squash_dscp '1'
option squash_ingress '1'
IF you do indeed see anything other than 0x0, well happy marking!
I'm curious as to why your set RTT is 20ms?
I was getting poor results on deviation of latency when running the bufferbloat test. This helped lower the variance.
This will first use your internally set DSCPs to select the priority tin but then reset the DSCP value to CS0, so you get your desired priority selection in cake but do not leak your internal DSCPs into your ISP's network...
Ah, I stand corrected on that one. Thank you.
Typical RTTs nowadays in America coast2coast are between 70-105ms. It's possible either your tap at the dmark is going bad or your neighborhood is over-subscribed.
Docsis 3.1 optimal ranges are typically between 40-47 dBmV. Your figures are indeed within spec.
Just wanted to update everyone on a finding that came to light after placing a technician call to my ISP for daily T3 errors that happened around the same timeframe each morning (10am-noon). Technician came out and mentioned he couldn't find anything wrong but did mention that my neighbor across the street also reported the same intermittent behavior. My ISP is sending out a maintenance truck for the potential tap or line issue that's impacting both of us. So maybe, as it turns out, this whole issue is because of failing hardware on the main trunk line. Will edit this post or add a new one once I find out if it's resolved after they come out tomorrow.
Apologies on the delay. While I do think the issue was related to the temperature changes impacting the strength of the signal, I think there's still another underlying issue causing much lower throughput than what I'm paying for. The maintenance techs made adjustments on the line, which did help lower T3 error frequency and complete drops, though.
So the good news is that after my ISP has made some changes, my advertised download and upload is what I expect it to be (1G/50M). DOCSIS 3.1 on downstream with SC-QAM on upload.
However, this is with SQM off. When I enable it, my bufferbloat is actually even worse on download. Lowering the limits does not help.
Here are my configs:
rc.local:
# Put your custom commands here that should be executed once
# the system init finished. By default this file does nothing.
dmesg -n 1
# This will disable eth2
echo 1 > /sys/bus/pci/devices/0000:04:00.0/remove
# Optimizations for eth0 and eth1
ethtool --set-eee eth0 eee off
ethtool --set-eee eth1 eee off
ethtool -A eth0 autoneg off rx off tx off
ethtool -A eth1 autoneg off rx off tx off
ethtool -K eth0 hw-tc-offload off
ethtool -K eth1 hw-tc-offload off
ethtool -G eth0 rx 2048
ethtool -G eth0 tx 2048
ethtool -G eth1 rx 2048
ethtool -G eth1 tx 2048
echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu2/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu3/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu4/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu5/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu6/cpufreq/scaling_governor
echo performance > /sys/devices/system/cpu/cpu7/cpufreq/scaling_governor
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy0/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy1/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy2/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy3/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy4/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy5/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy6/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy7/scaling_min_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy0/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy1/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy2/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy3/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy4/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy5/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy6/scaling_max_freq
echo -n '2500000' > /sys/devices/system/cpu/cpufreq/policy7/scaling_max_freq
(sleep 15; /etc/init.d/ddns restart)&
exit 0
/etc/config/sqm:
config queue
option enabled '1'
option interface 'eth1'
option download '1000000'
option upload '50000'
option debug_logging '1'
option verbosity '5'
option qdisc 'cake'
option script 'piece_of_cake.qos'
option linklayer 'ethernet'
option overhead '44'
Intel i3-N300 & OpenWrt 23.05.3 (r23809-234f1a2efa) & irqbalance enabled & Packet Steering Enabled (was disabled but does not seem to make a difference)