Thanks for sharing these, will try it out on a MT621A platform I'm trying to tune.
To improve my testing setup, I picked up a 4-port gigabit PCIe NIC (HP NC365T) and installed it in my "lab" PC. It shows up as four network interfaces. My testing reported earlier in this thread depended on a USB NIC which I found could not reach 900Mbps in iperf3 in bidirectional mode.
I created 3 separate systemd-nspawn containers, each owning one of the network interfaces:
- 10.0.0.1 (static) with DHCP server, attached to router-under-test WAN port
- dynamic IP, attached to router-under-test LAN port 1
- 192.168.1.10 (static), attached to router-under-test LAN port 2 (if available)
The container on LAN port 1 acts as a testing client, while the container on the WAN port acts as the testing server. I use the container on LAN port 2 as a best-case baseline, since by default traffic flows between the LAN ports just through the router's switch.
I've been running iperf3 with 1, 4, and 16 parallel connections in normal (UP), reverse (DN), and bidirectional (BI) modes.
I also added the caddy HTTP server in the "server" container and run the hey HTTP load-tester on the testing client, recording the requests-per-second result getting a 4MiB random file with 50 concurrent workers for 60 seconds.
Both the iperf3 and caddy/hey tests are run with SW flow offloading off and on, and with SQM enabled but with 800Mbps/800Mbps rate limits.
Here's what I have so far, with just the Linksys EA8500, running 19.07.4 without any special tuning. The rates are in Mbps or requests per second.
SQM SW OFFLOAD ROUTER 1:UP/DN/BI 4:UP/DN/BI 16:UP/DN/BI caddy-hey-4M
NA NA Switch only 942/940/927 933/931/926 928/926/920 28.1
OFF OFF EA8500 932/893/361 25/718/236-433 553/540/224-261 13.8
OFF ON EA8500 932/932/463 914/864/430-459 869/833/373-403 22.3
800/800 OFF EA8500 762/495/2.2-760 598/411/2.5-608 486/322/2.4-487 9.4
Like before, I'm seeing one direction of the bidirectional iperf3 test having very low rates (around 2.5 Mbps) with SQM enabled. I'll see if this shows up on the Raspberry Pi 4 again---it seemed to be common to the two ARM-based routers I tested.
If there are any suggestions on my test methods, now would be a great time to share them. I'm about to run the same tests on several other routers.
Intriguing, could you post (again) the output of tc -s qdisc from just before and after the iperf3 test, as well as the iperf output and a redacted (check for passwords and maybe IP addresses) version of the output of dmesg and logread (in both cases it should be sufficient to just post the output starting around the time you did the first tc -s qdisc invocation). Also run top -d 1 on the router during the test and observe the %idel column for signs of CPU overload.
And for ARM CPUs it also is important to rule out voltage/frequency scaling (I believe the performance governor should be selected). And finally receive packet steering might cause issues as it might cause all SQM processing to bunch up on the same CPU resulting potentially in one CPU idling while the other is overloaded.
OK, I've run the single-connection bidirectional iperf3 test again, collecting before and after information.
It appears the OpenWrt default for this router uses the ondemand governor:
# cat /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
ondemand
iperf3 output (the RX-C result was much faster this time than last time, at the expense of the RX-C result):
$ iperf3 -c 10.0.0.1 --bidir --time 60 | tee -a iperf3-output.txt
Connecting to host 10.0.0.1, port 5201
[ 5] local 192.168.1.225 port 36008 connected to 10.0.0.1 port 5201
[ 7] local 192.168.1.225 port 36010 connected to 10.0.0.1 port 5201
[ ID][Role] Interval Transfer Bitrate Retr Cwnd
[ 5][TX-C] 0.00-1.00 sec 74.0 MBytes 621 Mbits/sec 0 3.06 MBytes
[ 7][RX-C] 0.00-1.00 sec 11.7 MBytes 98.2 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 90.0 MBytes 755 Mbits/sec 355 2.28 MBytes
[ 7][RX-C] 1.00-2.00 sec 877 KBytes 7.18 Mbits/sec
[ 5][TX-C] 2.00-3.00 sec 90.0 MBytes 755 Mbits/sec 451 1.71 MBytes
[ 7][RX-C] 2.00-3.00 sec 382 KBytes 3.13 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 90.0 MBytes 755 Mbits/sec 159 1.28 MBytes
[ 7][RX-C] 3.00-4.00 sec 967 KBytes 7.92 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 88.8 MBytes 744 Mbits/sec 163 987 KBytes
[ 7][RX-C] 4.00-5.00 sec 2.61 MBytes 21.9 Mbits/sec
[ 5][TX-C] 5.00-6.00 sec 78.8 MBytes 661 Mbits/sec 141 755 KBytes
[ 7][RX-C] 5.00-6.00 sec 9.45 MBytes 79.3 Mbits/sec
[ 5][TX-C] 6.00-7.00 sec 70.0 MBytes 587 Mbits/sec 9 598 KBytes
[ 7][RX-C] 6.00-7.00 sec 17.6 MBytes 147 Mbits/sec
[ 5][TX-C] 7.00-8.00 sec 66.2 MBytes 556 Mbits/sec 0 680 KBytes
[ 7][RX-C] 7.00-8.00 sec 20.6 MBytes 173 Mbits/sec
[ 5][TX-C] 8.00-9.00 sec 71.2 MBytes 598 Mbits/sec 0 755 KBytes
[ 7][RX-C] 8.00-9.00 sec 15.1 MBytes 127 Mbits/sec
[ 5][TX-C] 9.00-10.00 sec 68.8 MBytes 577 Mbits/sec 16 597 KBytes
[ 7][RX-C] 9.00-10.00 sec 18.0 MBytes 151 Mbits/sec
[ 5][TX-C] 10.00-11.00 sec 65.0 MBytes 545 Mbits/sec 0 673 KBytes
[ 7][RX-C] 10.00-11.00 sec 21.2 MBytes 177 Mbits/sec
[ 5][TX-C] 11.00-12.00 sec 71.2 MBytes 598 Mbits/sec 0 752 KBytes
[ 7][RX-C] 11.00-12.00 sec 15.3 MBytes 128 Mbits/sec
[ 5][TX-C] 12.00-13.00 sec 70.0 MBytes 587 Mbits/sec 14 594 KBytes
[ 7][RX-C] 12.00-13.00 sec 16.1 MBytes 135 Mbits/sec
[ 5][TX-C] 13.00-14.00 sec 66.2 MBytes 556 Mbits/sec 0 669 KBytes
[ 7][RX-C] 13.00-14.00 sec 20.2 MBytes 170 Mbits/sec
[ 5][TX-C] 14.00-15.00 sec 72.5 MBytes 608 Mbits/sec 0 749 KBytes
[ 7][RX-C] 14.00-15.00 sec 15.9 MBytes 134 Mbits/sec
[ 5][TX-C] 15.00-16.00 sec 75.0 MBytes 629 Mbits/sec 0 822 KBytes
[ 7][RX-C] 15.00-16.00 sec 12.1 MBytes 102 Mbits/sec
[ 5][TX-C] 16.00-17.00 sec 78.8 MBytes 661 Mbits/sec 0 892 KBytes
[ 7][RX-C] 16.00-17.00 sec 9.87 MBytes 82.8 Mbits/sec
[ 5][TX-C] 17.00-18.00 sec 78.8 MBytes 661 Mbits/sec 126 690 KBytes
[ 7][RX-C] 17.00-18.00 sec 10.1 MBytes 85.0 Mbits/sec
[ 5][TX-C] 18.00-19.00 sec 72.5 MBytes 608 Mbits/sec 0 778 KBytes
[ 7][RX-C] 18.00-19.00 sec 14.2 MBytes 119 Mbits/sec
[ 5][TX-C] 19.00-20.00 sec 76.2 MBytes 640 Mbits/sec 0 840 KBytes
[ 7][RX-C] 19.00-20.00 sec 11.4 MBytes 95.5 Mbits/sec
[ 5][TX-C] 20.00-21.00 sec 77.5 MBytes 650 Mbits/sec 55 636 KBytes
[ 7][RX-C] 20.00-21.00 sec 11.1 MBytes 93.2 Mbits/sec
[ 5][TX-C] 21.00-22.00 sec 70.0 MBytes 587 Mbits/sec 0 714 KBytes
[ 7][RX-C] 21.00-22.00 sec 16.3 MBytes 137 Mbits/sec
[ 5][TX-C] 22.00-23.00 sec 72.5 MBytes 608 Mbits/sec 0 790 KBytes
[ 7][RX-C] 22.00-23.00 sec 13.5 MBytes 113 Mbits/sec
[ 5][TX-C] 23.00-24.00 sec 77.5 MBytes 650 Mbits/sec 0 863 KBytes
[ 7][RX-C] 23.00-24.00 sec 10.7 MBytes 89.4 Mbits/sec
[ 5][TX-C] 24.00-25.00 sec 81.2 MBytes 682 Mbits/sec 0 932 KBytes
[ 7][RX-C] 24.00-25.00 sec 8.69 MBytes 72.9 Mbits/sec
[ 5][TX-C] 25.00-26.00 sec 82.5 MBytes 692 Mbits/sec 125 710 KBytes
[ 7][RX-C] 25.00-26.00 sec 7.30 MBytes 61.3 Mbits/sec
[ 5][TX-C] 26.00-27.00 sec 73.8 MBytes 619 Mbits/sec 0 803 KBytes
[ 7][RX-C] 26.00-27.00 sec 12.4 MBytes 104 Mbits/sec
[ 5][TX-C] 27.00-28.00 sec 78.8 MBytes 661 Mbits/sec 0 871 KBytes
[ 7][RX-C] 27.00-28.00 sec 10.3 MBytes 86.2 Mbits/sec
[ 5][TX-C] 28.00-29.00 sec 81.2 MBytes 682 Mbits/sec 0 922 KBytes
[ 7][RX-C] 28.00-29.00 sec 8.68 MBytes 72.8 Mbits/sec
[ 5][TX-C] 29.00-30.00 sec 82.5 MBytes 692 Mbits/sec 133 689 KBytes
[ 7][RX-C] 29.00-30.00 sec 7.12 MBytes 59.7 Mbits/sec
[ 5][TX-C] 30.00-31.00 sec 75.0 MBytes 629 Mbits/sec 0 766 KBytes
[ 7][RX-C] 30.00-31.00 sec 12.3 MBytes 103 Mbits/sec
[ 5][TX-C] 31.00-32.00 sec 76.2 MBytes 640 Mbits/sec 0 841 KBytes
[ 7][RX-C] 31.00-32.00 sec 11.3 MBytes 95.1 Mbits/sec
[ 5][TX-C] 32.00-33.00 sec 80.0 MBytes 671 Mbits/sec 0 912 KBytes
[ 7][RX-C] 32.00-33.00 sec 9.20 MBytes 77.2 Mbits/sec
[ 5][TX-C] 33.00-34.00 sec 83.8 MBytes 703 Mbits/sec 0 979 KBytes
[ 7][RX-C] 33.00-34.00 sec 7.26 MBytes 60.9 Mbits/sec
[ 5][TX-C] 34.00-35.00 sec 73.8 MBytes 619 Mbits/sec 159 778 KBytes
[ 7][RX-C] 34.00-35.00 sec 12.8 MBytes 108 Mbits/sec
[ 5][TX-C] 35.00-36.00 sec 77.5 MBytes 650 Mbits/sec 0 854 KBytes
[ 7][RX-C] 35.00-36.00 sec 11.1 MBytes 93.2 Mbits/sec
[ 5][TX-C] 36.00-37.00 sec 80.0 MBytes 671 Mbits/sec 0 906 KBytes
[ 7][RX-C] 36.00-37.00 sec 8.95 MBytes 75.0 Mbits/sec
[ 5][TX-C] 37.00-38.00 sec 72.5 MBytes 608 Mbits/sec 30 710 KBytes
[ 7][RX-C] 37.00-38.00 sec 14.8 MBytes 124 Mbits/sec
[ 5][TX-C] 38.00-39.00 sec 73.8 MBytes 619 Mbits/sec 0 785 KBytes
[ 7][RX-C] 38.00-39.00 sec 13.8 MBytes 116 Mbits/sec
[ 5][TX-C] 39.00-40.00 sec 76.2 MBytes 640 Mbits/sec 0 857 KBytes
[ 7][RX-C] 39.00-40.00 sec 11.1 MBytes 93.2 Mbits/sec
[ 5][TX-C] 40.00-41.00 sec 81.2 MBytes 682 Mbits/sec 0 925 KBytes
[ 7][RX-C] 40.00-41.00 sec 8.67 MBytes 72.7 Mbits/sec
[ 5][TX-C] 41.00-42.00 sec 73.8 MBytes 619 Mbits/sec 86 732 KBytes
[ 7][RX-C] 41.00-42.00 sec 14.0 MBytes 118 Mbits/sec
[ 5][TX-C] 42.00-43.00 sec 75.0 MBytes 629 Mbits/sec 0 810 KBytes
[ 7][RX-C] 42.00-43.00 sec 12.8 MBytes 107 Mbits/sec
[ 5][TX-C] 43.00-44.00 sec 77.5 MBytes 650 Mbits/sec 0 867 KBytes
[ 7][RX-C] 43.00-44.00 sec 10.3 MBytes 86.2 Mbits/sec
[ 5][TX-C] 44.00-45.00 sec 81.2 MBytes 682 Mbits/sec 0 935 KBytes
[ 7][RX-C] 44.00-45.00 sec 8.55 MBytes 71.7 Mbits/sec
[ 5][TX-C] 45.00-46.00 sec 75.0 MBytes 629 Mbits/sec 93 738 KBytes
[ 7][RX-C] 45.00-46.00 sec 13.1 MBytes 110 Mbits/sec
[ 5][TX-C] 46.00-47.00 sec 73.8 MBytes 619 Mbits/sec 0 813 KBytes
[ 7][RX-C] 46.00-47.00 sec 12.8 MBytes 107 Mbits/sec
[ 5][TX-C] 47.00-48.00 sec 71.2 MBytes 598 Mbits/sec 21 638 KBytes
[ 7][RX-C] 47.00-48.00 sec 15.7 MBytes 132 Mbits/sec
[ 5][TX-C] 48.00-49.00 sec 68.8 MBytes 577 Mbits/sec 0 717 KBytes
[ 7][RX-C] 48.00-49.00 sec 17.2 MBytes 144 Mbits/sec
[ 5][TX-C] 49.00-50.00 sec 73.8 MBytes 619 Mbits/sec 0 793 KBytes
[ 7][RX-C] 49.00-50.00 sec 13.4 MBytes 112 Mbits/sec
[ 5][TX-C] 50.00-51.00 sec 77.5 MBytes 650 Mbits/sec 0 865 KBytes
[ 7][RX-C] 50.00-51.00 sec 10.4 MBytes 87.3 Mbits/sec
[ 5][TX-C] 51.00-52.00 sec 81.2 MBytes 682 Mbits/sec 0 935 KBytes
[ 7][RX-C] 51.00-52.00 sec 8.66 MBytes 72.6 Mbits/sec
[ 5][TX-C] 52.00-53.00 sec 82.5 MBytes 692 Mbits/sec 0 998 KBytes
[ 7][RX-C] 52.00-53.00 sec 6.50 MBytes 54.5 Mbits/sec
[ 5][TX-C] 53.00-54.00 sec 81.2 MBytes 682 Mbits/sec 129 775 KBytes
[ 7][RX-C] 53.00-54.00 sec 7.76 MBytes 65.1 Mbits/sec
[ 5][TX-C] 54.00-55.00 sec 77.5 MBytes 650 Mbits/sec 0 864 KBytes
[ 7][RX-C] 54.00-55.00 sec 10.6 MBytes 88.5 Mbits/sec
[ 5][TX-C] 55.00-56.00 sec 75.0 MBytes 629 Mbits/sec 121 665 KBytes
[ 7][RX-C] 55.00-56.00 sec 13.5 MBytes 113 Mbits/sec
[ 5][TX-C] 56.00-57.00 sec 70.0 MBytes 587 Mbits/sec 0 741 KBytes
[ 7][RX-C] 56.00-57.00 sec 15.7 MBytes 132 Mbits/sec
[ 5][TX-C] 57.00-58.00 sec 75.0 MBytes 629 Mbits/sec 0 816 KBytes
[ 7][RX-C] 57.00-58.00 sec 12.5 MBytes 105 Mbits/sec
[ 5][TX-C] 58.00-59.00 sec 73.8 MBytes 619 Mbits/sec 23 641 KBytes
[ 7][RX-C] 58.00-59.00 sec 13.3 MBytes 112 Mbits/sec
[ 5][TX-C] 59.00-60.00 sec 70.0 MBytes 587 Mbits/sec 0 718 KBytes
[ 7][RX-C] 59.00-60.00 sec 17.0 MBytes 143 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-60.00 sec 4.47 GBytes 640 Mbits/sec 2409 sender
[ 5][TX-C] 0.00-60.00 sec 4.47 GBytes 639 Mbits/sec receiver
[ 7][RX-C] 0.00-60.00 sec 707 MBytes 98.9 Mbits/sec 444 sender
[ 7][RX-C] 0.00-60.00 sec 705 MBytes 98.5 Mbits/sec receiver
iperf Done.
tc -s qdisc before:
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 23414 bytes 144 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 23414 bytes 144 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev eth0.2 root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
Sent 1454 bytes 12 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 120: dev eth0.2 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 1454 bytes 12 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 164 drop_overlimit 0 new_flow_count 4 ecn_mark 0
new_flows_len 0 old_flows_len 1
qdisc fq_codel 130: dev eth0.2 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 110: dev eth0.2 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
Sent 264 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev ifb4eth0.2 root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 direct_qlen 32
Sent 292 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4eth0.2 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 292 bytes 5 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 70 drop_overlimit 0 new_flow_count 2 ecn_mark 0
new_flows_len 1 old_flows_len 0
tc -s qdisc after:
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 6784101267 bytes 4820857 pkt (dropped 0, overlimits 0 requeues 289352)
backlog 0b 0p requeues 289352
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 6784101267 bytes 4820857 pkt (dropped 0, overlimits 0 requeues 289352)
backlog 0b 0p requeues 289352
maxpacket 1514 drop_overlimit 0 new_flow_count 305497 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev eth0.2 root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
Sent 5917911859 bytes 3962089 pkt (dropped 0, overlimits 3669 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 120: dev eth0.2 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 5917911859 bytes 3962089 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 12112 drop_overlimit 0 new_flow_count 54217 ecn_mark 0
new_flows_len 0 old_flows_len 3
qdisc fq_codel 130: dev eth0.2 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 110: dev eth0.2 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
Sent 834903770 bytes 856289 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev ifb4eth0.2 root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 direct_qlen 32
Sent 864891508 bytes 855416 pkt (dropped 532, overlimits 1212 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4eth0.2 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 864891508 bytes 855416 pkt (dropped 532, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 12112 drop_overlimit 0 new_flow_count 17101 ecn_mark 0
new_flows_len 1 old_flows_len 2
Snapshot of top -d during the run:
Mem: 38104K used, 442348K free, 80K shrd, 2608K buff, 8116K cached
CPU: 0% usr 0% sys 0% nic 21% idle 0% io 0% irq 77% sirq
Load average: 0.55 0.19 0.07 1/61 3661
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
14 2 root RW 0 0% 68% [ksoftirqd/1]
7 2 root RW 0 0% 7% [ksoftirqd/0]
3661 3646 root R 1096 0% 1% top -d 1
50 2 root IW 0 0% 1% [kworker/1:1]
3645 788 root S 908 0% 1% /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p 22 -K 300 -T
692 1 root S 1828 0% 0% /sbin/rpcd -s /var/run/ubus.sock -t 30
845 1 root S 1556 0% 0% /sbin/netifd
1 0 root S 1360 0% 0% /sbin/procd
884 1 root S 1248 0% 0% /usr/sbin/odhcpd
3623 1 dnsmasq S 1176 0% 0% /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf.cfg01411c -k -x /var/r
2183 1 root S 1112 0% 0% /usr/sbin/uhttpd -f -h /www -r OpenWrt -x /cgi-bin -t 60 -T 30 -k
3646 3645 root S 1088 0% 0% -ash
2880 1 root S< 1080 0% 0% /usr/sbin/ntpd -n -N -S /usr/sbin/ntpd-hotplug -p 0.openwrt.pool.
2113 845 root S 1080 0% 0% udhcpc -p /var/run/udhcpc-eth0.2.pid -s /lib/netifd/dhcp.script -
661 1 root S 1032 0% 0% /sbin/logd -S 64
164 1 root S 1008 0% 0% /sbin/ubusd
788 1 root S 844 0% 0% /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -p 22 -K 300 -T
2116 845 root S 840 0% 0% odhcp6c -s /lib/netifd/dhcpv6.script -P0 -t120 eth0.2
182 1 root S 816 0% 0% /sbin/urngd
165 1 root S 704 0% 0% /sbin/askfirst /usr/libexec/login.sh
5 2 root IW 0 0% 0% [kworker/u4:0]
58 2 root IW 0 0% 0% [kworker/0:2]
17 2 root IW 0 0% 0% [kworker/0:1]
350 2 root IW 0 0% 0% [kworker/u4:2]
8 2 root IW 0 0% 0% [rcu_sched]
13 2 root SW 0 0% 0% [migration/1]
10 2 root SW 0 0% 0% [migration/0]
59 2 root SW 0 0% 0% [ubi_bgt0d]
57 2 root IW< 0 0% 0% [dsa_ordered]
SQM config:
config queue 'eth1'
option enabled '1'
option qdisc 'fq_codel'
option script 'simple.qos'
option qdisc_advanced '0'
option linklayer 'none'
option interface 'eth0.2'
option download '800000'
option upload '800000'
option debug_logging '0'
option verbosity '5'
dmesg output collected after the run:
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.195 (builder@buildhost) (gcc version 7.5.0 (OpenWrt GCC 7.5.0 r11208-ce6496d796)) #0 SMP Sun Sep 6 16:19:39 2020
[ 0.000000] CPU: ARMv7 Processor [512f04d0] revision 0 (ARMv7), cr=10c5787d
[ 0.000000] CPU: div instructions available: patching division code
[ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[ 0.000000] OF: fdt: Machine model: Linksys EA8500 WiFi Router
[ 0.000000] Memory policy: Data cache writealloc
[ 0.000000] On node 0 totalpages: 122880
[ 0.000000] free_area_init_node: node 0, pgdat c0a27880, node_mem_map ddc39000
[ 0.000000] Normal zone: 960 pages used for memmap
[ 0.000000] Normal zone: 0 pages reserved
[ 0.000000] Normal zone: 122880 pages, LIFO batch:31
[ 0.000000] random: get_random_bytes called from 0xc09008e0 with crng_init=0
[ 0.000000] percpu: Embedded 15 pages/cpu s29388 r8192 d23860 u61440
[ 0.000000] pcpu-alloc: s29388 r8192 d23860 u61440 alloc=15*4096
[ 0.000000] pcpu-alloc: [0] 0 [0] 1
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 121920
[ 0.000000] Kernel command line: console=ttyMSM0,115200n8 ubi.mtd=16
[ 0.000000] Bootloader command line (ignored): console=ttyHSL1,115200n8 init=/sbin/init rootfstype=squashfs root=31:16
[ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
[ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
[ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
[ 0.000000] Memory: 479428K/491520K available (4978K kernel code, 160K rwdata, 756K rodata, 1024K init, 235K bss, 12092K reserved, 0K cma-reserved, 0K highmem)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB)
[ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB)
[ 0.000000] vmalloc : 0xde800000 - 0xff800000 ( 528 MB)
[ 0.000000] lowmem : 0xc0000000 - 0xde000000 ( 480 MB)
[ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB)
[ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB)
[ 0.000000] .text : 0xc0208000 - 0xc07dc880 (5971 kB)
[ 0.000000] .init : 0xc0900000 - 0xc0a00000 (1024 kB)
[ 0.000000] .data : 0xc0a00000 - 0xc0a28180 ( 161 kB)
[ 0.000000] .bss : 0xc0a2a000 - 0xc0a64e18 ( 236 kB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[ 0.000000] clocksource: dg_timer: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 305801671480 ns
[ 0.000008] sched_clock: 32 bits at 6MHz, resolution 160ns, wraps every 343597383600ns
[ 0.000023] Switching to timer-based delay loop, resolution 160ns
[ 0.000234] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.50 BogoMIPS (lpj=62500)
[ 0.000261] pid_max: default: 32768 minimum: 301
[ 0.000390] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.000409] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[ 0.000977] CPU: Testing write buffer coherency: ok
[ 0.001751] Setting up static identity map for 0x42300000 - 0x42300060
[ 0.001915] Hierarchical SRCU implementation.
[ 0.002653] smp: Bringing up secondary CPUs ...
[ 0.004443] smp: Brought up 1 node, 2 CPUs
[ 0.004461] SMP: Total of 2 processors activated (25.00 BogoMIPS).
[ 0.004472] CPU: All CPU(s) started in SVC mode.
[ 0.014085] VFP support v0.3: implementor 51 architecture 64 part 4d variant 2 rev 0
[ 0.014236] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[ 0.014262] futex hash table entries: 512 (order: 3, 32768 bytes)
[ 0.014374] pinctrl core: initialized pinctrl subsystem
[ 0.015347] NET: Registered protocol family 16
[ 0.015623] DMA: preallocated 256 KiB pool for atomic coherent allocations
[ 0.016948] cpuidle: using governor ladder
[ 0.017012] cpuidle: using governor menu
[ 0.039868] msm_bus_fabric_init_driver
[ 0.041301] usbcore: registered new interface driver usbfs
[ 0.041482] usbcore: registered new interface driver hub
[ 0.041563] usbcore: registered new device driver usb
[ 0.041619] pps_core: LinuxPPS API ver. 1 registered
[ 0.041631] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.041664] PTP clock support registered
[ 0.043382] clocksource: Switched to clocksource dg_timer
[ 0.046535] NET: Registered protocol family 2
[ 0.047135] TCP established hash table entries: 4096 (order: 2, 16384 bytes)
[ 0.047174] TCP bind hash table entries: 4096 (order: 3, 32768 bytes)
[ 0.047228] TCP: Hash tables configured (established 4096 bind 4096)
[ 0.047320] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 0.047346] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 0.047520] NET: Registered protocol family 1
[ 0.047567] PCI: CLS 0 bytes, default 64
[ 0.048740] No memory allocated for crashlog
[ 0.049035] workingset: timestamp_bits=30 max_order=17 bucket_order=0
[ 0.053867] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 0.053885] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[ 0.060430] io scheduler noop registered
[ 0.060449] io scheduler deadline registered (default)
[ 0.062076] qcom-pcie 1b500000.pci: 1b500000.pci supply vdda not found, using dummy regulator
[ 0.062184] qcom-pcie 1b500000.pci: 1b500000.pci supply vdda_phy not found, using dummy regulator
[ 0.062279] qcom-pcie 1b500000.pci: 1b500000.pci supply vdda_refclk not found, using dummy regulator
[ 0.062984] OF: PCI: host bridge /soc/pci@1b500000 ranges:
[ 0.063026] OF: PCI: IO 0x0fe00000..0x0fefffff -> 0x0fe00000
[ 0.063050] OF: PCI: MEM 0x08000000..0x0fdfffff -> 0x08000000
[ 0.171213] qcom-pcie 1b500000.pci: link up
[ 0.171382] qcom-pcie 1b500000.pci: PCI host bridge to bus 0000:00
[ 0.171407] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.171426] pci_bus 0000:00: root bus resource [io 0x0000-0xfffff] (bus address [0xfe00000-0xfefffff])
[ 0.171442] pci_bus 0000:00: root bus resource [mem 0x08000000-0x0fdfffff]
[ 0.171459] pci_bus 0000:00: scanning bus
[ 0.171511] pci 0000:00:00.0: [17cb:0101] type 01 class 0x060400
[ 0.171601] pci 0000:00:00.0: calling 0xc030d3c0
[ 0.171676] pci 0000:00:00.0: supports D1
[ 0.171686] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[ 0.171702] pci 0000:00:00.0: PME# disabled
[ 0.171958] pci_bus 0000:00: fixups for bus
[ 0.171982] PCI: bus0: Fast back to back transfers disabled
[ 0.172003] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[ 0.172130] pci_bus 0000:01: scanning bus
[ 0.172318] pci 0000:01:00.0: [168c:0040] type 00 class 0x028000
[ 0.172696] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[ 0.173196] pci 0000:01:00.0: calling 0xc030d3c0
[ 0.174000] pci 0000:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.174043] pci 0000:01:00.0: PME# disabled
[ 0.174376] pci_bus 0000:01: fixups for bus
[ 0.174452] PCI: bus1: Fast back to back transfers disabled
[ 0.174467] pci_bus 0000:01: bus scan returning with max=01
[ 0.174485] pci 0000:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[ 0.174503] pci_bus 0000:00: bus scan returning with max=ff
[ 0.174550] pci 0000:00:00.0: BAR 8: assigned [mem 0x08000000-0x081fffff]
[ 0.174576] pci 0000:01:00.0: BAR 0: assigned [mem 0x08000000-0x081fffff 64bit]
[ 0.174711] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[ 0.174734] pci 0000:00:00.0: bridge window [mem 0x08000000-0x081fffff]
[ 0.174765] pci 0000:00:00.0: calling 0xc055ff28
[ 0.174953] pcieport 0000:00:00.0: assign IRQ: got 68
[ 0.175261] pcieport 0000:00:00.0: AER enabled with IRQ 36
[ 0.175377] pci 0000:01:00.0: calling 0xc055ff28
[ 0.175793] qcom-pcie 1b700000.pci: 1b700000.pci supply vdda not found, using dummy regulator
[ 0.175907] qcom-pcie 1b700000.pci: 1b700000.pci supply vdda_phy not found, using dummy regulator
[ 0.176000] qcom-pcie 1b700000.pci: 1b700000.pci supply vdda_refclk not found, using dummy regulator
[ 0.176688] OF: PCI: host bridge /soc/pci@1b700000 ranges:
[ 0.176727] OF: PCI: IO 0x31e00000..0x31efffff -> 0x31e00000
[ 0.176751] OF: PCI: MEM 0x2e000000..0x31dfffff -> 0x2e000000
[ 0.286609] qcom-pcie 1b700000.pci: link up
[ 0.286756] qcom-pcie 1b700000.pci: PCI host bridge to bus 0001:00
[ 0.286776] pci_bus 0001:00: root bus resource [bus 00-ff]
[ 0.286792] pci_bus 0001:00: root bus resource [mem 0x2e000000-0x31dfffff]
[ 0.286806] pci_bus 0001:00: scanning bus
[ 0.286846] pci 0001:00:00.0: [17cb:0101] type 01 class 0x060400
[ 0.286917] pci 0001:00:00.0: calling 0xc030d3c0
[ 0.286977] pci 0001:00:00.0: supports D1
[ 0.286987] pci 0001:00:00.0: PME# supported from D0 D1 D3hot
[ 0.287001] pci 0001:00:00.0: PME# disabled
[ 0.287226] pci_bus 0001:00: fixups for bus
[ 0.287246] PCI: bus0: Fast back to back transfers disabled
[ 0.287264] pci 0001:00:00.0: scanning [bus 01-ff] behind bridge, pass 0
[ 0.287376] pci_bus 0001:01: scanning bus
[ 0.287552] pci 0001:01:00.0: [168c:0040] type 00 class 0x028000
[ 0.287896] pci 0001:01:00.0: reg 0x10: [mem 0x00000000-0x001fffff 64bit]
[ 0.288360] pci 0001:01:00.0: calling 0xc030d3c0
[ 0.289002] pci 0001:01:00.0: PME# supported from D0 D3hot D3cold
[ 0.289043] pci 0001:01:00.0: PME# disabled
[ 0.289364] pci_bus 0001:01: fixups for bus
[ 0.289435] PCI: bus1: Fast back to back transfers disabled
[ 0.289450] pci_bus 0001:01: bus scan returning with max=01
[ 0.289465] pci 0001:00:00.0: scanning [bus 01-ff] behind bridge, pass 1
[ 0.289483] pci_bus 0001:00: bus scan returning with max=ff
[ 0.289519] pci 0001:00:00.0: BAR 8: assigned [mem 0x2e000000-0x2e1fffff]
[ 0.289541] pci 0001:01:00.0: BAR 0: assigned [mem 0x2e000000-0x2e1fffff 64bit]
[ 0.289665] pci 0001:00:00.0: PCI bridge to [bus 01-ff]
[ 0.289685] pci 0001:00:00.0: bridge window [mem 0x2e000000-0x2e1fffff]
[ 0.289711] pci 0001:00:00.0: calling 0xc055ff28
[ 0.289884] pcieport 0001:00:00.0: assign IRQ: got 101
[ 0.290150] pcieport 0001:00:00.0: AER enabled with IRQ 69
[ 0.290257] pci 0001:01:00.0: calling 0xc055ff28
[ 0.290680] qcom-pcie 1b900000.pci: 1b900000.pci supply vdda not found, using dummy regulator
[ 0.290783] qcom-pcie 1b900000.pci: 1b900000.pci supply vdda_phy not found, using dummy regulator
[ 0.290888] qcom-pcie 1b900000.pci: 1b900000.pci supply vdda_refclk not found, using dummy regulator
[ 0.291565] OF: PCI: host bridge /soc/pci@1b900000 ranges:
[ 0.291602] OF: PCI: IO 0x35e00000..0x35efffff -> 0x35e00000
[ 0.291624] OF: PCI: MEM 0x32000000..0x35dfffff -> 0x32000000
[ 1.325435] qcom-pcie 1b900000.pci: phy link never came up
[ 1.327423] qcom-pcie 1b900000.pci: cannot initialize host
[ 1.328007] qcom-pcie: probe of 1b900000.pci failed with error -110
[ 1.330216] L2 @ QSB rate. Forcing new rate.
[ 1.330432] L2 @ 384000 KHz
[ 1.330604] CPU0 @ 800000 KHz
[ 1.330617] CPU1 @ QSB rate. Forcing new rate.
[ 1.330742] CPU1 @ 384000 KHz
[ 1.334735] gsbi 16300000.gsbi: GSBI port protocol: 6 crci: 0
[ 1.336665] Serial: 8250/16550 driver, 16 ports, IRQ sharing enabled
[ 1.339176] msm_serial 16340000.serial: msm_serial: detected port #0
[ 1.339273] msm_serial 16340000.serial: uartclk = 7372800
[ 1.339356] 16340000.serial: ttyMSM0 at MMIO 0x16340000 (irq = 134, base_baud = 460800) is a MSM
[ 1.339396] msm_serial: console setup on port #0
[ 2.127097] console [ttyMSM0] enabled
[ 2.131833] msm_serial: driver initialized
[ 2.140298] loop: module loaded
[ 2.142335] nand: device found, Manufacturer ID: 0x01, Chip ID: 0xa1
[ 2.142371] nand: AMD/Spansion S34MS01G2
[ 2.148949] nand: 128 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
[ 2.152940] 18 fixed-partitions partitions found on MTD device qcom_nand.0
[ 2.160224] Creating 18 MTD partitions on "qcom_nand.0":
[ 2.167076] 0x000000000000-0x000000040000 : "SBL1"
[ 2.173796] 0x000000040000-0x000000180000 : "MIBIB"
[ 2.179992] 0x000000180000-0x0000002c0000 : "SBL2"
[ 2.184788] 0x0000002c0000-0x000000540000 : "SBL3"
[ 2.186916] random: fast init done
[ 2.196415] 0x000000540000-0x000000660000 : "DDRCONFIG"
[ 2.199137] 0x000000660000-0x000000780000 : "SSD"
[ 2.203122] 0x000000780000-0x000000a00000 : "TZ"
[ 2.210477] 0x000000a00000-0x000000c80000 : "RPM"
[ 2.215637] 0x000000c80000-0x000000dc0000 : "art"
[ 2.218545] 0x000000dc0000-0x000000ec0000 : "APPSBL"
[ 2.221806] 0x000000ec0000-0x000000f00000 : "u_env"
[ 2.225633] 0x000000f00000-0x000000f40000 : "s_env"
[ 2.230143] 0x000000f40000-0x000000f80000 : "devinfo"
[ 2.235043] 0x000000f80000-0x000003780000 : "kernel1"
[ 2.309913] 0x000001280000-0x000003780000 : "rootfs1"
[ 2.375700] 0x000003780000-0x000005f80000 : "kernel2"
[ 2.446733] 0x000003a80000-0x000005f80000 : "rootfs2"
[ 2.507207] random: crng init done
[ 2.512560] 0x000005f80000-0x000008000000 : "syscfg"
[ 2.571787] libphy: GPIO Bitbanged MDIO: probed
[ 2.592993] switch0: Atheros AR8337 rev. 2 switch registered on gpio-0
[ 3.436137] ar8327: qca,phy-rgmii-en is not specified
[ 3.436523] libphy: Fixed MDIO Bus: probed
[ 3.441680] ipq806x-gmac-dwmac 37200000.ethernet: PTP uses main clock
[ 3.444573] stmmac - user ID: 0x10, Synopsys ID: 0x37
[ 3.450677] ipq806x-gmac-dwmac 37200000.ethernet: Ring mode enabled
[ 3.455793] ipq806x-gmac-dwmac 37200000.ethernet: DMA HW capability register supported
[ 3.461792] ipq806x-gmac-dwmac 37200000.ethernet: Enhanced/Alternate descriptors
[ 3.469837] ipq806x-gmac-dwmac 37200000.ethernet: Enabled extended descriptors
[ 3.477400] ipq806x-gmac-dwmac 37200000.ethernet: RX Checksum Offload Engine supported
[ 3.484427] ipq806x-gmac-dwmac 37200000.ethernet: COE Type 2
[ 3.492253] ipq806x-gmac-dwmac 37200000.ethernet: TX Checksum insertion supported
[ 3.498143] ipq806x-gmac-dwmac 37200000.ethernet: Wake-Up On Lan supported
[ 3.505527] ipq806x-gmac-dwmac 37200000.ethernet (unnamed net_device) (uninitialized): device MAC address aa:aa:aa:bb:bb:bb
[ 3.512233] ipq806x-gmac-dwmac 37200000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[ 3.524960] ipq806x-gmac-dwmac 37400000.ethernet: PTP uses main clock
[ 3.532117] stmmac - user ID: 0x10, Synopsys ID: 0x37
[ 3.538361] ipq806x-gmac-dwmac 37400000.ethernet: Ring mode enabled
[ 3.543298] ipq806x-gmac-dwmac 37400000.ethernet: DMA HW capability register supported
[ 3.549437] ipq806x-gmac-dwmac 37400000.ethernet: Enhanced/Alternate descriptors
[ 3.557435] ipq806x-gmac-dwmac 37400000.ethernet: Enabled extended descriptors
[ 3.564994] ipq806x-gmac-dwmac 37400000.ethernet: RX Checksum Offload Engine supported
[ 3.571947] ipq806x-gmac-dwmac 37400000.ethernet: COE Type 2
[ 3.579926] ipq806x-gmac-dwmac 37400000.ethernet: TX Checksum insertion supported
[ 3.585745] ipq806x-gmac-dwmac 37400000.ethernet: Wake-Up On Lan supported
[ 3.593048] ipq806x-gmac-dwmac 37400000.ethernet (unnamed net_device) (uninitialized): device MAC address bb:bb:bb:cc:cc:cc
[ 3.599892] ipq806x-gmac-dwmac 37400000.ethernet: Enable RX Mitigation via HW Watchdog Timer
[ 3.611575] i2c /dev entries driver
[ 3.620425] Calibration not found.
[ 3.623916] Speed bin: 0
[ 3.626189] PVS bin: 1
[ 3.630174] cpuidle: enable-method property 'qcom,kpss-acc-v1' found operations
[ 3.631124] cpuidle: enable-method property 'qcom,kpss-acc-v1' found operations
[ 3.638904] sdhci: Secure Digital Host Controller Interface driver
[ 3.645622] sdhci: Copyright(c) Pierre Ossman
[ 3.651800] sdhci-pltfm: SDHCI platform and OF driver helper
[ 3.657637] NET: Registered protocol family 10
[ 3.663363] Segment Routing with IPv6
[ 3.666397] NET: Registered protocol family 17
[ 3.669998] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[ 3.675006] 8021q: 802.1Q VLAN Support v1.8
[ 3.687564] Registering SWP/SWPB emulation handler
[ 3.704033] qcom_rpm 108000.rpm: RPM firmware 3.0.16777342
[ 3.715671] s1a: supplied by regulator-dummy
[ 3.715763] s1a: Bringing 0uV into 1050000-1050000uV
[ 3.719265] s1b: supplied by regulator-dummy
[ 3.724107] s1b: Bringing 0uV into 1050000-1050000uV
[ 3.728457] s2a: supplied by regulator-dummy
[ 3.733204] s2a: Bringing 0uV into 800000-800000uV
[ 3.737740] s2b: supplied by regulator-dummy
[ 3.742073] s2b: Bringing 0uV into 800000-800000uV
[ 3.750428] ubi0: attaching mtd16
[ 4.035651] ubi0: scanning is finished
[ 4.045341] ubi0: attached mtd16 (name "rootfs2", size 37 MiB)
[ 4.045365] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[ 4.050058] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[ 4.056998] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[ 4.063826] ubi0: good PEBs: 296, bad PEBs: 0, corrupted PEBs: 0
[ 4.070543] ubi0: user volume: 2, internal volumes: 1, max. volumes count: 128
[ 4.076862] ubi0: max/mean erase counter: 3/1, WL threshold: 4096, image sequence number: 2129532265
[ 4.083884] ubi0: available PEBs: 0, total reserved PEBs: 296, PEBs reserved for bad PEB handling: 20
[ 4.093207] ubi0: background thread "ubi_bgt0d" started, PID 59
[ 4.093923] block ubiblock0_0: created from ubi0:0(rootfs)
[ 4.108032] ubiblock: device ubiblock0_0 (rootfs) set to be root filesystem
[ 4.113516] hctosys: unable to open rtc device (rtc0)
[ 4.128991] VFS: Mounted root (squashfs filesystem) readonly on device 254:0.
[ 4.131996] Freeing unused kernel memory: 1024K
[ 4.708612] init: Console is alive
[ 4.708760] init: - watchdog -
[ 5.396242] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[ 5.430283] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 5.431993] SCSI subsystem initialized
[ 5.437836] libata version 3.00 loaded.
[ 5.439764] ahci 29000000.sata: forcing port_map 0x0 -> 0x1
[ 5.439813] ahci 29000000.sata: AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl platform mode
[ 5.445000] ahci 29000000.sata: flags: ncq sntf pm led clo only pmp pio slum part ccc apst
[ 5.454147] scsi host0: ahci
[ 5.461859] ata1: SATA max UDMA/133 mmio [mem 0x29000000-0x2900017f] port 0x100 irq 26
[ 5.795867] ata1: SATA link down (SStatus 0 SControl 300)
[ 5.797591] ehci-platform: EHCI generic platform driver
[ 5.803200] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[ 5.805820] ohci-platform: OHCI generic platform driver
[ 5.941534] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[ 5.941633] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 1
[ 5.946416] xhci-hcd xhci-hcd.0.auto: hcc params 0x0228f065 hci version 0x100 quirks 0x0000000002010010
[ 5.953588] xhci-hcd xhci-hcd.0.auto: irq 138, io mem 0x11000000
[ 5.963717] hub 1-0:1.0: USB hub found
[ 5.969124] hub 1-0:1.0: 1 port detected
[ 5.972954] xhci-hcd xhci-hcd.0.auto: xHCI Host Controller
[ 5.976827] xhci-hcd xhci-hcd.0.auto: new USB bus registered, assigned bus number 2
[ 5.982039] xhci-hcd xhci-hcd.0.auto: Host supports USB 3.0 SuperSpeed
[ 5.989746] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
[ 5.996580] hub 2-0:1.0: USB hub found
[ 6.004607] hub 2-0:1.0: 1 port detected
[ 6.008319] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[ 6.012162] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 3
[ 6.017663] xhci-hcd xhci-hcd.1.auto: hcc params 0x0228f065 hci version 0x100 quirks 0x0000000002010010
[ 6.025109] xhci-hcd xhci-hcd.1.auto: irq 139, io mem 0x10000000
[ 6.034826] hub 3-0:1.0: USB hub found
[ 6.040650] hub 3-0:1.0: 1 port detected
[ 6.044463] xhci-hcd xhci-hcd.1.auto: xHCI Host Controller
[ 6.048269] xhci-hcd xhci-hcd.1.auto: new USB bus registered, assigned bus number 4
[ 6.053568] xhci-hcd xhci-hcd.1.auto: Host supports USB 3.0 SuperSpeed
[ 6.061275] usb usb4: We don't know the algorithms for LPM for this host, disabling LPM.
[ 6.068110] hub 4-0:1.0: USB hub found
[ 6.076153] hub 4-0:1.0: 1 port detected
[ 6.080785] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[ 6.085852] init: - preinit -
[ 7.972093] Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=POLL)
[ 7.974446] dwmac1000: Master AXI performs any burst length
[ 7.981021] ipq806x-gmac-dwmac 37200000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[ 7.986776] ipq806x-gmac-dwmac 37200000.ethernet eth0: registered PTP clock
[ 7.996785] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 9.033822] ipq806x-gmac-dwmac 37200000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[ 9.033920] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 11.170849] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 155
[ 11.221601] UBIFS (ubi0:1): recovery needed
[ 11.359095] UBIFS (ubi0:1): recovery completed
[ 11.359272] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
[ 11.362448] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
[ 11.370456] UBIFS (ubi0:1): FS size: 30093312 bytes (28 MiB, 237 LEBs), journal size 1523712 bytes (1 MiB, 12 LEBs)
[ 11.380363] UBIFS (ubi0:1): reserved for root: 1421380 bytes (1388 KiB)
[ 11.390564] UBIFS (ubi0:1): media format: w4/r0 (latest is w5/r0), UUID C63A4B12-97D5-4A95-988D-129D5AE8F3EB, small LPT model
[ 11.415875] mount_root: switching to ubifs overlay
[ 11.435875] urandom-seed: Seeding with /etc/urandom.seed
[ 11.545958] procd: - early -
[ 11.546116] procd: - watchdog -
[ 12.262007] procd: - watchdog -
[ 12.262529] procd: - ubus -
[ 12.355057] procd: - init -
[ 12.640277] urngd: v1.0.2 started.
[ 12.734852] kmodloader: loading kernel modules from /etc/modules.d/*
[ 12.756673] ip6_tables: (C) 2000-2006 Netfilter Core Team
[ 12.760000] Mirror/redirect action on
[ 12.765340] u32 classifier
[ 12.765363] input device check on
[ 12.767292] Actions configured
[ 12.779374] nf_conntrack version 0.5.0 (8192 buckets, 32768 max)
[ 12.787155] Loading modules backported from Linux version v4.19.137-0-gc076c79e03c6
[ 12.787179] Backport generated by backports.git v4.19.137-1-0-g60c3a249
[ 12.794599] ip_tables: (C) 2000-2006 Netfilter Core Team
[ 12.827834] xt_time: kernel timezone is -0000
[ 12.846087] PPP generic driver version 2.4.2
[ 12.846617] NET: Registered protocol family 24
[ 12.856408] ath10k_pci 0000:01:00.0: assign IRQ: got 68
[ 12.856432] ath10k 4.19 driver, optimized for CT firmware, probing pci device: 0x40.
[ 12.856849] ath10k_pci 0000:01:00.0: enabling device (0140 -> 0142)
[ 12.863307] ath10k_pci 0000:01:00.0: enabling bus mastering
[ 12.863936] ath10k_pci 0000:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[ 13.038108] ath10k_pci 0000:01:00.0: Direct firmware load for ath10k/fwcfg-pci-0000:01:00.0.txt failed with error -2
[ 13.038160] ath10k_pci 0000:01:00.0: Falling back to user helper
[ 13.075746] firmware ath10k!fwcfg-pci-0000:01:00.0.txt: firmware_loading_store: map pages failed
[ 13.079119] ath10k_pci 0000:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/ct-firmware-5.bin failed with error -2
[ 13.083620] ath10k_pci 0000:01:00.0: Falling back to user helper
[ 13.117610] firmware ath10k!QCA99X0!hw2.0!ct-firmware-5.bin: firmware_loading_store: map pages failed
[ 13.117858] ath10k_pci 0000:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/ct-firmware-2.bin failed with error -2
[ 13.125910] ath10k_pci 0000:01:00.0: Falling back to user helper
[ 13.154257] firmware ath10k!QCA99X0!hw2.0!ct-firmware-2.bin: firmware_loading_store: map pages failed
[ 13.154437] ath10k_pci 0000:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/firmware-6.bin failed with error -2
[ 13.162463] ath10k_pci 0000:01:00.0: Falling back to user helper
[ 13.191103] firmware ath10k!QCA99X0!hw2.0!firmware-6.bin: firmware_loading_store: map pages failed
[ 13.504705] ath10k_pci 0000:01:00.0: qca99x0 hw2.0 target 0x01000000 chip_id 0x003801ff sub 168c:0002
[ 13.504756] ath10k_pci 0000:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[ 13.516978] ath10k_pci 0000:01:00.0: firmware ver 10.4b-ct-9980-fW-13-10af6a005 api 5 features mfp,peer-flow-ctrl,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,set-special-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 4a0fa880
[ 13.586224] ath10k_pci 0000:01:00.0: board_file api 2 bmi_id 1:2 crc32 08fa09f2
[ 14.776839] ath10k_pci 0000:01:00.0: 10.4 wmi init: vdevs: 16 peers: 48 tid: 96
[ 14.776872] ath10k_pci 0000:01:00.0: msdu-desc: 2500 skid: 32
[ 14.854795] ath10k_pci 0000:01:00.0: wmi print 'P 48/48 V 16 K 144 PH 176 T 186 msdu-desc: 2500 sw-crypt: 0 ct-sta: 0'
[ 14.855585] ath10k_pci 0000:01:00.0: wmi print 'free: 31080 iram: 23092 sram: 9596'
[ 15.052773] ath10k_pci 0000:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 32 raw 0 hwcrypto 1
[ 15.202552] ath: EEPROM regdomain: 0x0
[ 15.202566] ath: EEPROM indicates default country code should be used
[ 15.202576] ath: doing EEPROM country->regdmn map search
[ 15.202591] ath: country maps to regdmn code: 0x3a
[ 15.202602] ath: Country alpha2 being used: US
[ 15.202610] ath: Regpair used: 0x3a
[ 15.207102] ath10k_pci 0001:01:00.0: assign IRQ: got 101
[ 15.207136] ath10k 4.19 driver, optimized for CT firmware, probing pci device: 0x40.
[ 15.207988] ath10k_pci 0001:01:00.0: enabling device (0140 -> 0142)
[ 15.214147] ath10k_pci 0001:01:00.0: enabling bus mastering
[ 15.214723] ath10k_pci 0001:01:00.0: pci irq msi oper_irq_mode 2 irq_mode 0 reset_mode 0
[ 15.385363] ath10k_pci 0001:01:00.0: Direct firmware load for ath10k/fwcfg-pci-0001:01:00.0.txt failed with error -2
[ 15.385398] ath10k_pci 0001:01:00.0: Falling back to user helper
[ 15.539998] firmware ath10k!fwcfg-pci-0001:01:00.0.txt: firmware_loading_store: map pages failed
[ 15.542209] ath10k_pci 0001:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/ct-firmware-5.bin failed with error -2
[ 15.547963] ath10k_pci 0001:01:00.0: Falling back to user helper
[ 15.597070] firmware ath10k!QCA99X0!hw2.0!ct-firmware-5.bin: firmware_loading_store: map pages failed
[ 15.597256] ath10k_pci 0001:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/ct-firmware-2.bin failed with error -2
[ 15.605365] ath10k_pci 0001:01:00.0: Falling back to user helper
[ 15.643800] firmware ath10k!QCA99X0!hw2.0!ct-firmware-2.bin: firmware_loading_store: map pages failed
[ 15.643997] ath10k_pci 0001:01:00.0: Direct firmware load for ath10k/QCA99X0/hw2.0/firmware-6.bin failed with error -2
[ 15.652008] ath10k_pci 0001:01:00.0: Falling back to user helper
[ 15.684047] firmware ath10k!QCA99X0!hw2.0!firmware-6.bin: firmware_loading_store: map pages failed
[ 15.684213] ath10k_pci 0001:01:00.0: qca99x0 hw2.0 target 0x01000000 chip_id 0x003801ff sub 168c:0002
[ 15.691909] ath10k_pci 0001:01:00.0: kconfig debug 0 debugfs 1 tracing 0 dfs 1 testmode 0
[ 15.703797] ath10k_pci 0001:01:00.0: firmware ver 10.4b-ct-9980-fW-13-10af6a005 api 5 features mfp,peer-flow-ctrl,txstatus-noack,wmi-10.x-CT,ratemask-CT,regdump-CT,txrate-CT,flush-all-CT,pingpong-CT,ch-regs-CT,nop-CT,set-special-CT,tx-rc-CT,cust-stats-CT,txrate2-CT,beacon-cb-CT,wmi-block-ack-CT,wmi-bcn-rc-CT crc32 4a0fa880
[ 15.773937] ath10k_pci 0001:01:00.0: board_file api 2 bmi_id 1:1 crc32 08fa09f2
[ 16.960105] ath10k_pci 0001:01:00.0: 10.4 wmi init: vdevs: 16 peers: 48 tid: 96
[ 16.960134] ath10k_pci 0001:01:00.0: msdu-desc: 2500 skid: 32
[ 17.036098] ath10k_pci 0001:01:00.0: wmi print 'P 48/48 V 16 K 144 PH 176 T 186 msdu-desc: 2500 sw-crypt: 0 ct-sta: 0'
[ 17.036865] ath10k_pci 0001:01:00.0: wmi print 'free: 31080 iram: 23092 sram: 9596'
[ 17.249487] ath10k_pci 0001:01:00.0: htt-ver 2.2 wmi-op 6 htt-op 4 cal pre-cal-file max-sta 32 raw 0 hwcrypto 1
[ 17.373982] ath: EEPROM regdomain: 0x0
[ 17.373994] ath: EEPROM indicates default country code should be used
[ 17.374004] ath: doing EEPROM country->regdmn map search
[ 17.374019] ath: country maps to regdmn code: 0x3a
[ 17.374032] ath: Country alpha2 being used: US
[ 17.374041] ath: Regpair used: 0x3a
[ 17.384151] kmodloader: done loading kernel modules from /etc/modules.d/*
[ 20.751437] Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=POLL)
[ 20.752231] dwmac1000: Master AXI performs any burst length
[ 20.760457] ipq806x-gmac-dwmac 37200000.ethernet eth0: IEEE 1588-2008 Advanced Timestamp supported
[ 20.765875] ipq806x-gmac-dwmac 37200000.ethernet eth0: registered PTP clock
[ 20.774913] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
[ 20.783992] br-lan: port 1(eth0.1) entered blocking state
[ 20.787496] br-lan: port 1(eth0.1) entered disabled state
[ 20.793128] device eth0.1 entered promiscuous mode
[ 20.798414] device eth0 entered promiscuous mode
[ 20.804756] IPv6: ADDRCONF(NETDEV_UP): br-lan: link is not ready
[ 20.815585] IPv6: ADDRCONF(NETDEV_UP): eth0.2: link is not ready
[ 21.833787] ipq806x-gmac-dwmac 37200000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
[ 21.833837] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 21.851508] br-lan: port 1(eth0.1) entered blocking state
[ 21.851542] br-lan: port 1(eth0.1) entered forwarding state
[ 21.856170] IPv6: ADDRCONF(NETDEV_CHANGE): eth0.2: link becomes ready
[ 21.874027] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
logread collected after the run:
[...]
Wed Oct 14 16:10:54 2020 user.notice SQM: Starting SQM script: simple.qos on eth0.2, in: 800000 Kbps, out: 800000 Kbps
Wed Oct 14 16:10:54 2020 daemon.notice procd: /etc/rc.d/S50sqm: SQM: Starting SQM script: simple.qos on eth0.2, in: 800000 Kbps, out: 800000 Kbps
Wed Oct 14 16:10:54 2020 user.notice SQM: Using generic sqm_start_default function.
Wed Oct 14 16:10:54 2020 daemon.notice procd: /etc/rc.d/S50sqm: SQM: Using generic sqm_start_default function.
Wed Oct 14 16:10:55 2020 user.notice SQM: simple.qos was started on eth0.2 successfully
Wed Oct 14 16:10:55 2020 daemon.notice procd: /etc/rc.d/S50sqm: SQM: simple.qos was started on eth0.2 successfully
Wed Oct 14 16:10:55 2020 daemon.err odhcpd[884]: Failed to send to ff02::1%lan@br-lan (Address not available)
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Network device 'eth0' link is up
Wed Oct 14 16:10:55 2020 kern.info kernel: [ 21.833787] ipq806x-gmac-dwmac 37200000.ethernet eth0: Link is Up - 1Gbps/Full - flow control off
Wed Oct 14 16:10:55 2020 kern.info kernel: [ 21.833837] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Wed Oct 14 16:10:55 2020 kern.info kernel: [ 21.851508] br-lan: port 1(eth0.1) entered blocking state
Wed Oct 14 16:10:55 2020 kern.info kernel: [ 21.851542] br-lan: port 1(eth0.1) entered forwarding state
Wed Oct 14 16:10:55 2020 daemon.notice netifd: VLAN 'eth0.1' link is up
Wed Oct 14 16:10:55 2020 daemon.notice netifd: VLAN 'eth0.2' link is up
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Interface 'wan' has link connectivity
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Interface 'wan' is setting up now
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Interface 'wan6' has link connectivity
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Interface 'wan6' is setting up now
Wed Oct 14 16:10:55 2020 kern.info kernel: [ 21.856170] IPv6: ADDRCONF(NETDEV_CHANGE): eth0.2: link becomes ready
Wed Oct 14 16:10:55 2020 daemon.notice netifd: bridge 'br-lan' link is up
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Interface 'lan' has link connectivity
Wed Oct 14 16:10:55 2020 kern.info kernel: [ 21.874027] IPv6: ADDRCONF(NETDEV_CHANGE): br-lan: link becomes ready
Wed Oct 14 16:10:55 2020 daemon.notice netifd: wan (2113): udhcpc: started, v1.30.1
Wed Oct 14 16:10:55 2020 daemon.err odhcp6c[2116]: Failed to send RS (Address not available)
Wed Oct 14 16:10:55 2020 user.notice firewall: Reloading firewall due to ifup of lan (br-lan)
Wed Oct 14 16:10:55 2020 daemon.notice netifd: wan (2113): udhcpc: sending discover
Wed Oct 14 16:10:55 2020 daemon.notice netifd: wan (2113): udhcpc: sending select for 10.0.0.189
Wed Oct 14 16:10:55 2020 daemon.notice netifd: wan (2113): udhcpc: lease of 10.0.0.189 obtained, lease time 3600
Wed Oct 14 16:10:55 2020 user.notice ucitrack: Setting up /etc/config/network reload dependency on /etc/config/dhcp
Wed Oct 14 16:10:55 2020 user.notice ucitrack: Setting up /etc/config/network reload dependency on /etc/config/radvd
Wed Oct 14 16:10:55 2020 user.notice ucitrack: Setting up /etc/config/wireless reload dependency on /etc/config/network
Wed Oct 14 16:10:55 2020 daemon.notice netifd: Interface 'wan' is now up
Wed Oct 14 16:10:55 2020 user.notice ucitrack: Setting up /etc/config/firewall reload dependency on /etc/config/luci-splash
Wed Oct 14 16:10:55 2020 user.notice ucitrack: Setting up /etc/config/firewall reload dependency on /etc/config/qos
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up /etc/config/firewall reload dependency on /etc/config/miniupnpd
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up /etc/config/firewall reload dependency on /etc/config/sqm
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up /etc/config/dhcp reload dependency on /etc/config/odhcpd
Wed Oct 14 16:10:56 2020 user.notice SQM: Stopping SQM on eth0.2
Wed Oct 14 16:10:56 2020 daemon.err odhcp6c[2116]: Failed to send DHCPV6 message to ff02::1:2 (Address not available)
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up non-init /etc/config/fstab reload handler: /sbin/block mount
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up /etc/config/system reload trigger for non-procd /etc/init.d/led
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up /etc/config/system reload dependency on /etc/config/luci_statistics
Wed Oct 14 16:10:56 2020 user.notice ucitrack: Setting up /etc/config/system reload dependency on /etc/config/dhcp
Wed Oct 14 16:10:56 2020 user.notice SQM: Starting SQM script: simple.qos on eth0.2, in: 800000 Kbps, out: 800000 Kbps
Wed Oct 14 16:10:56 2020 user.notice SQM: Using generic sqm_start_default function.
Wed Oct 14 16:10:56 2020 daemon.notice procd: /etc/rc.d/S99bootcount: Boot count sucessfully reset to zero.
Wed Oct 14 16:10:56 2020 daemon.info procd: - init complete -
Wed Oct 14 16:10:57 2020 daemon.err odhcp6c[2116]: Failed to send DHCPV6 message to ff02::1:2 (Address not available)
Wed Oct 14 16:10:57 2020 user.notice SQM: simple.qos was started on eth0.2 successfully
Wed Oct 14 16:10:57 2020 user.notice firewall: Reloading firewall due to ifup of wan (eth0.2)
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[755]: exiting on receipt of SIGTERM
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: started, version 2.80 cachesize 150
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: DNS service limited to local subnets
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-DNSSEC no-ID loop-detect inotify dumpfile
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq-dhcp[3548]: DHCP, IP range 192.168.1.100 -- 192.168.1.249, lease time 12h
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain test
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain onion
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain localhost
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain local
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain invalid
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain bind
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: using local addresses only for domain lan
Wed Oct 14 16:10:59 2020 daemon.warn dnsmasq[3548]: no servers found in /tmp/resolv.conf.auto, will retry
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: read /etc/hosts - 4 addresses
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: read /tmp/hosts/dhcp.cfg01411c - 1 addresses
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq-dhcp[3548]: read /etc/ethers - 0 addresses
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3548]: exiting on receipt of SIGTERM
Wed Oct 14 16:10:59 2020 daemon.err procd: Got unexpected signal 1
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: started, version 2.80 cachesize 150
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: DNS service limited to local subnets
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: compile time options: IPv6 GNU-getopt no-DBus no-i18n no-IDN DHCP no-DHCPv6 no-Lua TFTP no-conntrack no-ipset no-auth no-DNSSEC no-ID loop-detect inotify dumpfile
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq-dhcp[3623]: DHCP, IP range 192.168.1.100 -- 192.168.1.249, lease time 12h
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain test
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain onion
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain localhost
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain local
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain invalid
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain bind
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: using local addresses only for domain lan
Wed Oct 14 16:10:59 2020 daemon.warn dnsmasq[3623]: no servers found in /tmp/resolv.conf.auto, will retry
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: read /etc/hosts - 4 addresses
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq[3623]: read /tmp/hosts/dhcp.cfg01411c - 1 addresses
Wed Oct 14 16:10:59 2020 daemon.info dnsmasq-dhcp[3623]: read /etc/ethers - 0 addresses
Wed Oct 14 16:11:06 2020 daemon.info dnsmasq[3623]: read /etc/hosts - 4 addresses
Wed Oct 14 16:11:06 2020 daemon.info dnsmasq[3623]: read /tmp/hosts/odhcpd - 1 addresses
Wed Oct 14 16:11:06 2020 daemon.info dnsmasq[3623]: read /tmp/hosts/dhcp.cfg01411c - 1 addresses
Wed Oct 14 16:11:06 2020 daemon.info dnsmasq-dhcp[3623]: read /etc/ethers - 0 addresses
Wed Oct 14 16:11:12 2020 daemon.info dnsmasq[3623]: read /etc/hosts - 4 addresses
Wed Oct 14 16:11:12 2020 daemon.info dnsmasq[3623]: read /tmp/hosts/odhcpd - 2 addresses
Wed Oct 14 16:11:12 2020 daemon.info dnsmasq[3623]: read /tmp/hosts/dhcp.cfg01411c - 1 addresses
Wed Oct 14 16:11:12 2020 daemon.info dnsmasq-dhcp[3623]: read /etc/ethers - 0 addresses
Wed Oct 14 16:11:16 2020 daemon.info dnsmasq-dhcp[3623]: DHCPDISCOVER(br-lan) 192.168.1.225 aa:aa:aa:bb:bb:bb
Wed Oct 14 16:11:16 2020 daemon.info dnsmasq-dhcp[3623]: DHCPOFFER(br-lan) 192.168.1.225 aa:aa:aa:bb:bb:bb
Wed Oct 14 16:11:16 2020 daemon.info dnsmasq-dhcp[3623]: DHCPREQUEST(br-lan) 192.168.1.225 aa:aa:aa:bb:bb:bb
Wed Oct 14 16:11:16 2020 daemon.info dnsmasq-dhcp[3623]: DHCPACK(br-lan) 192.168.1.225 aa:aa:aa:bb:bb:bb arch-machine-0-c2d6120c765ec16a
Wed Oct 14 16:12:03 2020 authpriv.info dropbear[3645]: Child connection from 192.168.1.10:53296
Wed Oct 14 16:12:03 2020 authpriv.notice dropbear[3645]: Pubkey auth succeeded for 'root' with key sha1!! 99:3a:ca:0f:e7:c9:92:b7:1b:3a:c8:40:b1:7a:cb:cc:35:39:74:21 from 192.168.1.10:53296
I would recommend to switch to the performance governor or configure the ondemand one to be less twitchy. The issue seems to be that cake is a bursty load, that on average does not keep the CPU busy enough to make the ondemand governor keeping the CPU in high frequency mode, but is latency sensitive enough, that the shaper does not get enough CPU cycles at lower frequencies (but is too short to profit from the increasing frequencies once the CPU scales up again).
Unfortunately, neither HTB nor fq_codel offer as nice statistics output as cake does (I believe that tc -d qdisc might give some more detail for fq_codel).
Mmmh, 21% is not terrible, if that is the absolute minimum, but top -d 1 samples in 1 second intervals, and 21% idle might be any 210milliseconds in that interval and cake is sufficient latency sensitive that not getting enough CPU for 790ms will show either in decreased throughput or increased latency under load or both... that would call for a closer monitoring of CPU load (which in itself will cause a noticeable CPU load)...
Thanks for all the data.
I've tried another iperf3 run with the performance governor.
# cat /sys/devices/system/cpu/cpufreq/policy?/scaling_governor
performance
performance
The iperf3 results aren't significantly different.
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-60.00 sec 4.57 GBytes 654 Mbits/sec 1925 sender
[ 5][TX-C] 0.00-60.00 sec 4.57 GBytes 654 Mbits/sec receiver
[ 7][RX-C] 0.00-60.00 sec 659 MBytes 92.1 Mbits/sec 422 sender
[ 7][RX-C] 0.00-60.00 sec 656 MBytes 91.7 Mbits/sec receiver
Here's tc -d qdisc after this run:
# tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc mq 0: dev eth0 root
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth0.1 root refcnt 2
qdisc htb 1: dev eth0.2 root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 ver 3.17 direct_qlen 1000
qdisc fq_codel 120: dev eth0.2 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 130: dev eth0.2 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 110: dev eth0.2 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
qdisc htb 1: dev ifb4eth0.2 root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 ver 3.17 direct_qlen 32
qdisc fq_codel 110: dev ifb4eth0.2 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
A bit later, and after another run, here are the cpufreq stats:
root@OpenWrt:~# cat /sys/devices/system/cpu/cpufreq/policy0/stats/time_in_state
384000 956
600000 102
800000 64
1000000 96
1200000 76
1400000 221166
root@OpenWrt:~# cat /sys/devices/system/cpu/cpufreq/policy1/stats/time_in_state
384000 838
600000 90
800000 57
1000000 98
1200000 81
1400000 222070
I'm set up to measure current at the 12VDC input. With the performance governor, I'm seeing 389mA at idle and 485mA while running a bidirectional iperf3 test. The ondemand numbers are almost the same, which is not what I would expect.
This is another iperf3 run, still with the performance governor, and comparing the client and server output is interesting. The server reports sending 0 bytes for long stretches of time.
Client output:
$ iperf3 -c 10.0.0.1 --bidir --time 60 | tee iperf3-output.txt
Connecting to host 10.0.0.1, port 5201
[ 5] local 192.168.1.225 port 36082 connected to 10.0.0.1 port 5201
[ 7] local 192.168.1.225 port 36084 connected to 10.0.0.1 port 5201
[ ID][Role] Interval Transfer Bitrate Retr Cwnd
[ 5][TX-C] 0.00-1.00 sec 76.3 MBytes 640 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 0.00-1.00 sec 11.2 MBytes 94.3 Mbits/sec
[ 5][TX-C] 1.00-2.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 1.00-2.00 sec 60.8 KBytes 498 Kbits/sec
[ 5][TX-C] 2.00-3.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 2.00-3.00 sec 180 KBytes 1.47 Mbits/sec
[ 5][TX-C] 3.00-4.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 3.00-4.00 sec 810 KBytes 6.64 Mbits/sec
[ 5][TX-C] 4.00-5.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 4.00-5.00 sec 214 KBytes 1.75 Mbits/sec
[ 5][TX-C] 5.00-6.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 5.00-6.00 sec 344 KBytes 2.81 Mbits/sec
[ 5][TX-C] 6.00-7.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 6.00-7.00 sec 270 KBytes 2.21 Mbits/sec
[ 5][TX-C] 7.00-8.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 7.00-8.00 sec 257 KBytes 2.11 Mbits/sec
[ 5][TX-C] 8.00-9.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 8.00-9.00 sec 266 KBytes 2.18 Mbits/sec
[ 5][TX-C] 9.00-10.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 9.00-10.00 sec 266 KBytes 2.18 Mbits/sec
[ 5][TX-C] 10.00-11.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 10.00-11.00 sec 264 KBytes 2.17 Mbits/sec
[ 5][TX-C] 11.00-12.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 11.00-12.00 sec 263 KBytes 2.15 Mbits/sec
[ 5][TX-C] 12.00-13.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 12.00-13.00 sec 256 KBytes 2.10 Mbits/sec
[ 5][TX-C] 13.00-14.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 13.00-14.00 sec 252 KBytes 2.06 Mbits/sec
[ 5][TX-C] 14.00-15.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 14.00-15.00 sec 249 KBytes 2.04 Mbits/sec
[ 5][TX-C] 15.00-16.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 15.00-16.00 sec 283 KBytes 2.32 Mbits/sec
[ 5][TX-C] 16.00-17.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 16.00-17.00 sec 269 KBytes 2.20 Mbits/sec
[ 5][TX-C] 17.00-18.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 17.00-18.00 sec 270 KBytes 2.21 Mbits/sec
[ 5][TX-C] 18.00-19.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 18.00-19.00 sec 276 KBytes 2.26 Mbits/sec
[ 5][TX-C] 19.00-20.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 19.00-20.00 sec 256 KBytes 2.10 Mbits/sec
[ 5][TX-C] 20.00-21.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 20.00-21.00 sec 272 KBytes 2.22 Mbits/sec
[ 5][TX-C] 21.00-22.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 21.00-22.00 sec 255 KBytes 2.09 Mbits/sec
[ 5][TX-C] 22.00-23.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 22.00-23.00 sec 262 KBytes 2.14 Mbits/sec
[ 5][TX-C] 23.00-24.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 23.00-24.00 sec 253 KBytes 2.07 Mbits/sec
[ 5][TX-C] 24.00-25.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 24.00-25.00 sec 260 KBytes 2.13 Mbits/sec
[ 5][TX-C] 25.00-26.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 25.00-26.00 sec 250 KBytes 2.05 Mbits/sec
[ 5][TX-C] 26.00-27.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 26.00-27.00 sec 255 KBytes 2.08 Mbits/sec
[ 5][TX-C] 27.00-28.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 27.00-28.00 sec 277 KBytes 2.27 Mbits/sec
[ 5][TX-C] 28.00-29.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 28.00-29.00 sec 279 KBytes 2.28 Mbits/sec
[ 5][TX-C] 29.00-30.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 29.00-30.00 sec 255 KBytes 2.09 Mbits/sec
[ 5][TX-C] 30.00-31.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 30.00-31.00 sec 272 KBytes 2.22 Mbits/sec
[ 5][TX-C] 31.00-32.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 31.00-32.00 sec 272 KBytes 2.22 Mbits/sec
[ 5][TX-C] 32.00-33.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 32.00-33.00 sec 255 KBytes 2.09 Mbits/sec
[ 5][TX-C] 33.00-34.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 33.00-34.00 sec 264 KBytes 2.17 Mbits/sec
[ 5][TX-C] 34.00-35.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 34.00-35.00 sec 257 KBytes 2.11 Mbits/sec
[ 5][TX-C] 35.00-36.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 35.00-36.00 sec 262 KBytes 2.14 Mbits/sec
[ 5][TX-C] 36.00-37.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 36.00-37.00 sec 230 KBytes 1.89 Mbits/sec
[ 5][TX-C] 37.00-38.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 37.00-38.00 sec 269 KBytes 2.20 Mbits/sec
[ 5][TX-C] 38.00-39.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 38.00-39.00 sec 277 KBytes 2.27 Mbits/sec
[ 5][TX-C] 39.00-40.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 39.00-40.00 sec 205 KBytes 1.68 Mbits/sec
[ 5][TX-C] 40.00-41.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 40.00-41.00 sec 199 KBytes 1.63 Mbits/sec
[ 5][TX-C] 41.00-42.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 41.00-42.00 sec 273 KBytes 2.24 Mbits/sec
[ 5][TX-C] 42.00-43.00 sec 92.5 MBytes 776 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 42.00-43.00 sec 269 KBytes 2.20 Mbits/sec
[ 5][TX-C] 43.00-44.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 43.00-44.00 sec 269 KBytes 2.20 Mbits/sec
[ 5][TX-C] 44.00-45.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 44.00-45.00 sec 257 KBytes 2.11 Mbits/sec
[ 5][TX-C] 45.00-46.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 45.00-46.00 sec 263 KBytes 2.15 Mbits/sec
[ 5][TX-C] 46.00-47.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 46.00-47.00 sec 263 KBytes 2.15 Mbits/sec
[ 5][TX-C] 47.00-48.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 47.00-48.00 sec 256 KBytes 2.10 Mbits/sec
[ 5][TX-C] 48.00-49.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 48.00-49.00 sec 260 KBytes 2.13 Mbits/sec
[ 5][TX-C] 49.00-50.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 49.00-50.00 sec 277 KBytes 2.27 Mbits/sec
[ 5][TX-C] 50.00-51.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 50.00-51.00 sec 240 KBytes 1.97 Mbits/sec
[ 5][TX-C] 51.00-52.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 51.00-52.00 sec 277 KBytes 2.27 Mbits/sec
[ 5][TX-C] 52.00-53.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 52.00-53.00 sec 276 KBytes 2.26 Mbits/sec
[ 5][TX-C] 53.00-54.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 53.00-54.00 sec 266 KBytes 2.18 Mbits/sec
[ 5][TX-C] 54.00-55.00 sec 92.5 MBytes 776 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 54.00-55.00 sec 269 KBytes 2.20 Mbits/sec
[ 5][TX-C] 55.00-56.00 sec 90.0 MBytes 755 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 55.00-56.00 sec 264 KBytes 2.17 Mbits/sec
[ 5][TX-C] 56.00-57.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 56.00-57.00 sec 259 KBytes 2.12 Mbits/sec
[ 5][TX-C] 57.00-58.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 57.00-58.00 sec 255 KBytes 2.09 Mbits/sec
[ 5][TX-C] 58.00-59.00 sec 91.2 MBytes 766 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 58.00-59.00 sec 263 KBytes 2.15 Mbits/sec
[ 5][TX-C] 59.00-60.00 sec 91.2 MBytes 765 Mbits/sec 0 3.01 MBytes
[ 7][RX-C] 59.00-60.00 sec 276 KBytes 2.26 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-60.00 sec 5.32 GBytes 762 Mbits/sec 0 sender
[ 5][TX-C] 0.00-60.00 sec 5.32 GBytes 761 Mbits/sec receiver
[ 7][RX-C] 0.00-60.00 sec 29.5 MBytes 4.12 Mbits/sec 1021 sender
[ 7][RX-C] 0.00-60.00 sec 26.6 MBytes 3.72 Mbits/sec receiver
iperf Done.
Server output:
Accepted connection from 10.0.0.189, port 36080
[ 5] local 10.0.0.1 port 5201 connected to 10.0.0.189 port 36082
[ 8] local 10.0.0.1 port 5201 connected to 10.0.0.189 port 36084
[ ID][Role] Interval Transfer Bitrate Retr Cwnd
[ 5][RX-S] 0.00-1.00 sec 75.9 MBytes 637 Mbits/sec
[ 8][TX-S] 0.00-1.00 sec 13.7 MBytes 115 Mbits/sec 40 276 KBytes
[ 5][RX-S] 1.00-2.00 sec 90.9 MBytes 762 Mbits/sec
[ 8][TX-S] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 100 195 KBytes
[ 5][RX-S] 2.00-3.00 sec 90.9 MBytes 762 Mbits/sec
[ 8][TX-S] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 125 18.4 KBytes
[ 5][RX-S] 3.00-4.00 sec 90.5 MBytes 759 Mbits/sec
[ 8][TX-S] 3.00-4.00 sec 1.25 MBytes 10.5 Mbits/sec 145 130 KBytes
[ 5][RX-S] 4.00-5.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 98 43.8 KBytes
[ 5][RX-S] 5.00-6.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 31 7.07 KBytes
[ 5][RX-S] 6.00-7.00 sec 90.8 MBytes 762 Mbits/sec
[ 8][TX-S] 6.00-7.00 sec 1.11 MBytes 9.32 Mbits/sec 14 4.24 KBytes
[ 5][RX-S] 7.00-8.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 12 5.66 KBytes
[ 5][RX-S] 8.00-9.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 7 11.3 KBytes
[ 5][RX-S] 9.00-10.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 12 7.07 KBytes
[ 5][RX-S] 10.00-11.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 10.00-11.00 sec 1.12 MBytes 9.38 Mbits/sec 8 7.07 KBytes
[ 5][RX-S] 11.00-12.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 11.00-12.00 sec 0.00 Bytes 0.00 bits/sec 7 11.3 KBytes
[ 5][RX-S] 12.00-13.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 12.00-13.00 sec 0.00 Bytes 0.00 bits/sec 9 8.48 KBytes
[ 5][RX-S] 13.00-14.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 13.00-14.00 sec 0.00 Bytes 0.00 bits/sec 10 5.66 KBytes
[ 5][RX-S] 14.00-15.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 14.00-15.00 sec 1.12 MBytes 9.38 Mbits/sec 6 7.07 KBytes
[ 5][RX-S] 15.00-16.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 15.00-16.00 sec 0.00 Bytes 0.00 bits/sec 10 5.66 KBytes
[ 5][RX-S] 16.00-17.00 sec 91.0 MBytes 764 Mbits/sec
[ 8][TX-S] 16.00-17.00 sec 0.00 Bytes 0.00 bits/sec 11 7.07 KBytes
[ 5][RX-S] 17.00-18.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 17.00-18.00 sec 0.00 Bytes 0.00 bits/sec 10 7.07 KBytes
[ 5][RX-S] 18.00-19.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 18.00-19.00 sec 1.12 MBytes 9.38 Mbits/sec 9 7.07 KBytes
[ 5][RX-S] 19.00-20.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 19.00-20.00 sec 0.00 Bytes 0.00 bits/sec 10 7.07 KBytes
[ 5][RX-S] 20.00-21.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 20.00-21.00 sec 0.00 Bytes 0.00 bits/sec 8 7.07 KBytes
[ 5][RX-S] 21.00-22.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 21.00-22.00 sec 0.00 Bytes 0.00 bits/sec 7 5.66 KBytes
[ 5][RX-S] 22.00-23.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 22.00-23.00 sec 0.00 Bytes 0.00 bits/sec 7 9.90 KBytes
[ 5][RX-S] 23.00-24.00 sec 91.3 MBytes 765 Mbits/sec
[ 8][TX-S] 23.00-24.00 sec 1.12 MBytes 9.38 Mbits/sec 11 5.66 KBytes
[ 5][RX-S] 24.00-25.00 sec 91.1 MBytes 765 Mbits/sec
[ 8][TX-S] 24.00-25.00 sec 0.00 Bytes 0.00 bits/sec 7 7.07 KBytes
[ 5][RX-S] 25.00-26.00 sec 91.0 MBytes 764 Mbits/sec
[ 8][TX-S] 25.00-26.00 sec 0.00 Bytes 0.00 bits/sec 8 5.66 KBytes
[ 5][RX-S] 26.00-27.00 sec 90.9 MBytes 762 Mbits/sec
[ 8][TX-S] 26.00-27.00 sec 0.00 Bytes 0.00 bits/sec 8 9.90 KBytes
[ 5][RX-S] 27.00-28.00 sec 90.9 MBytes 762 Mbits/sec
[ 8][TX-S] 27.00-28.00 sec 1.12 MBytes 9.38 Mbits/sec 6 5.66 KBytes
[ 5][RX-S] 28.00-29.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 28.00-29.00 sec 0.00 Bytes 0.00 bits/sec 13 5.66 KBytes
[ 5][RX-S] 29.00-30.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 29.00-30.00 sec 0.00 Bytes 0.00 bits/sec 11 2.83 KBytes
[ 5][RX-S] 30.00-31.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 30.00-31.00 sec 0.00 Bytes 0.00 bits/sec 6 9.90 KBytes
[ 5][RX-S] 31.00-32.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 31.00-32.00 sec 0.00 Bytes 0.00 bits/sec 7 8.48 KBytes
[ 5][RX-S] 32.00-33.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 32.00-33.00 sec 1.12 MBytes 9.38 Mbits/sec 11 9.90 KBytes
[ 5][RX-S] 33.00-34.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 33.00-34.00 sec 0.00 Bytes 0.00 bits/sec 9 8.48 KBytes
[ 5][RX-S] 34.00-35.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 34.00-35.00 sec 0.00 Bytes 0.00 bits/sec 7 8.48 KBytes
[ 5][RX-S] 35.00-36.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 35.00-36.00 sec 0.00 Bytes 0.00 bits/sec 8 7.07 KBytes
[ 5][RX-S] 36.00-37.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 36.00-37.00 sec 1.12 MBytes 9.38 Mbits/sec 8 9.90 KBytes
[ 5][RX-S] 37.00-38.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 37.00-38.00 sec 0.00 Bytes 0.00 bits/sec 10 5.66 KBytes
[ 5][RX-S] 38.00-39.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 38.00-39.00 sec 0.00 Bytes 0.00 bits/sec 11 4.24 KBytes
[ 5][RX-S] 39.00-40.00 sec 90.8 MBytes 762 Mbits/sec
[ 8][TX-S] 39.00-40.00 sec 0.00 Bytes 0.00 bits/sec 9 7.07 KBytes
[ 5][RX-S] 40.00-41.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 40.00-41.00 sec 0.00 Bytes 0.00 bits/sec 9 8.48 KBytes
[ 5][RX-S] 41.00-42.00 sec 90.9 MBytes 762 Mbits/sec
[ 8][TX-S] 41.00-42.00 sec 1.12 MBytes 9.38 Mbits/sec 9 8.48 KBytes
[ 5][RX-S] 42.00-43.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 42.00-43.00 sec 0.00 Bytes 0.00 bits/sec 13 7.07 KBytes
[ 5][RX-S] 43.00-44.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 43.00-44.00 sec 0.00 Bytes 0.00 bits/sec 7 8.48 KBytes
[ 5][RX-S] 44.00-45.00 sec 91.1 MBytes 765 Mbits/sec
[ 8][TX-S] 44.00-45.00 sec 0.00 Bytes 0.00 bits/sec 9 4.24 KBytes
[ 5][RX-S] 45.00-46.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 45.00-46.00 sec 1.12 MBytes 9.38 Mbits/sec 5 8.48 KBytes
[ 5][RX-S] 46.00-47.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 46.00-47.00 sec 0.00 Bytes 0.00 bits/sec 12 7.07 KBytes
[ 5][RX-S] 47.00-48.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 47.00-48.00 sec 0.00 Bytes 0.00 bits/sec 7 7.07 KBytes
[ 5][RX-S] 48.00-49.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 48.00-49.00 sec 0.00 Bytes 0.00 bits/sec 8 7.07 KBytes
[ 5][RX-S] 49.00-50.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 49.00-50.00 sec 1.12 MBytes 9.38 Mbits/sec 10 5.66 KBytes
[ 5][RX-S] 50.00-51.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 50.00-51.00 sec 0.00 Bytes 0.00 bits/sec 10 9.90 KBytes
[ 5][RX-S] 51.00-52.00 sec 91.0 MBytes 763 Mbits/sec
[ 8][TX-S] 51.00-52.00 sec 0.00 Bytes 0.00 bits/sec 13 8.48 KBytes
[ 5][RX-S] 52.00-53.00 sec 90.9 MBytes 762 Mbits/sec
[ 8][TX-S] 52.00-53.00 sec 0.00 Bytes 0.00 bits/sec 8 8.48 KBytes
[ 5][RX-S] 53.00-54.00 sec 91.0 MBytes 764 Mbits/sec
[ 8][TX-S] 53.00-54.00 sec 0.00 Bytes 0.00 bits/sec 9 7.07 KBytes
[ 5][RX-S] 54.00-55.00 sec 90.9 MBytes 763 Mbits/sec
[ 8][TX-S] 54.00-55.00 sec 1.12 MBytes 9.38 Mbits/sec 7 5.66 KBytes
[ 5][RX-S] 55.00-56.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 55.00-56.00 sec 0.00 Bytes 0.00 bits/sec 9 7.07 KBytes
[ 5][RX-S] 56.00-57.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 56.00-57.00 sec 0.00 Bytes 0.00 bits/sec 5 8.48 KBytes
[ 5][RX-S] 57.00-58.00 sec 91.2 MBytes 765 Mbits/sec
[ 8][TX-S] 57.00-58.00 sec 0.00 Bytes 0.00 bits/sec 8 8.48 KBytes
[ 5][RX-S] 58.00-59.00 sec 91.1 MBytes 764 Mbits/sec
[ 8][TX-S] 58.00-59.00 sec 1.12 MBytes 9.38 Mbits/sec 9 7.07 KBytes
[ 5][RX-S] 59.00-60.00 sec 91.0 MBytes 764 Mbits/sec
[ 8][TX-S] 59.00-60.00 sec 0.00 Bytes 0.00 bits/sec 8 9.90 KBytes
[ 5][RX-S] 60.00-60.00 sec 317 KBytes 800 Mbits/sec
[ 8][TX-S] 60.00-60.00 sec 0.00 Bytes 0.00 bits/sec 0 9.90 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][RX-S] 0.00-60.00 sec 5.32 GBytes 761 Mbits/sec receiver
[ 8][TX-S] 0.00-60.00 sec 29.5 MBytes 4.12 Mbits/sec 1021 sender
Here are some new results from the multi-port NIC test setup. There are 5 different router models (plus a no-router, switch-only baseline), each tested with and without SQM (800Mbit/800Mbit) and with and without software flow offloading.
The upload, download, and bidir columns are the average results in Mbps from goben, a golang tool like iperf. The 4p means these were runs with 4 parallel streams. The http columns are for caddy serving a random file of the size given, and the hey tool retrieving the file for 60 seconds, with 50 workers, reporting the average requests per second.
The Atomic Pi (AAEON MF-001 in the table) and Pi 4 were tested with an RTL8153-based USB 3.0 gigabit adapter as a WAN link.
| Run | Router | SQM | SoftFlowOffload | upload-4p | download-4p | bidir-up-4p | bidir-dn-4p | http-1KiB | http-100KiB | http-10000KiB |
|---|---|---|---|---|---|---|---|---|---|---|
| 1604156365 | no router, 8000 mtu both NICs | NA | OFF | 986 | 984 | 982 | 982 | 28294.50 | 1203.19 | 12.04 |
| 1604153917 | no router | NA | OFF | 940 | 940 | 931 | 926 | 28228.82 | 1145.91 | 11.49 |
| 1603936584 | AAEON MF-001 | OFF | ON | 932 | 940 | 864 | 849 | 27849.07 | 1145.73 | 11.49 |
| 1604011594 | AAEON MF-001 | OFF | OFF | 932 | 939 | 796 | 875 | 27691.97 | 1145.84 | 11.49 |
| 1604020449 | Raspberry Pi 4 Model B Rev 1.1 | OFF | ON | 939 | 939 | 693 | 812 | 27986.27 | 1145.75 | 11.49 |
| 1604022831 | Raspberry Pi 4 Model B Rev 1.1 | OFF | OFF | 939 | 939 | 698 | 645 | 28002.81 | 1146.00 | 11.49 |
| 1604028557 | Linksys EA8500 WiFi Router | OFF | ON | 902 | 902 | 454 | 434 | 27697.64 | 996.42 | 9.11 |
| 1603934190 | AAEON MF-001 | ON | OFF | 764 | 764 | 739 | 744 | 26892.87 | 859.82 | 9.13 |
| 1604018070 | Raspberry Pi 4 Model B Rev 1.1 | ON | OFF | 764 | 764 | 688 | 572 | 27849.89 | 931.12 | 9.22 |
| 1604105001 | Western Digital My Net N750 | OFF | ON | 787 | 763 | 376 | 374 | 28126.49 | 766.55 | 7.73 |
| 1604030957 | Linksys EA8500 WiFi Router | OFF | OFF | 728 | 711 | 247 | 422 | 26018.58 | 573.10 | 5.62 |
| 1604065573 | Buffalo WZR-HP-G300NH | OFF | ON | 691 | 679 | 424 | 330 | 27875.83 | 758.39 | 7.56 |
| 1604026160 | Linksys EA8500 WiFi Router | ON | OFF | 608 | 409 | 570 | 0 | 19108.70 | 380.10 | 3.87 |
| 1604068022 | Buffalo WZR-HP-G300NH | OFF | OFF | 300 | 303 | 156 | 156 | 16340.22 | 324.24 | 3.23 |
| 1604107427 | Western Digital My Net N750 | OFF | OFF | 284 | 294 | 147 | 135 | 15386.65 | 317.91 | 3.16 |
| 1604102579 | Western Digital My Net N750 | ON | OFF | 140 | 148 | 99 | 61 | 8858.54 | 138.47 | 2.50 |
The old Buffalo router was not stable running this benchmark with SQM enabled. The Western Digital router also crashed once when running with SQM enabled. The EA8500 has a puzzling very poor upload rate in the bidirectional test with SQM.
I also collected DC power measurements on the Atomic Pi and Raspberry Pi 4; they are very similar, at about 5.8W while running a load test. The Atomic Pi actually draws a bit less power at idle: 4.4W vs 5.0W.
2 Likes
Probably worth mentioning, that with the default parameters (no explicit overhead specified, IPv4 TCP, no IP or TCP options) a gross shaper rate of 800 Mbps will result in a theoretical maximum of:
800 * (1500-20-20)/(1514) = 771.5 Mbps
So for the uni-directional tests both the AAEON and the RPi4B do quite well. In the bidirectional case the achievable goodput is lower, as there is the unavoidable reverse ACK traffic that is not accounted for. Since the exact parameters of ACK generation are not set in stone it is hard to estimate the rate of the ACK traffic. Maybe looking at cake"s traffic counters before and after a speed-test might help
1 Like
Could you specify the exact model, please? Not all USB3 dongles are of equal quality so documenting which work reliably would be helpful.
It was the "ABLEWE 3-Port USB 3.0 Hub with RJ45 10/100/1000 Gigabit Ethernet Adapter" purchased from Amazon.
Noob question. What is everyones preference for using SQM? I have a raspberry pi 4 on 21.02 using vlans for wan and lan and gigabit coax internet. From my understanding enabling sqm is a tradeoff where it will decrease the speedtest number but improve the overall quality for e.g. video calls. Also, from my testing I noticed software offloading helps slightly and hardware offloading makes no difference since it isn't supported.
On most fixed internet connections you give up a few percent of speed with a well tuned SQM.
If you have a variable speed connection it's a different story (like LTE modem or WISP)
1 Like
Wondering what settings were used for "SQM" on the new run? As well as the type of qdisc?
Going to Cake, with various features on might add some load to the resulting task, and may influence speeds somewhat negatively. But what might be more interesting is the ack-filter feature, which should reduce the ack traffic load, and thereby be a positive speed benefit.
Also, most folks using SQM would probably end up using Cake and some of the features, so that would be closer to "real world", I would think..
With traditional TCP Reno we expect one ACK packet per two full MSS-sized packets, which given typical MSS around 1460 Bytes results in ACKs taking ~1/40 of the datarate, and if either side uses GSO or GRO the ACK rate will ne even lower, so there is not much throughput to be gained by ACK filtering.... One advantage however is that on bursty links like DOCSIS and WiFi ACKs might bunch up and a whole train of ACKs will be released at once, which does not help very much as it screws up the ACK clocking a bit.... (out of such a back to back train of ACKs for the same connection only the latest is actually relevant because ACKs are accumulative, so all earlier ACKs in such are train are wasted).
1 Like
Not sure what you mean by "gigabit coax" internet, you mean DOCSIS cable? Some of the below settings will vary if you don't have that.
Read thru the SQM docs on the main openwrt.org site, all three sections.
I have DOCSIS cable internet, 940/35mbit.
Personally, I've been using something like the below:
Speeds:
900000
30000
Cake
layer_cake.qos
Advanced settings: defaults
"Dangerous" settings:
ingress - nat dual-dsthost ingress mpu 64
egress - nat dual-srchost ack-filter mpu 64
Link Layer Adaptation:
ethernet with overhead
22 (18 for DOCSIS, plus 4 for VLAN possibly needed someday?)
Again, these are optimal for my Cable modem based interenet, other types will need other settings, probably just in the Speeds and Link Layer Adaptation sections, see the docs.
Mmmh, mpu 64 (note the space between the keyword and the value) should continue to work, but if it does not, could you post the output of logread | grep -e SQM which should show what went wrong.
Arrgh... OK, good catch, I sure didn't catch it! Embarrassed now...
Been long enough since I edited those lines, when I updated from 19.07.8 to 21.02.0 I managed to put that in wrong (mpu64 not mpu 64) and thought it was supposed to be that way. D'oh. Nope, it works fine at the end of the "dangerous" section lines, though you could do it either way. I'll edit the post above, to not confuse others...
1 Like